AI service and microservice hybrid orchestration method and device, equipment and storage medium

Abstract

An AI service and micro-service hybrid arrangement method, device and equipment and storage medium, the method comprises: determining the required AI service instance number of each service graph and the instance number of each type of micro-service in each request chain under each service graph based on the idea of balancing service delay and deployment cost; establishing the arrangement process as a Markov decision process, introducing a double time scale concept, and arranging each request sub-chain in each sub-service graph in the system according to the order based on the reinforcement learning agent, and constantly updating the edge node state value in real time, deploying the AI service instance corresponding to the sub-service graph based on the greedy strategy, and updating the request routing of each request; the training of the reinforcement learning agent is carried out through the collection of the decision trajectory of the arrangement strategy, the neural network parameters of the agent are constantly updated, and finally the globally optimal arrangement scheme is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of service computing, and in particular to an AI service and microservice hybrid orchestration method and device, equipment and a computer readable storage medium. BACKGROUND

[0002] In recent years, generative artificial intelligence (GAI) is developing rapidly, which provides artificial intelligence generated content (AIGC) for users through pre-trained base models (PFMs), giving rise to many new Internet applications, such as ChatGPT and DALL-E 2. However, the complex architecture of these intelligent applications poses great challenges for maintenance. To ensure user quality of service (Qos), microservice architecture and model as a service (MaaS) architecture provide an efficient solution. Microservice architecture can effectively handle different types of user requests by decomposing complex applications into multiple independent microservice instances, while reusing common service modules to improve system efficiency. MaaS architecture allows developers to provide pre-trained AI models as services to users, who can invoke these trained models through API interfaces. These service computing architectures bring more flexible solutions to complex new network applications.

[0003] To further improve QoS, many service providers currently sink services from the cloud to the edge to provide users with more low-latency response services. Unlike cloud data centers, these edge servers are usually geographically distributed and resource-constrained, and cannot load all services on a single edge node. Therefore, these edge servers need to communicate and cooperate with each other to provide services for various intelligent applications. However, these intelligent applications usually rely on the cooperative processing of multiple services, forming a complete service chain through sequential interaction and invocation of AI services between different microservices, in which key microservices and AI services are invoked by multiple request streams, forming a complex dependency service call graph. A large number of microservices and AI services call and communicate with each other, forming a complex intelligent system. These service instances are deployed on edge servers and run for a long time to provide services for a large number of users. Furthermore, different user requests can invoke multiple instances of a single service to cope with high concurrency scenarios. User request streams are dispatched to the request queues of multiple corresponding instances for queuing processing, and multiple instances exacerbate the complex dependencies between different services. To capture the complex dependencies of services under a large number of users and multiple instances, we establish a queuing network model for steady-state latency analysis.

[0004] In the massive user and distributed scenario, efficient mixed orchestration strategy of microservices and AI services can reduce the end-to-end latency of user requests and significantly improve the quality of service of users. However, due to the above problems, we will face some challenges in actual orchestration. In fact, the mixed orchestration strategy involves joint decision of service deployment and request routing, and the two decisions are tightly coupled. On the one hand, the response time of the request flow end-to-end is simultaneously dependent on the deployment and routing strategy; on the other hand, the service deployment and request routing strategy are interdependent. After deployment, if the routing is not determined, the deployment strategy cannot be judged, and if the deployment strategy is not determined, the routing cannot be performed. Due to the tight coupling between instance deployment and request routing, the decision complexity of the orchestration strategy is increased. For instance deployment, in addition to deciding the deployment location, the number of instances required needs to be determined to balance the interests of service providers and users and achieve maximum utility. For users, the lower the end-to-end latency, the better the Qos, and for service providers, excessive requirements for the minimum response time of requests will lead to the deployment of a large number of instances, resulting in huge resource overhead. In addition, in edge computing, user requests and network computing environments are distributed, and multiple calling paths are generated in the process of user requests accessing and calling service instances. Specifically, each service may deploy multiple instances on different servers for load balancing, and the request flow is dispatched to the request buffer queue of the corresponding service instance in different server clusters through request routing. The existence of request routing makes the request flow have multiple calling paths in the edge network, which further aggravates the complexity of service calling. Considering the above complexity, in order to improve the quality of service of users, it is necessary to study the joint optimization of service deployment and request routing of multi-instance queuing network of microservices and AI services in edge environment. SUMMARY

[0005] To solve the above technical problems, the application provides an AI service and microservice mixed orchestration method, device, equipment and computer readable storage medium.

[0006] In a first aspect, the embodiments of the application provide an AI service and microservice mixed orchestration method, which comprises:

[0007] For each sub-service graph , determine a corresponding number of microservice instances and an AI service instance , wherein the request service chain composed by calling the same AI service instance ;

[0008] Based on the processing capacity of the request arrival rate , corresponding to the sub-service graph and deployment cost determination The number of units to be deployed;

[0009] Request arrival rate based on the corresponding sub-service graph Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of units to be deployed;

[0010] based on The number of deployments to be made, using reinforcement learning agents Each The microservice instances are deployed one by one in sequence, and the temporary request routing is determined adaptively in a probabilistic routing manner to obtain All Temporary deployment and request results, To remove of ;

[0011] based on The number of waiting deployments, using greedy strategy for asynchronous deployment , and update according to the deployment results All Request routing, get the sub-service graph The arrangement results;

[0012] The decision trajectories generated by the reinforcement learning agent during the entire orchestration process are collected, and the agent training and neural network update are performed to obtain a fully trained agent.

[0013] In a second aspect, an embodiment of the present application provides an AI service and microservice hybrid orchestration device, the AI ​​service and microservice hybrid orchestration device comprising:

[0014] Determine the module for each sub-service graph ,Sure Corresponding microservice instances and an AI service instance ,in, By calling the same AI service instance Request service chain constitute;

[0015] Quantity determination module, used to determine the request arrival rate based on the sub-service graph 、 Processing power and deployment cost determination The number of pending deployments; based on the request arrival rate corresponding to the sub-service graph Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of units to be deployed;

[0016] Orchestration module for The number of deployments to be made, using reinforcement learning agents Each The microservice instances are deployed one by one in sequence, and the temporary request routing is determined adaptively in a probabilistic routing manner to obtain All Temporary deployment and request results, To remove of ;based on The number of waiting deployments, using greedy strategy for asynchronous deployment , and update according to the deployment results All Request routing, get the sub-service graph The arrangement results;

[0017] The iteration module is used to collect the decision trajectories generated by the reinforcement learning agent during the entire orchestration process, perform agent training and neural network updates, and obtain a fully trained agent.

[0018] In a third aspect, an embodiment of the present application provides an AI service and microservice hybrid orchestration device, which includes a processor, a memory, and an AI service and microservice hybrid orchestration program stored on the memory and executable by the processor, wherein when the AI ​​service and microservice hybrid orchestration program is executed by the processor, the steps of the AI ​​service and microservice hybrid orchestration method described in the first aspect are implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a hybrid orchestration program for AI services and microservices is stored. When the hybrid orchestration program for AI services and microservices is executed by a processor, the steps of the hybrid orchestration method for AI services and microservices as described in the first aspect are implemented.

[0020] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0021] This application achieves efficient orchestration of AI services and microservices by comprehensively analyzing user requests and edge node status. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A flowchart of an embodiment of the AI service and microservice hybrid orchestration method of the present application;

[0023] Figure 2 A functional module diagram of an embodiment of the AI service and microservice hybrid orchestration apparatus of the present application;

[0024] Figure 3 A hardware structure diagram of the AI service and microservice hybrid orchestration device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0025] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the drawings.

[0027] In the first aspect, the embodiments of the present application provide an AI service and microservice hybrid orchestration method.

[0028] In the intelligent edge computing scenario, there are various user intelligent application program requests, and service orchestration is to deploy and make joint decisions on request routing for various service instances required by users. For example, in the intelligent edge computing scenario, there may be multiple types of intelligent application programs, such as intelligent customer service, chat robots, AI image generation, etc. These programs need multiple different AI models and microservices to communicate and call each other to complete the service together, so we need to deploy service instantiation according to user requests, and determine request routing according to the calling relationship. The goal of our service orchestration is to trade off user service quality and operator cost, i.e., to minimize the weighted value of end-to-end response latency and deployment cost of the user.

[0029] Specifically, the intelligent edge computing scenario usually includes multiple distributed server nodes, each node contains a certain amount of computing resources, which is used for the instantiation deployment of various services, and the edge nodes are cooperated through communication links. In addition, there are edge user requests, considering the geographical distribution characteristics of user requests, users access the network through the nearest edge node, and multiple micro-service instances and an AI service instance are jointly completed in chain calling order, and a batch of requests calling the same AI service constitutes a sub-service calling graph, and we model a batch of user request traffic as a request flow subject to Poisson distribution. The system uploads the current request in the service area and the state data of each edge server to the central control system through the implementation of monitoring technology, and the central system arranges the services according to the edge node state and user request. The arrangement process needs to solve the following problems:

[0030] 1. Based on the processing capacity and deployment cost of each service, determine the required instance number of each service in the current system.

[0031] 2. When arranging all request chains in each sub-service graph, the arrangement process involves the joint optimization of service deployment and request routing.

[0032] 3. Ensure that the service instances deployed on any edge node do not violate the resource constraints when arranging, and each service instance meets the basic service rate.

[0033] 4. On the basis of meeting the above constraints, minimize the weighted sum of the total service time of all requests and the deployment cost.

[0034] The embodiment of the application provides a service arrangement algorithm based on reinforcement learning, which realizes efficient arrangement of services by comprehensively analyzing user requests and edge node states through a central controller. In the implementation process of the application, the central controller first collects the relevant attribute information of each edge server in the current system, including the number of remaining computing resources, the deployed instance information; collect user request information, these information includes request constitutes user access node, service chain constitutes, request arrival rate and so on. Through comprehensive collection of these information, the central controller can formulate more comprehensive global optimization arrangement strategy.

[0035] As Figure 1As shown, in the orchestration start stage, the central controller determines the required instance number of each type of service according to the service instance number calculation method, that is, the required AI service instance number of each sub-service graph and the number of each type of microservice called in all request sub-chains in the sub-service graph. In the orchestration strategy execution process, the central controller deploys the microservice instances of each request sub-chain in the order of service chain calling based on the reinforcement learning agent for the current required orchestration sub-service graph, until all the microservice instances of the current sub-chain are deployed, the central controller will adaptively determine the request routing according to the deployment result. Specifically, to balance the load of each deployed microservice instance, the request flow will be evenly distributed to the current deployed instance number to achieve load balancing of the microservice instance. After each request sub-chain is orchestrated, the edge node state will be updated, the central controller will constantly monitor and read the node state, and the next request sub-chain will be orchestrated, until the orchestration of all request sub-chains in the current sub-service graph is completed, that is, the microservice deployment position of all request sub-chains and the request routing method of each request sub-chain are determined.

[0036] Next, the AI service that has not been deployed in the current sub-service graph is orchestrated. Specifically, we count the microservice instances located at the adjacent calling position of the current AI service, and then iteratively select edge nodes using the greedy strategy. Under the condition of not exceeding the resource constraints of the edge node, the edge node with the minimum additional communication time with the adjacent microservice is selected for the actual deployment of the AI service instance. After the deployment of the AI service instance is completed, the request routing of each request in the sub-service graph is updated according to the deployment position.

[0037] The above process will continue to circulate until all the sub-service graphs in the system are orchestrated. After the entire service orchestration operation is completed, the central controller will collect the decision trajectory samples generated during the entire service orchestration process to obtain training experience, and the reinforcement learning agent in the central controller will train the model according to the obtained experience to enhance the decision-making ability of the agent.

[0038] In an embodiment, the AI service and microservice hybrid orchestration method comprises:

[0039] Step S10, for each sub-service graph , determine a plurality of microservice instances and an AI service instance corresponding to each sub-service graph , wherein the request service chain calling the same AI service instance ;

[0040] In this embodiment, the edge server contains multiple computing resources and is geographically distributed and capable of communicating with each other:

[0041] For a geographically distributed multi-edge server system, an undirected graph The system contains multiple edge nodes, denoted as where represents the i-th edge node. While edge nodes contain a large amount of computing storage resources, including CPU, memory and GPU resources, denoted as , , , These resources are available for AI models and microservices to be instantiated and deployed in servers, denoted as Each edge node has a set of attributes, denoted as .

[0042] Edge nodes transmit data through the network to collaborate on computing tasks. We denote the set of links between any two edge nodes as The network environment between edge nodes can be heterogeneous due to geographical environment, infrastructure, or different communication protocols between different operators, resulting in different communication qualities of links between edge nodes. We denote the transmission bandwidth between two edge nodes and as , and the transmission bandwidth of the entire system is:

[0043]

[0044] We denote the physical distance between two edge nodes as , which is calculated by the Euclidean distance formula , and the physical distance of the entire system is:

[0045]

[0046] In particular, the communication between service instances within the same edge node will be completed through internal high-speed optical fiber communication, so , and since the location coordinates of each node are fixed, we have . Each link has a set of attributes, denoted as .

[0047] In the microservice architecture, due to the reusability of service instances, the entire intelligent application can be modeled as a directed acyclic graph (DAG), denoted as The intelligent application can support different functionalities, and a functionality can be decoupled into several service instances with data dependency relationships. These service instances form a service function chain by calling adjacent service instances through data communication. We denote the set of all microservices as and the set of all service instances as denotes the set of all AI services, for convenience, we use service in the following denotes the set of all AI services, for convenience, we use service in the following .

[0048] Each user request flow can be completed by a corresponding function, and we use a chain structure to represent the service function chain, denoted as We define that each request is completed by multiple microservice instances and an AI service instance. The AI service instance is usually completed in batch processing form, and an AI service instance may be called by multiple requests. The requests calling the same AI service instance have similarity, and there are several key reusable microservice instances, so the service graph forms several sub-service call graphs, each of which is composed of request service chains calling the same AI service instance, denoted as

[0049]

[0050] Further, the service graph can be decomposed into multiple sub-service graphs calling different AI service instances, that is

[0051]

[0052] In order to be more close to the real network environment, we model the traffic of each subgraph of a batch of user requests as a request flow with Poisson distribution of arrival rate, denoted as , and all user arrival rates . At the same time, considering the geographical distribution characteristics of user requests, the geographical location of users is represented by , and users access the network through the nearest edge node, the access node of the request flow can be represented as . In addition, since the user request needs to go through the inference calculation of the AI service instance to output the content, the user input, also known as the prompt word, is represented as , and the AI model iteratively calculates the content, and the expected iteration number is represented as . Therefore, each user request flow can be represented by a five-tuple .

[0053] Step S20, based on the processing capacity , of the request arrival rate corresponding to the sub-service graph and the deployment cost to determine the number of to-be-deployed ;

[0054] Step S30, based on the request arrival rate Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of units to be deployed;

[0055] In this embodiment, step S20 includes: based on the request arrival rate corresponding to the sub-service graph 、 Processing power And the deployment cost construction function , the function by The number of to-be-deployed The independent variable corresponding to the minimum function value is The number of pending deployments.

[0056] Step S30 includes: based on the request arrival rate corresponding to the sub-service graph Each microservice instance in the several microservice instances Processing power And the deployment cost construction function , the function by The number of to-be-deployed The independent variable corresponding to the minimum function value is The number of pending deployments.

[0057] Specifically:

[0058] The processing capacity of a microservice instance is expressed as , which can be determined by the amount of processing required Divide by CPU computing power (That is, the number of calculations that the CPU can perform per unit time, which can be obtained by monitoring the server performance) is obtained, that is, .

[0059] The processing power of AI service instances It is expressed as the number of batch requests that can be processed per unit time, which is mainly determined by the model architecture. , users in a batch request prompt word input , expected number of iterations And the computing power of the model after instantiation , which is determined by the number of FLOPs that can be calculated per unit time. Generally, we use a function Indicates the model's processing capability for a request:

[0060]

[0061] We take a typical decoder-only architecture large language model as an example, a complete inference process can be divided into two stages: initialization stage and autoregressive stage. In the initialization stage, the user's prompt word is input into the model, and all transformer layers in it are calculated in turn to generate the first output word. In the autoregressive stage, the model uses the newly generated token as input and re-enters the model for iterative calculation. Each iteration of the model generates a new token until the generated sequence reaches the expected number of iterations. For the common large model GPT-3 architecture, which is stacked by multiple transformer blocks, each module is composed of a self-attention module and a feedforward neural network, and the model architecture can be abstractly represented as , and represent the hidden layer dimensions of the self-attention module and the feedforward neural network respectively, represents the number of transformer block layers. In the initialization stage, for a prompt token with a length of , the FLOPs required for its calculation on all layers of the model can be represented as:

[0062]

[0063] In the autoregressive stage, the newly generated output word is re-input into the large model until the iteration reaches the expected number of iterations to generate the complete output, so the calculation amount of the autoregressive stage can be represented as:

[0064]

[0065] For a single request, the required calculation amount is the sum of the calculation amount of the initialization stage and the autoregressive stage, that is, , so the total calculation amount in a batch is , and the processing capacity of the AI service instance is represented as the number of requests that can be processed per unit time, that is, the ratio of the model's computing capacity after instantiation to the calculation amount required by a batch, which can be represented as:

[0066]

[0067] The deployment of each service instance will incur a certain cost, including the deployment cost of all request service chain microservice instances and AI service instances. Their instantiation deployment will consume a certain amount of computing resources and generate corresponding fees. Since the edge nodes are managed by the same cloud provider, the cost of using resources of different edge nodes is the same. Let and represent the total number of AI service instances and microservice instances deployed in all edge nodes, that is, , The deployment cost of a microservice is determined by the consumption of computing and storage resources, and the total deployment cost of microservices can be represented as:

[0068]

[0069] where the unit cost of a microservice instance is determined by the CPU and memory resources it consumes, specifically , , and are the weights of resource costs, and represent the CPU and memory resources consumed by the microservice . is the number of microservice instances deployed in all edge nodes .

[0070] Similarly, the unit cost of an AI service instance depends on the usage of CPU, memory, and GPU resources, specifically , , and are the weight coefficients of CPU, memory, and GPU resources. The total deployment cost of an AI service instance can be represented as:

[0071]

[0072] is the number of AI service instances deployed in all edge nodes .

[0073] Each edge node has available resource capacity, including CPU resources , storage resources , and GPU resources . The microservice instances and AI service instances deployed on the edge node must ensure that they do not exceed the resource capacity. The resource limit numbers on the edge node are , , and . The number of microservice instances and AI service instances deployed on the edge node is and , respectively. Therefore, the number of instances on each edge node must satisfy the following resource number limits:

[0074] ​

[0075]

[0076]

[0077] Total cost of the system The total deployment cost of the microservice instance and the AI ​​service instance is as follows:

[0078]

[0079] For the service delay of microservices, due to the reusability of microservices, edge nodes Multiple microservice instances on the same node can collaborate to call multiple requests, and the corresponding nodes Microservices on The workload can be represented as all the microservices The total request rate of all call paths of the request flow deployed on the edge node is expressed as ,Since the user request flow is continuous and an instance can be regarded as a service node to complete the user's request, we use the multi-instance queuing theory M\M\C to analyze the service delay of microservices in the queuing network. The processing capacity of the microservice instance is expressed as ,node Microservices The number of instances is , first, if the request arrival rate Beyond edge nodes Microservice instance Total processing capacity , will lead to infinitely long queues, thus seriously reducing the quality of network service. Therefore, service intensity Should be less than 1. Edge node Microservice instance on The service intensity is defined as:

[0080]

[0081] According to the little formula, the edge node Microservice instance on The queuing delay can be expressed as:

[0082]

[0083] In the formula Expressed as:

[0084]

[0085] The total service time is equal to the queuing time plus the processing time, which is expressed as follows:

[0086]

[0087] For each microservice , we need to weigh the user in microservices Service latency and deploying microservices on The service cost brought by the arrival rate is Each microservice in the service chain The number of instances we need to weigh is as follows

[0088]

[0089] The weight coefficient for balancing user latency and service cost.

[0090] For the service performance of AI service for a batch of requests, this application adopts M / M / 1 queuing model and arrival rate splitting mechanism, that is, the requests are evenly distributed to all AI service instances on the edge nodes according to the arrival rate of each batch of requests. Indicates the number of all AI service instances in all current edge nodes, and the arrival rate of each instance is It can be expressed by the following formula:

[0091]

[0092] The average service delay of AI service can be expressed as:

[0093]

[0094] For each sub-service graph The corresponding AI service instance to be called , we need to balance the following formula:

[0095]

[0096] Next, we iteratively determine each request subchain to remove the AI ​​service based on the idea of ​​gradient descent. The required number of instances of all microservices , each Required number of AI service instances , the specific steps are: set a minimum initial solution that meets the basic service rate, for and , respectively set as and , and then continue to increase the number of instances until it reaches a certain value. Further increasing the number of instances causes the optimization objective to increase instead of decrease, that is, it satisfies the following formula:

[0097]

[0098]

[0099] The number of instances at this time is the local minimum and also the expected instance value.

[0100] Step S40, based on The number of deployments to be made, using reinforcement learning agents Each The microservice instances are deployed one by one in sequence, and the temporary request routing is determined adaptively in a probabilistic routing manner to obtain All Temporary deployment and request results, To remove of ;

[0101] In this embodiment, the decision-making process of the reinforcement learning agent can be modeled by a Markov decision process (MDP). Specifically, the agent interacts with the environment continuously, and the agent observes the state of the environment at the current moment. , and based on the state Decision-making through the policy network, output action , the environment changes state according to the decision of the agent, becomes The agent will receive an immediate reward after performing an action. To evaluate the quality of the strategy. The continuous interaction between the agent and the environment forms a trajectory sequence, that is, The purpose of the intelligent agent is to make decisions at every moment to maximize long-term benefits. Among them, the design of the MDP quadruple is as follows:

[0102] :

[0103] At the moment , the state observed by the agent Expressed as:

[0104]

[0105] in, Represents an edge node The number of remaining resources, Represents the number of microservice instances deployed on each edge node at the current moment. Contains the resource requirements of the microservice instance to be deployed, the micro-instance number, and information about adjacent micro-instances in the request service chain, including the instance number and deployment location;

[0106] :

[0107] At the moment , the action of the agent is to select the edge node for deployment of the microservice instance at the current deployment moment, and the action space is represented as a one-dimensional discrete vector: ,in Is a binary variable, indicating whether the microservice instance Deploy to edge nodes Above, the constraints are met: , that is, each microservice instance can only be deployed to one edge node, and the agent outputs the deployment probability distribution of all edge nodes ,in Indicates that the microservice instance Deploy to edge nodes The probability of the actual deployment node is determined by sampling the probability distribution. After the instance deployment is completed, the agent performs request routing and determines the current request path The routing probability value ;

[0108] :

[0109] The agent is making a decision After that, the agent will receive an immediate reward. The goal of the agent is to optimize the deployment strategy as much as possible to obtain the maximum cumulative reward. As the request flow increases, the agent's decision-making process also increases. To alleviate the reward sparseness problem caused by long-term decision-making, we give immediate rewards based on the cumulative delay of the request flow at the current deployment moment. That is, the agent is making decisions. You will receive an instant reward , Indicates sequential deployment No. The rewards received by microservice instances are The cumulative request flow latency and cost after orchestration are determined and expressed as:

[0110]

[0111] in, , , which indicates the total number of microservice instances that need to be deployed for the current request flow. express length, The expression is as follows:

[0112]

[0113] wherein, denotes the average response latency of the current deployed request flow, is the minimum average response latency of the current deployment time in the historical training round;

[0114]

[0115] The state transition function describes the change process of the state after the agent performs the action. Specifically, the action of the agent is to deploy the microservice instance to a specific edge node, and the state value is updated according to the deployment of the instance on the edge node and the corresponding resource consumption. The new state after the action is represented by the following update:

[0116]

[0117] wherein the function contains the update of the remaining resources of the edge node, the information update of the number of deployed microservice instances, and the information update of the microservice instance to be deployed.

[0118] Step S50, based on the number of to-be-deployed , the microservice instances are deployed asynchronously using the greedy strategy , and the request routing of all in is updated based on the deployment result to obtain the orchestration result of the sub-service graph ;

[0119] In this embodiment, the entire orchestration process is based on reinforcement learning to deploy microservice instances, and the request routing is determined in a probabilistic routing manner in an adaptive manner, and finally the AI service instance orchestration is completed in an asynchronous manner.

[0120] First, the sub-service call graph is split into multiple service chains, and then the calculated microservice instances are deployed for the sub-chain corresponding to the current request, and then the routing path is calculated through probabilistic routing to obtain the temporary latency and deployment cost. After completing the orchestration of all microservice instances in the current sub-service graph, the AI service instance is deployed based on the greedy strategy, and the complete routing path and complete request latency of all request flows in the sub-service graph are updated.

[0121] Wherein, the orchestration strategy constitutes two time scales, i.e. , each large time period contains multiple small time periods ,​ wherein . Each large time scale is deployed for all instances, while each small time scale is deployed for request sub-chain in , step by step deployment of micro-service instances and complete deployment. Finally, under the large time scale, the deployment of AI service instances that have not been deployed and request routing updates are completed.

[0122] Specifically, for the request flow in the service graph , the deployment of micro-service instances is determined by the order of , and in each small time scale , the RL Agent calculates the edge node selected for the current micro-service instance based on the policy network under the current deployment time slot. According to the number of all micro-service instances calculated, they are deployed in order to the edge node. At the same time, in order to prevent the deployment strategy from violating the resource constraints, an additional action mask is introduced to shield the possible illegal actions of the agent under the current time slot. For micro-service instances , the reinforcement learning agent calculates the policy distribution based on the policy network , and the additional action mask prevents the deployment operation from violating the edge node resource limit, which is obtained according to the remaining resources of the edge node under the current time slot, that is ; for each edge node , its mask value is defined as follows:

[0123]

[0124] According to the mask, update the policy distribution:

[0125]

[0126] Sample the new policy distribution to get the deployment strategy , update the remaining resources of the corresponding edge node, and perform state transition .

[0127] Next, when instance deployment is completed, the agent performs request routing to determine the routing probability value of the current request path , which is specifically:

[0128] According to the micro-service instance deployment result in the current , calculate the temporary routing path in the routing probability way:

[0129]

[0130] One of the call paths is represented as And the call path probability value The calculation is actually the product of the forward probability of each edge node on the path, and the probability of selecting the next edge node will be determined by the number of corresponding service instances deployed on the edge node. Similarly, the access shunting probability of the request Similarly, follow the strategy, that is:

[0131]

[0132]

[0133]

[0134] Where Indicates The number of microservice instances deployed at the edge node , The call path Corresponding shunting arrival rate is calculated by

[0135] As mentioned above, the orchestration strategy consists of two time scales, the current time scale Corresponding After all the microservice instances in the Arrival rate matrix :

[0136]

[0137]

[0138] Based on the current deployed results, that is, the instance number matrix and the arrival rate matrix of the edge node environment, the service time of each microservice on the call path is calculated And update the latency of the request flow deployed before; The transmission bandwidth of the virtual link between the two edge nodes Is , and the transmission data volume under the unit arrival rate between the two types of services Is For microservice instances Located on the edge node , and the corresponding arrival rate is Its transmission delay is represented as:

[0139]

[0140] The propagation delay depends on the physical distance between the two edge nodes ​​The propagation speed of the signal in the air medium is expressed as The propagation delay is expressed as:

[0141]

[0142] The communication time of adjacent microservice calls is given by the following formula:

[0143]

[0144] If the microservice call location is adjacent to the AI service, the two types of microservices do not directly communicate, so the communication delay at this point is temporarily 0, that is, , ; The temporary delay of The delay of all call paths is weighted to obtain:

[0145]

[0146] After the current orchestration is completed, the reward is calculated, and the deployment trajectory of the microservice is updated: , , where , and and are the calculation results of the agent's policy network and value network at the current time, and the current small time scale ends and enters the next orchestration.

[0147] Next, after completing all orchestration in the current sub-service graph , asynchronously deploy the AI service instances that have not been deployed. Specifically, count the microservice instances located at the adjacent call location of the AI service instance in the current sub-service graph , and then use a greedy strategy to iteratively select edge nodes, calculate the additional service time brought by deploying the AI service instance at the selected edge node, the communication time of the adjacent microservice instance call, and the additional cost generated by the deployment, where the additional service time , the additional communication time , and the additional deployment cost are given by the following formulas:

[0148]

[0149]

[0150]

[0151] where , ;

[0152] Since the extra service time is not affected by the deployment strategy, the actual deployment of the AI service instance is selected to make the sum of the extra communication time and the extra cost minimum, that is:

[0153]

[0154] At the current time scale , After the deployment of the AI service instance in the current time scale , the newly deployed node set is represented as , and all the routes in are updated, one of which is the call path , which is updated as: The corresponding routing probability is also updated, satisfying The arrival rate of the shunt request of each call path is also updated, satisfying The call path set is updated as , satisfying , and and respectively represent the number of call paths before and after updating, and the end-to-end delay is updated, that is, the service delay of each call path will increase the AI service delay and the communication time:

[0155]

[0156] The current large time scale ends, and the next sub-service graph arrangement is entered.

[0157] Step S60, collect the decision trajectory generated by the reinforcement learning agent in the entire arrangement process, and perform agent training and neural network updating to obtain a completely trained agent.

[0158] In this embodiment, the experience sampled from is used to calculate the advantage function at each time according to the GAE formula, where is a discount factor, is a bias-variance trade-off coefficient, and then the target value function at each time is calculated:

[0159]

[0160]

[0161] The policy loss and the value loss are calculated:

[0162]

[0163]

[0164] wherein denotes the ratio of the new policy to the old policy, , limiting it to the range ensures that the target policy update does not deviate too much from the sampled policy, updating the parameters with gradient ascent and gradient descent, respectively:

[0165]

[0166]

[0167] The agent is trained continuously with decisions until convergence.

[0168] The embodiment of the application provides a hybrid service orchestration method of AI services and microservices based on the Maas paradigm in an edge computing scenario, aiming to solve the complexity problem caused by the mutual coupling of service deployment and request routing decision involved in heterogeneous service orchestration. For the edge computing scenario, the method designs a multi-service call graph orchestration strategy based on a deep reinforcement learning method, so as to realize weighted loss minimization of end-to-end response time delay and service deployment cost. The method determines the number of required AI service instances of each service graph and the number of each type of microservice instance in each request chain under each service graph based on the idea of balancing service time delay and deployment cost; the orchestration process is established as a Markov decision process, the concept of double time scale is introduced, the reinforcement learning agent performs orchestration of each request sub-chain in each sub-service graph in the system in sequence and continuously updates the edge node state value in real time, the AI service instance corresponding to the sub-service graph is deployed asynchronously based on the greedy strategy, and the request routing of each request is updated; the training of the reinforcement learning agent is performed through the collection of the decision trajectory of the orchestration strategy, the neural network parameters of the agent are continuously updated, and finally the globally optimal orchestration scheme is realized.

[0169] In a second aspect, the embodiment of the application further provides an AI service and microservice hybrid orchestration device.

[0170] In an embodiment, refer to Figure 2 , Figure 2 is a functional module schematic diagram of the AI service and microservice hybrid orchestration device of the embodiment of the application. As shown in Figure 2 , the AI service and microservice hybrid orchestration device comprises:

[0171] The determination module 10 is configured to determine, for each sub-service graph , a plurality of types of microservice instances and an AI service instance corresponding to the sub-service graph, wherein, the same AI service instance is called request service chain configuration;

[0172] The number determination module 20 is configured to determine the to-be-deployed number based on the request arrival rate corresponding to the sub-service graph, the processing capacity of each of the plurality of micro-service instances, and the deployment cost. 、 The processing capacity of each of the plurality of micro-service instances and the deployment cost to determine the to-be-deployed number. The request arrival rate corresponding to the sub-service graph , the processing capacity of each of the plurality of micro-service instances and the deployment cost to determine the to-be-deployed number. The orchestration module 30 is configured to, based on the to-be-deployed number, sequentially deploy each of the micro-service instances in the sub-service graph by using the reinforcement learning agent, and adaptively determine a temporary request routing in a probabilistic routing manner to obtain temporary deployment and request results of all the micro-service instances in the sub-service graph.

[0173] The orchestration module 30 is configured to, based on the to-be-deployed number, sequentially deploy each of the micro-service instances in the sub-service graph by using the reinforcement learning agent, and adaptively determine a temporary request routing in a probabilistic routing manner to obtain temporary deployment and request results of all the micro-service instances in the sub-service graph. The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph The orchestration result of the sub-service graph

[0174] The iteration module 40 is configured to collect decision trajectories generated by the reinforcement learning agent in the entire orchestration process, perform agent training and neural network updating, and obtain a completely trained agent.

[0175] The functions of each module in the AI service and micro-service hybrid orchestration apparatus correspond to the steps in the AI service and micro-service hybrid orchestration method, and the functions and implementation processes will not be repeated here.

[0176] In a third aspect, the embodiments of the present application provide an AI service and micro-service hybrid orchestration device. The AI service and micro-service hybrid orchestration device can be a personal computer (PC), a notebook computer, a server, or other devices with data processing functions.

[0177] Referring to Figure 3 , Figure 3 ​Schematic diagram of the hardware structure of the AI ​​service and microservice hybrid orchestration device involved in the embodiment of the present application. In the embodiment of the present application, the AI ​​service and microservice hybrid orchestration device may include a processor, a memory, a communication interface, and a communication bus.

[0178] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0179] Communication interfaces include input / output (I / O), physical, and logical interfaces, which interconnect components within the AI ​​and microservices hybrid orchestration device, as well as interfaces that connect the AI ​​and microservices hybrid orchestration device to other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet, fiber, and ATM interfaces; user devices can include displays and keyboards.

[0180] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0181] The processor may be a general-purpose processor that can invoke the AI ​​service and microservice hybrid orchestration program stored in the memory and execute the AI ​​service and microservice hybrid orchestration method provided in the embodiments of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the AI ​​service and microservice hybrid orchestration program is invoked can be referenced in the various embodiments of the AI ​​service and microservice hybrid orchestration method of the present application and will not be further described here.

[0182] Those skilled in the art will understand that Figure 3 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0183] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0184] The computer readable storage medium stores an AI service and microservice hybrid orchestration program. When the AI service and microservice hybrid orchestration program is executed by a processor, the steps of the AI service and microservice hybrid orchestration method described above are implemented.

[0185] The method implemented when the AI service and microservice hybrid orchestration program is executed can refer to the embodiments of the AI service and microservice hybrid orchestration method, which will not be described here.

[0186] It should be noted that the above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0187] The terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above drawings are intended to cover the non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or device. The terms "first", "second" and "third" and the like descriptions are used to distinguish different objects, and do not represent the order or limit the types of "first", "second" and "third".

[0188] In the description of the embodiments of the present application, "exemplary", "for example" or "for instance" is used to represent an example, illustration or description. Any embodiment or design scheme described as "exemplary", "for example" or "for instance" in the embodiments of the present application should not be interpreted as more preferred or more advantageous than other embodiments or design schemes. In fact, the words "exemplary", "for example" or "for instance" are intended to present the relevant concept in a specific way.

[0189] In the description of the embodiments of the present application, unless otherwise specified, " / " represents or, for example, A / B can represent A or B; "and / or" in the text only describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent three cases of A alone, A and B together, and B alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0190] In some of the processes described in this specification, the order of operations or steps can be modified. Specifically, the serial order of any two consecutive steps carried out according to the processes described in this specification can be changed so that these two steps can be carried out in parallel or simultaneously, or the order of these two steps can be reversed.

[0191] Those skilled in the art can clearly understand the above-mentioned embodiment method from the description of the above embodiments, which can be realized by software and a necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disc) and includes a plurality of instructions for causing a terminal device to execute the methods described in the various embodiments of the present application.

[0192] The above is only a preferred embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A hybrid orchestration method for AI services and microservices, characterized in that: The hybrid orchestration method of AI services and microservices includes: For each sub-service graph ,Sure Corresponding microservice instances and an AI service instance ,in, By calling the same AI service instance Request service chain constitute; Request arrival rate based on the corresponding sub-service graph 、 Processing power and deployment cost determination The number of units to be deployed; Request arrival rate based on the corresponding sub-service graph Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of units to be deployed; based on The number of deployments to be made, using reinforcement learning agents Each The microservice instances are deployed one by one in sequence, and the temporary request routing is determined adaptively in a probabilistic routing manner to obtain All Temporary deployment and request results, To remove of ; based on The number of waiting deployments, using greedy strategy for asynchronous deployment , and update according to the deployment results All Request routing, get the sub-service graph The arrangement results; The decision trajectories generated by the reinforcement learning agent during the entire orchestration process are collected, and the agent training and neural network update are performed to obtain a fully trained agent.

2. The hybrid orchestration method of AI services and microservices according to claim 1, characterized in that: Request arrival rate based on the corresponding sub-service graph 、 Processing power and deployment cost determination Number of pending deployments: Request arrival rate based on the corresponding sub-service graph 、 Processing power And the deployment cost construction function , the function by The number of to-be-deployed The independent variable corresponding to the minimum function value is The number of units to be deployed; Request arrival rate based on the corresponding sub-service graph Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of deployments to be made includes: Request arrival rate based on the corresponding sub-service graph Each microservice instance in the several microservice instances Processing power And the deployment cost construction function , the function by The number of to-be-deployed The independent variable corresponding to the minimum function value is The number of pending deployments.

3. The hybrid orchestration method of AI services and microservices according to claim 1, characterized in that: The decision-making process of the reinforcement learning agent can be modeled by a Markov decision process (MDP). The design of the MDP quadruple is as follows: At the moment , the state observed by the agent Expressed as: in, Represents an edge node The number of remaining resources, Represents the number of microservice instances deployed on each edge node at the current moment. Contains the resource requirements of the microservice instance to be deployed, the micro-instance number, and information about adjacent micro-instances in the request service chain, including the instance number and deployment location; At the moment , the action of the agent is to select the edge node for deployment of the microservice instance at the current deployment moment, and the action space is represented as a one-dimensional discrete vector: ,in Is a binary variable, indicating whether the microservice instance Deploy to edge nodes Above, the constraints are met: , that is, each microservice instance can only be deployed to one edge node, and the agent outputs the deployment probability distribution of all edge nodes , by sampling the probability distribution, the actual deployment node is determined. After the instance deployment is completed, the agent performs request routing and determines the current request path The routing probability value ; The agent is making a decision You will receive an instant reward , Indicates sequential deployment No. The rewards received by microservice instances are The cumulative request flow latency and cost after orchestration are determined and expressed as: in, , which indicates the total number of microservice instances that need to be deployed for the current request flow. express length, The expression is as follows: in, Indicates the average response delay of the currently deployed request flow. The minimum average response delay at the current deployment time in the historical training rounds; The state transition function describes the state change process after the agent performs an action. Specifically, the action of the agent It is to deploy the microservice instance to a specific edge node. The status value is updated according to the deployment of the instance on the edge node and the corresponding resource consumption. New status This is indicated by the following update: The function This includes updates on the remaining resources of edge nodes, the number of deployed microservice instances, and the number of microservice instances to be deployed.

4. The hybrid orchestration method of AI services and microservices according to claim 3, characterized in that: Introduce additional action masks to mask possible illegal actions of the agent in the current time slot. , the reinforcement learning agent calculates the policy distribution based on the policy network as ,The additional action mask prevents the deployment operation from violating the ,resource limit of the edge node, which is obtained based on the number of ,remaining resources of the edge node in the current time slot, i.e. ; For each edge node , define its mask value as follows: Distribution according to mask update strategy: Sampling new strategy distribution to get deployment strategy , update the remaining resources of the corresponding edge node and perform state transfer .

5. The hybrid orchestration method of AI services and microservices according to claim 4, characterized in that: when After the instance deployment is completed, the agent performs request routing and determines the current request path The routing probability value include: According to the current The microservice instance deployment results in the calculation of temporary routing paths are as follows: One of the call paths is represented as , and the calling path probability value The calculation is actually the product of the forward probabilities of each edge node on the path. The probability of selecting the next edge node is determined by the number of corresponding service instances deployed on the edge node. The access diversion probability of the same request is Following the same strategy: in express At the edge node Deployed Number of microservice instances, Indicates the call path The corresponding diversion arrival rate is given by Calculated.

6. The hybrid orchestration method of AI services and microservices according to claim 5, characterized in that: The orchestration strategy consists of two time scales, large and small, namely , each large time period Contains multiple small time periods , ,in , within each large time scale All instances in the deployment are orchestrated, and each small time scale in Perform orchestration, deploy microservice instances in sequence and complete the orchestration, and finally complete the deployment of undeployed AI service instances and request routing updates on a large time scale. The current time scale Corresponding After all microservice instances in the are deployed, update the corresponding instance number matrix and the arrival rate matrix : Calculate the dwell time of each microservice on the call path based on the instance number matrix and arrival rate matrix of the edge node environment , and update the latency of the previously deployed request flow; the two edge nodes The transmission bandwidth of the virtual link between , two types of services The amount of data transmitted under the unit arrival rate is , for edge nodes Microservice instance , and the corresponding arrival rate is , its transmission delay is expressed as: The propagation delay depends on the physical distance between two edge nodes , the propagation speed of the signal in the air medium is expressed as c0, and the propagation delay is expressed as: The communication time between adjacent microservice calls is given by the following formula: If microservices and When the calling location is adjacent to the AI ​​service, the two types of microservices do not communicate directly, that is, , ; The temporary delay is caused by The latency weights of all call paths are: In the current After the orchestration is completed, the rewards are calculated and the deployment trajectory of the microservice is updated: ,in , and The calculation results of the strategy network and value network of the intelligent agent at the current moment are respectively. The current small time scale ends and enters the next arrangement.

7. The hybrid orchestration method of AI services and microservices according to claim 6, characterized in that: After completing the current sub-service graph All After orchestration, asynchronously deploy AI service instances that have not yet been deployed. Specifically, count the current sub-service graph The microservice instances located in the adjacent call location of the AI ​​service instance are then iterated using a greedy strategy to select edge nodes, calculate the additional service time after deploying the AI ​​service instance on the selected edge node, the communication time with the adjacent microservice instance call, and the additional cost of deployment, where the additional service time is , additional communication time , additional deployment costs They are given by the following formulas: ； Select the edge node that minimizes the sum of the additional communication time and the additional cost for the actual deployment of the AI ​​service instance, that is: In the current time scale Down, After the AI ​​service instance is deployed, its newly added deployment node set is represented as , update accordingly All Routing, one of the call paths Updated to: , the corresponding routing probability is also updated to meet , the corresponding diversion request arrival rate of each call path is also updated to meet , the call path collection is updated to ,satisfy , and Represents the number of call paths before and after the update, and updates accordingly End-to-end delay, that is, each The service delay will increase AI service delay and communication time: The current large time scale ends and the next sub-service graph orchestration begins.

8. A hybrid orchestration device for AI services and microservices, characterized in that: The AI ​​service and microservice hybrid orchestration device includes: Determine the module for each sub-service graph ,Sure Corresponding microservice instances and an AI service instance ,in, By calling the same AI service instance Request service chain constitute; Quantity determination module, used to determine the request arrival rate based on the sub-service graph 、 Processing power and deployment cost determination The number of pending deployments; based on the request arrival rate corresponding to the sub-service graph Each microservice instance in the several microservice instances Processing power and deployment cost determination The number of units to be deployed; Orchestration module for The number of deployments to be made, using reinforcement learning agents Each The microservice instances are deployed one by one in sequence, and the temporary request routing is determined adaptively in a probabilistic routing manner to obtain All Temporary deployment and request results, To remove of ;based on The number of waiting deployments, using greedy strategy for asynchronous deployment , and update according to the deployment results All Request routing, get the sub-service graph The arrangement results; The iteration module is used to collect the decision trajectories generated by the reinforcement learning agent during the entire orchestration process, perform agent training and neural network updates, and obtain a fully trained agent.

9. A hybrid orchestration device for AI services and microservices, characterized in that: The AI ​​service and microservice hybrid orchestration device includes a processor, a memory, and an AI service and microservice hybrid orchestration program stored on the memory and executable by the processor, wherein when the AI ​​service and microservice hybrid orchestration program is executed by the processor, the steps of the AI ​​service and microservice hybrid orchestration method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an AI service and microservice hybrid orchestration program, wherein when the AI ​​service and microservice hybrid orchestration program is executed by the processor, the steps of the AI ​​service and microservice hybrid orchestration method according to any one of claims 1 to 7 are implemented.

Images