A containerized microservice intelligent collaborative orchestration method and device

Abstract

The application provides a containerized microservice intelligent collaborative scheduling method and device, which comprises the following steps: determining a service completion time delay index of a microservice architecture system, determining a system load balancing index according to environment information of the microservice architecture system, and then arranging a training initialization scheduling model according to the indexes to obtain a microservice scheduling model; inputting request state information corresponding to a user service request obtained by the user into the microservice scheduling model for policy scheduling processing to obtain a scheduling policy; deploying the containerized microservice according to the scheduling policy, and running the deployed containerized microservice to process the user service request. Through the application, the technical problem that the service performance of the microservice architecture system is low and the management cost is increased due to the difficulty in balancing the service completion time delay and the system load balancing of the microservice architecture system when the existing technology uses the scalarization method to schedule the containerized microservice is solved.

Description

Technical Field

[0001] The present invention relates to the field of microservice technology, and in particular to a method and device for intelligent collaborative orchestration of containerized microservices. Background Art

[0002] With the rapid increase in the complexity of IoT applications, the more flexible microservices architecture is gradually replacing traditional monolithic architectures. Container technology, due to its lightweight, portable, fast startup, and secure isolation, has become the preferred choice for microservice deployment. However, containerized services require pulling and installing container images from an image repository upon startup, which incurs startup delays. In bandwidth-constrained edge computing, this startup delay can significantly impact service quality. Furthermore, in a microservices architecture, applications are decoupled into multiple small, autonomous microservices, which form a multi-chain directed acyclic graph structure through lightweight communication. Microservice completion must consider both execution order and parallel computation, leading to two optimization objectives for containerized microservices: service completion latency and system load balancing. While orchestration of containerized microservices can optimize these objectives—reducing completion latency or ensuring system load balancing—these two optimization objectives are contradictory. Considering only latency can lead to system load imbalance. Reducing service completion latency benefits service consumers, while ensuring system load balancing benefits service providers. This conflict can impact service performance and increase management costs.

[0003] In the existing technology, scalarization methods are generally used to convert multiple optimization objectives into a single objective for optimization, thereby designing an intelligent collaborative orchestration method for containerized microservices. However, when optimizing multiple conflicting optimization objectives at the same time, that is, when orchestrating containerized microservices, both application completion latency and system load balancing must be considered. The scalarization method has limitations and poor optimization effect, and cannot achieve the optimization effect of balancing service completion latency and system load balancing. Summary of the Invention

[0004] The present invention provides a method and device for intelligent collaborative orchestration of containerized microservices, which is used to solve the technical problem that when the existing technology uses a scalarization method to orchestrate containerized microservices, it is difficult to balance the two optimization indicators of service completion latency and system load balancing, resulting in poor service performance of the microservice architecture system and increased management costs.

[0005] The present invention provides a containerized microservice intelligent collaborative orchestration method and device, comprising the following steps:

[0006] Determine a service completion delay index of the microservice architecture system, and determine a system load balancing index of the microservice architecture system based on environmental information of the microservice architecture system, wherein the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request;

[0007] Based on the service completion delay indicator and the system load balancing indicator, the initialization orchestration model is orchestrated and trained to obtain a microservice orchestration model;

[0008] When the microservice architecture system obtains a user service request, it obtains request status information of the user service request, wherein the request status information includes a target microservice and a user preference feature. The target microservice is a containerized microservice to be deployed when the microservice architecture system processes the user service request. The user preference feature is used to represent the user's preference for the service completion delay indicator and the system load balancing indicator.

[0009] Inputting the request status information into the microservice orchestration model for strategy orchestration processing to obtain an orchestration strategy for the target microservice;

[0010] The containerized microservice is deployed according to the orchestration strategy, and the deployed containerized microservice is run through the microservice architecture system to process the user service request.

[0011] In some embodiments, the microservice architecture system includes a terminal layer, an edge layer, and a mirror layer. The terminal layer includes multiple containerized microservices obtained by splitting user service requests. The edge layer includes multiple edge nodes. The mirror layer includes multiple container images. The container images are used to deploy the containerized microservices to the edge nodes for execution. Determining the service completion delay indicator of the microservice architecture system includes:

[0012] When the containerized microservice is deployed to the edge node, determining, in the edge node, an image pulling time for the containerized microservice to pull a container image from the image layer;

[0013] After the containerized microservice pulls the container image, determining the service processing time of the containerized microservice by the edge node, and determining the data transmission time between the edge nodes;

[0014] Determine a service completion delay index of the microservice architecture system based on the image pulling time, the service processing time, and the data transmission time.

[0015] In some embodiments, the microservice system architecture includes an edge layer, the edge layer includes multiple edge nodes, the environmental information includes the remaining computing resources and remaining memory of each edge node, and determining the system load balancing index of the microservice architecture system based on the environmental information of the microservice architecture system includes:

[0016] For each of the edge nodes, determining a load computing resource according to the remaining computing resource, and determining a load memory according to the remaining memory;

[0017] Determining a first average value of a load computing resource of the edge node, and determining a second average value of a load memory of the edge node;

[0018] The total load balancing value of the edge nodes in the microservice architecture system is determined according to the first average value and the second average value, and the average of the total load balancing values ​​is used as the system load balancing indicator of the microservice architecture system.

[0019] In some embodiments, the initialization orchestration model includes a policy network and a judgment network. The training process of the initialization orchestration model is performed according to the training time step. The initialization orchestration model is orchestrated and trained based on the service completion delay index and the system load balancing index to obtain a microservice orchestration model, including:

[0020] Obtaining environmental status data, wherein the environmental status data includes a target service request and a preference vector, the preference vector being constructed based on a delay weight and a load weight, the delay weight representing a preference of the target service request for the service completion delay indicator, and the load weight representing a preference of the target service request for the system load balancing indicator;

[0021] At a current training time step, the environment state data is input into the policy network to generate action design data, and the evaluation network is called to predict the environment state data to generate an expected reward vector value for the action design data, wherein the action design data is used to represent an edge node deployment action, and the edge node deployment action is a deployment action of deploying the containerized microservice of the terminal layer to the edge node in the edge layer when the microservice architecture system processes the target service request. The current time step is any training time step in the orchestration training process;

[0022] Constructing a strategy loss value of the initialized orchestration model according to the action design data, and constructing a value function loss value of the initialized orchestration model according to the action design data and the expected reward vector value;

[0023] The sum of the strategy loss value and the value function loss value is used as the final loss value of the initialized orchestration model in the current training time step, and the initialized orchestration model is trained by the final loss value to obtain a microservice orchestration model.

[0024] In some embodiments, constructing the strategy loss value of the initialization orchestration model according to the action design data includes:

[0025] Obtaining the predicted probability of the policy network when generating action design data at the current training time step, and determining the policy amplitude value of the policy network between training time steps based on the predicted probability;

[0026] In the current training time step, the generalized advantage estimates of the service completion delay indicator and the system load balancing indicator are determined respectively;

[0027] Determining, based on the generalized advantage estimate and the preference vector included in the environmental state data, a delay advantage value of the policy network of the initialized orchestration model for the service completion delay indicator, and a load advantage value of the policy network for the system load balancing indicator;

[0028] Determining an advantage function value of the strategy network according to the delay advantage value and the load advantage value;

[0029] A strategy loss value of the initialized orchestration model is constructed based on the advantage function value and the strategy amplitude value.

[0030] In some embodiments, constructing the value function loss value of the initialized orchestration model according to the action design data and the expected reward vector value includes:

[0031] In the current training time step, the action design data is executed by the microservice architecture system, and a current experience trajectory of executing the action design data is recorded, wherein the current experience trajectory includes the action design data, the preference vector, and the expected reward vector value;

[0032] Determine, based on the current experience trajectory, a total revenue value of the microservice architecture system for a service completion latency indicator and a system load balancing indicator, and determine a discounted revenue value for the total revenue value;

[0033] Determining a cumulative reward vector value in a current training time step, and determining an average value of a difference between the cumulative reward vector value and the discounted benefit value;

[0034] A value function loss value of the initialized orchestration model is constructed based on the average value of the difference and the preference vector.

[0035] In some embodiments, determining the total benefit value of the microservice architecture system for the service completion delay indicator and the system load balancing indicator based on the current experience trajectory includes:

[0036] Obtain the reward vector values ​​for the service completion delay indicator and the system load balancing indicator;

[0037] determining a linear reward scalar value according to the preference vector included in the current experience trajectory and the reward vector value;

[0038] Determining a penalty value obtained by the microservice architecture system after executing the constrained deployment action when executing the action design data corresponding to the current experience trajectory;

[0039] The difference between the linear reward scalar value and the penalty value is used as the total benefit value of the microservice architecture system for the service completion delay index and the system load balancing index.

[0040] In some embodiments, after recording the experience trace of executing the action design data, the method further includes:

[0041] Storing the current experience trajectory in a historical experience trajectory library, wherein the historical experience trajectory library is used to store the historical experience trajectory recorded by the initialized orchestration model at a previous training time step, wherein the previous training time step is any training time step before the current training time step;

[0042] Determine the total benefit value of the microservice architecture system for the service completion delay index and the system load balancing index based on the historical experience trajectory and the current experience trajectory, and construct the value function loss value of the initialization orchestration model based on the total benefit value;

[0043] Among them, the historical experience trajectory includes historical environment state data, the current experience trajectory includes current environment state data, the historical target service request in the historical environment state data is the same as the current target service request in the current environment state data, and the historical preference vector in the historical environment state data is different from the current preference vector in the current environment state data.

[0044] The present invention also provides a containerized microservice intelligent collaborative orchestration device, which specifically includes the following modules:

[0045] a determination module for determining a service completion delay index of the microservice architecture system and, based on environmental information of the microservice architecture system, determining a system load balancing index of the microservice architecture system, wherein the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request;

[0046] A training module, configured to perform orchestration training on the initialization orchestration model based on the service completion delay indicator and the system load balancing indicator to obtain a microservice orchestration model;

[0047] An acquisition module is configured to obtain request status information of a user service request when the microservice architecture system obtains the user service request, wherein the request status information includes a target microservice and a user preference feature. The target microservice is a containerized microservice to be deployed when the microservice architecture system processes the user service request. The user preference feature is used to represent the user's preference for a service completion delay indicator and a system load balancing indicator.

[0048] An orchestration module, configured to input the request status information into the microservice orchestration model for strategy orchestration processing to obtain an orchestration strategy for the target microservice;

[0049] A deployment module deploys the containerized microservice according to the orchestration strategy and runs the deployed containerized microservice through a microservice architecture system to process the user service request.

[0050] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for intelligent collaborative orchestration of containerized microservices as described above is implemented.

[0051] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the containerized microservice intelligent collaborative orchestration methods described above.

[0052] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the above-mentioned containerized microservice intelligent collaborative orchestration methods.

[0053] The containerized microservice intelligent collaborative orchestration method and device provided by the present invention first determines the service completion delay index and the system load balancing index of the microservice architecture system, and then performs orchestration training on the initialization orchestration model based on these two indicators to obtain a microservice orchestration model. When there is a user request, the request status information containing the user preference characteristics is input into the microservice orchestration model, and the microservice orchestration model will generate an orchestration strategy for the containerized microservice based on the user preference characteristics, and then the microservice architecture system can deploy the containerized microservice according to the orchestration strategy. In this way, the containerized microservice orchestration model is used to formulate an orchestration strategy based on the user's preference for the two indicators to reasonably balance the two indicators of the service completion delay index and the system load balancing index, thereby improving the service performance of the microservice architecture system and reducing management costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 This is a flow chart of the containerized microservice intelligent collaborative orchestration method provided by the present invention.

[0056] Figure 2 This is a schematic diagram of the architecture of the microservice architecture system provided by the present invention.

[0057] Figure 3 This is a directed acyclic diagram formed by the microservices for establishing communication provided by the present invention.

[0058] Figure 4 It is a schematic diagram of the microservice life cycle provided by the present invention.

[0059] Figure 5 This is a diagram illustrating the training principle of the initialization orchestration model provided by an embodiment of the present invention.

[0060] Figure 6 It is a structural diagram of the containerized microservice intelligent collaborative orchestration device provided by the present invention.

[0061] Figure 7 It is a schematic diagram of the physical structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0062] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0063] The following combination Figure 1 To describe the containerized microservice intelligent collaborative orchestration method of the present invention, Figure 1 This is a flow chart of the containerized microservice intelligent collaborative orchestration method provided by the present invention. Figure 1 As shown, the method includes the following steps.

[0064] In step 101, a service completion delay index of the microservice architecture system is determined, and a system load balancing index of the microservice architecture system is determined based on environmental information of the microservice architecture system.

[0065] First, the microservice architecture system in the embodiment of the present invention is introduced. Figure 2 As shown, Figure 2 This is a schematic diagram of the microservice architecture system provided by the present invention. Figure 2 As shown, the microservice architecture system includes a terminal layer, an edge layer and a mirror layer. The terminal layer includes multiple containerized microservices obtained by splitting user service requests, the edge layer includes multiple edge nodes, and the mirror layer includes multiple container images. The container image is used to deploy the containerized microservices to the edge nodes for execution.

[0066] Specifically, the terminal layer is used to accept some delay-sensitive request services randomly generated by users (i.e. Figure 2 These request services S are then broken down into multiple microservices m, which are then deployed to edge nodes n in the edge layer for execution. Microservices m rely on corresponding containers C during execution, so they need to pull images from the image repository deployed in the cloud. After the containerized microservices are deployed on edge nodes n in the edge layer, they are executed by pulling container images. Figure 2 As shown in Figure 1, in the edge layer, edge nodes n are typically located close to end users and access base stations via wireless networks within their geographic coverage area. The deployment and execution of containerized microservices at the end layer impact the service completion time metric, denoted as QoS. Containerized microservice deployment also requires adherence to certain constraints. Service completion time and constraints are discussed below and are not detailed here.

[0067] After the user service request of the terminal is split into multiple microservices, it can be said that the service request is composed of multiple microservices. These microservices may need to frequently transmit data to each other. Therefore, it can be said that there is communication affinity between the two microservices performing data transmission. Of course, a microservice can establish communication affinity relationships with multiple microservices. According to communication affinity, all microservices can be represented as a directed acyclic graph, such as Figure 3 As shown, Figure 3 It is a directed acyclic graph formed by the microservices for establishing communication provided by the present invention. Each rectangle in the graph represents a microservice m, and each directed edge represents communication affinity. The level of affinity is represented by the width of the edge.

[0068] Based on the above description, the following Figure 1 In step 101, the service completion latency metric for the microservice architecture system is first determined to represent the service completion time (QoS). This metric represents the time required for the microservice architecture system to process user service requests and is one of the most critical metrics in a microservice architecture system. This metric reflects the speed at which the system processes user service requests and directly impacts the user experience.

[0069] The completion of a service request is marked by the completion of all its microservices, meaning that all microservices exiting the directed acyclic graph have completed. Typically, within a microservice directed acyclic graph, some microservices can execute in parallel, including non-essential data transmission and processing. Therefore, service completion time is not simply the sum of the completion and transmission times of each microservice. When calculating the completion time of each microservice, it is necessary to consider the execution sequence and parallel execution.

[0070] See also Figure 4 , Figure 4 This is a schematic diagram of the microservice life cycle provided by the present invention. Figure 4 This article explains the specific process of determining the service completion latency indicator of a microservice architecture system.

[0071] like Figure 4 As shown in Figure 1, in a microservice architecture system, the complete lifecycle of a microservice MS involves the microservice request sending phase (i.e., sending user requests to edge nodes), the container startup phase (pull container images), the microservice execution phase (data computation), and the data transmission phase (computation and data transfer). The data transmission phase only exists if and only if two microservices m with communication affinity are deployed on different edge nodes n.

[0072] First, in the microservice sending phase, generally speaking, the connection between the user and the base station is carried out through an internal network with stable bandwidth. In addition, since the network topology between the edge node and the base station is fixed, in the microservice sending phase, each microservice ( Figure 4 The sending time of the microservices MS1, MS2, ..., MSn shown in the figure is considered to be a constant value, which is not affected by the orchestration strategy and can be ignored. Without loss of generality, it can be uniformly assumed that the sending time of all microservices is a constant .

[0073] Next is the container startup phase, which primarily involves pulling and installing images from the image layer's image repository, preparing for containerized microservices. Container startup occurs after microservices are scheduled and before they are executed on edge nodes. Containers share the operating system kernel, and installation time remains stable at around one second regardless of container image size. Therefore, the embodiments of this invention do not focus on the time required to install container images.

[0074] Pulling container images incurs a time cost, which is primarily related to the size of the image layers that the edge node needs to pull and the available bandwidth for scheduling the edge node. When deploying containerized microservices to edge nodes, the edge node determines the time it takes for the containerized microservice to pull the container image from the image layers.

[0075] Assume that the microservice m belonging to the service request s is deployed on the edge node n. The required image layer that the edge node lacks is represented as a 0-1 matrix, that is, .in, Does the representative need Ladi? Layer mirroring, ( ) means need, ( ) means it is not needed, so according to Shannon's theorem, the image layer pull time of microservice m The calculation formula is as follows:

[0076] (1)

[0077] (2)

[0078] In the above formula (2), L represents the number of layers of the pulled container image. Indicates the pulled Layer mirroring, Indicates pulling The time required for layer mirroring, j represents the serial number of the container image. C is the channel capacity, calculated as shown in the above formula (2). In the above formula (2), S / N represents the signal-to-noise ratio of wireless channel transmission. Indicates the transmission bandwidth.

[0079] At the same time, assuming that the containerized microservice edge computing system is a single-threaded download system, that is, it can only pull one image layer at a time. Therefore, each edge node should maintain an image pull waiting queue A newly arrived containerized microservice container image needs to be added to this queue so that it can only wait for other container images. At the same time, once a layer of image layer is pulled, the layer buffer of the pulling edge node adds the layer record to the free storage space under the buffer. Therefore, when calculating the pull time of the image layer of microservice m, additional waiting time must be added. , which can be expressed as the following formula (3):

[0080] (3)

[0081] In the above formula (3), Indicates the pull time of the image layer. Indicates the number of container images waiting in the image pull waiting queue, and i indicates the i-th other container image being waited for.

[0082] Therefore, if the microservice m belonging to the service request s is scheduled to the edge node n, the complete image pull time can be expressed as the following formula (4):

[0083] (4)

[0084] Finally, the microservice execution phase occurs after the required container images have been pulled and installed. Once the containerized microservice (a microservice is called a containerized microservice after it has been pulled and installed, and can then be deployed to an edge node for execution) has been pulled into its container image, the edge node's processing time for the containerized microservice is determined, as well as the data transmission time between edge nodes.

[0085] Here, in edge node n, the CPU data processing speed is mainly related to the host bit rate and CPU main frequency. The mainstream bit rate of the current system is 32 bits, which means that one cycle can process 4 bytes of data. The CPU main frequency represents the number of cycles that can be calculated per second. Assuming that the operating system bit rate of all edge nodes n is 32 bits, the containerized microservice m is allocated CPU resources, the data processing speed of the containerized microservice m can be expressed as: , in byte / s. After obtaining the data processing speed, the service processing time of the containerized microservice m can be expressed as the following formula (5):

[0086] (5)

[0087] In the above formula (5), Indicates the amount of data that edge node n needs to process when processing containerized microservices.

[0088] In addition, the data transmission phase between edge nodes must also be considered, which occurs when and only when there is long-distance data transmission, which is caused by the unfortunate deployment of two containerized microservices with communication affinity on different edge nodes. If containerized microservice m and containerized microservice m' have communication affinity and are respectively scheduled to different edge nodes n and n' for data processing, it means that containerized microservice m can only start execution when and only when the processed data of m' is completely transmitted to arrive. Therefore, the data transmission time between edge nodes must be considered. On the contrary, if they are scheduled to the same edge node, the data transmission time can be ignored due to the huge transmission speed within the edge node. Therefore, the data transmission time of containerized microservice m can be expressed as the following formula (6):

[0089] (6)

[0090] In the above formula (6), m and m' represent two containerized microservices with communication affinity, n and n' represent different edge nodes, n = n' means that m and m' are deployed to the same edge node, n ≠ n' means that m and m' are deployed to two different edge nodes, C represents the channel capacity, Indicates the amount of data that needs to be transferred between containerized microservice m and containerized microservice m'. Represents a collection of containerized microservices that have communication affinity with m.

[0091] Finally, according to the mirror pull time , service processing time and data transmission time , determine the service completion delay indicators of the microservice architecture system , which is the completion time of the service request ( = ) is expressed as the following formula (7):

[0092] (7)

[0093] In the above formula (7), It represents the completion time of the containerized microservice m' on the edge node n that has communication affinity with the containerized microservice m in the service request s.

[0094] Next, based on the environmental information of the microservice architecture system, the system load balancing index of the microservice architecture system is determined. The system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing user service requests, and is calculated based on the current environmental information of the microservice architecture system. Environmental information can be collected and tested based on the actual system environment. The environmental information includes the remaining computing resources and remaining memory of each edge node n in the edge layer. Therefore, for each edge node n, the load computing resources can be determined based on the remaining computing resources, that is, the CPU computing resources used by the edge node, recorded as , and the load memory can be determined based on the remaining memory, that is, the memory used by the edge node, recorded as .

[0095] Next, since there are multiple edge nodes in the edge layer, the first average value of the load computing resources of the edge nodes can be determined, that is, the used CPU computing resources of all edge nodes can be calculated. The average value of The calculation method is to directly accumulate the sum of the CPU computing resources used by all edge nodes, and then divide it by the total number of edge nodes to get the first average value. Next, determine the second average value of the edge node's load memory, that is, calculate the used memory of all edge nodes The average value of The calculation method is to directly accumulate the sum of the CPU computing resources used by all edge nodes, and then divide it by the total number of edge nodes to get the first average value. .

[0096] The final calculation is based on the first average and the second mean Determine the total load balancing value of edge nodes in the microservice architecture system, denoted as Then, the average load balancing value of each edge node is calculated, that is, the mean of the total load balancing value is calculated, and the mean of the total load balancing value is used as the system load balancing index O of the microservice architecture system, which is expressed as the following formula (8):

[0097] (8)

[0098] In the above formula (8), N represents the total number of edge nodes in the edge layer of the microservice architecture system, and n represents edge node n, that is, the nth edge node.

[0099] In step 102, based on the service completion delay index and the system load balancing index, the initialization orchestration model is orchestrated and trained to obtain a microservice orchestration model.

[0100] Through step 101 above, the service completion latency and system load indicators of the microservice architecture system are calculated based on the environmental information. Both indicators are factors that need to be considered and balanced when orchestrating containerized microservices. Because these two indicators have different focuses for different users, service users generally prefer lower service processing latency to improve the service experience. Service providers, on the other hand, prefer lower system load indicators, which means a more balanced system load to improve overall resource utilization and overall system performance.

[0101] In the embodiments of the present invention, the orchestration of containerized microservices involves deploying multiple containerized microservices to corresponding edge nodes. Different orchestration methods can lead to different service completion latency and system load indicators. The technical problem addressed by the embodiments of the present invention is how to develop an appropriate orchestration strategy for deploying containerized microservices to balance these two indicators in the microservice system architecture—that is, to use these two indicators as optimization targets to optimize for different user needs.

[0102] Based on this, the present invention designs an initialization orchestration model to formulate an appropriate orchestration strategy to optimize both optimization objectives. Crucially, the orchestration model must provide an orchestration strategy for containerized microservices while balancing these two metrics. Therefore, the present invention employs a multi-objective reinforcement learning approach, using service completion latency and system load balancing as two optimization objectives. This approach trains the initialization orchestration model to generate a microservices orchestration model, which will be described in detail below.

[0103] First, we introduce the initialization orchestration model. According to the multi-objective reinforcement learning algorithm, the initialization orchestration model includes the policy network The policy network and the critic network can be structured as multi-layer perceptrons. The policy network is responsible for designing the orchestration strategy and generating action design data. The critic network is responsible for evaluating the orchestration strategy designed by the policy network and helping to optimize the policy network during training. The training process for initializing the orchestration model is performed according to training time steps. At each training time step, a new policy network and a new critic network are generated.

[0104] The following describes the training process of the initial arrangement model. For the sake of convenience and clarity, the embodiment of the present invention divides the training process into four steps, from step 1021 to step 1024, and combines Figure 5 To illustrate, Figure 5 This is a diagram illustrating the training principle of the initialization orchestration model provided by an embodiment of the present invention.

[0105] In step 1021, environmental status data is obtained.

[0106] like Figure 5 As shown, first, it is necessary to obtain training data for training the initialization orchestration model. In this embodiment of the present invention, the environmental status data of the microservice architecture system is used as training data, which is recorded as Environmental status data Include target service request and the preference vector , the target service request is the service request involved in the environment of the microservice architecture system, and the preference vector is the preference set collected from (in, ), each preference vector in the preference set is sampled from Represents the user's service completion delay indicator The preference degree is different from the system load balancing index O. Bias vector Specifically based on the delay weight and load weight Constructed, delay weight Characterizing target service requests For service completion delay indicators Preference degree, load weight Characterizing target service requests According to the preference of the system load balancing index O, the optimization target in the embodiment of the present invention is only the service completion delay index And the system load balancing index O two indicators, so + =1, so the environmental status data The expression formulas are as follows: Formula (9) and Formula (10):

[0107] (9)

[0108] (10)

[0109] In the above formulas (9) and (10), t represents the current time, That is, it represents the environmental status data at time t.

[0110] Therefore, at different times t, different environmental status data can be collected By designing such environment status data, the initialization orchestration model can learn and understand the actual needs of users through preference vectors, and then formulate corresponding orchestration strategies according to the needs. Furthermore, the initialization orchestration model can understand that even for the same target server request, , preference vector at different times t This will also lead to service completion delay indicators The different emphases generated by the system load balancing index O ultimately affect the action decision.

[0111] In step 1022, at the current training time step, the environment state data is input into the policy network to generate action design data, and the judgment network is called to predict the environment state data to generate an expected reward vector value for the action design data.

[0112] Obtain environmental status data at multiple times t After that, it can be used to train the initialization orchestration model, such as Figure 5 As shown, the initialization orchestration model given in the embodiment of the present invention is trained according to the training time step T, that is, the initialization orchestration model is updated once every training time step T. In a current training time step T, the model before the update is called the old strategy model, and the model after the update is called the new strategy model.

[0113] During the training process, first at the current training time step T (representing any training time step in the training process), the environment state data Input to the policy network that initializes the orchestration model Generate action design data Specifically, at the current training time step T, the environment state data At the same time, input into the new policy network and the old policy network , where the old policy network is the new policy network in the previous training time step (T-1), and the old policy network First, based on the state environment data Generate the current experience trajectory.

[0114] It should be noted that if it is the first training time step (T=1), there is no historical experience trajectory and it is impossible to sample. You can directly generate the current experience trajectory or randomly initialize the experience estimate for sampling. Here, the historical experience trajectory is the experience trajectory generated by the interaction between the action design data of the previous training time step (T-1) and the microservice architecture system environment, so it is called the historical experience trajectory, which is expressed as { }, which will be explained in detail below. Based on the status environment data Fit the normal probability distribution function, which will output the predicted probability of the corresponding choreography action, and extract the predicted probability from the old strategy network The experience trajectory corresponding to the sampled choreography action in the generated historical experience trajectory is used to generate the action design data for the current training time step T This action design data Used to characterize edge node deployment actions, which are the microservice architecture system processing target service requests When the containerized microservice m of the terminal layer is deployed to the edge node n in the edge layer, the deployment action is to initialize the orchestration model according to the preference vector Developed scheduling strategies.

[0115] In the process of updating the policy network, the old policy network is updated by sampling the historical experience trajectory Parameters in (The update method is to update the final loss value) to obtain the new strategy network , New Strategy Network After the parameter update is completed, the updated parameters Pass back to the old policy network After the parameter update is completed, the old policy network after the parameter update is used as the old policy network for the next training time step T+1.

[0116] In the strategic network Generate motion design data At the same time, the initialization orchestration model will call the critic network to evaluate the environmental status data. Make predictions and generate data designed for the action The expected reward vector value of ,in, , Indicates the service completion delay indicator The expected reward vector value of Represents the expected vector value of the system load balancing index O.

[0117] In step 1023, a policy loss value of the initialized orchestration model is constructed based on the action design data, and a value function loss value of the initialized orchestration model is constructed based on the action design data and the expected reward vector value.

[0118] At any training time step T, initialize the strategy network of the orchestration model Generate motion design data , and the judging network Critic is used to judge the action design data Conduct evaluation and generate motion design data The expected reward vector value of , then design data according to the action and the expected reward vector value Construct the final loss value of the initialized orchestration model in the current training time step T In this embodiment of the present invention, the final loss value of the initialization arrangement model is By Strategy Network The strategy loss value And the loss value of the value function of the judging network Critic of and get.

[0119] In the embodiment of the present invention, according to the action design data Constructing the policy loss value for the initial orchestration model , and design data based on actions And the expected reward vector value Construct the value function loss value of the initialized orchestration model , which will be explained below respectively.

[0120] First, we introduce the design data based on action Constructing the policy loss value for initializing the orchestration model , obtain the action design data generated by the policy network at the current training time step T The predicted probability of Because the new policy network is based on the input state environment data Fit the normal probability distribution function, which will output the predicted probability of the corresponding choreography action, and extract the predicted probability from the old strategy network The experience trajectory corresponding to the sampled choreography action in the generated historical experience trajectory is used to generate the action design data for the current training time step T , so here we select the generated action design data from the multiple predicted probabilities corresponding to the choreography action The predicted probability of .

[0121] Next, according to the predicted probability Determine the strategy amplitude value of the strategy network between training time steps T (that is, between the current training time step T and the previous training time step T-1) , so here we need to determine the action design data in the last training time step T-1 The predicted probability of First, based on the state environment data Generate the current experience trajectory. In the process of generating the current experience trajectory, the action design data in the last training time step T-1 will be fitted The predicted probability can be directly obtained as , used to determine the strategy amplitude value The determination method is to calculate the current training time step T for the action design data The predicted probability , and the action design data in the last training time step T-1 The predicted probability The probability ratio is used as the strategy amplitude value and the calculation formula is as follows (11):

[0122] (11)

[0123] Next, within the current training time step T, the generalized advantage estimates of the service completion delay index and the system load balancing index are determined, respectively, and recorded as According to multi-objective reinforcement learning, the generalized advantage estimate of m optimization objectives is calculated within a current training time step T. Since the optimization objectives in this embodiment of the present invention are only the service completion delay index and the system load balancing index, the value of m is 2. The generalized advantage estimate is calculated as follows: Formula (12) to Formula (14):

[0124] (12)

[0125] (13)

[0126] (14)

[0127] In the above formulas (12) to (14), Represents the expected reward vector value of the optimization target m, t represents the training time step of the unit, and the maximum value is T. is the discount factor, is the discount parameter, is the attenuation coefficient. represents the cumulative reward value of the optimization target m within the training time step T, Represents the time difference error of the optimization target m within the training time step T. r represents the reward gain value of the optimization target m.

[0128] Next, determine the advantage function value of the policy network, first estimate the value of the generalized advantage The preference vector included with the environmental state data , determine the delay advantage value of the policy network of the initialization orchestration model for the service completion delay indicator, and the load advantage value of the policy network for the system load balancing indicator. Here, the generalized advantage value of the optimization target is combined with the user's preference vector, allowing the policy network to learn the focus of the optimization target based on the preference vector. Here, the generalized advantage estimate can be weighted directly by the preference vector. Therefore, the delay advantage value of the policy network of the initialization orchestration model for the service completion delay indicator is , the load advantage value of the policy network for the system load balancing index is .

[0129] Then, according to the delay advantage value and the load advantage value, the advantage function value of the strategy network is determined. , the calculation formula can be expressed as the following formula (15):

[0130]

[0131] In the above formula (15), N represents the total number of preference vectors, i represents the i-th preference vector, and M represents the total number of optimization targets. In the embodiment of the present invention, the optimization targets are only the service completion delay index and the system load balancing index. Therefore, M is 2, and m represents the m-th optimization target.

[0132] Finally, according to the advantage function value and the strategy amplitude value Construct the strategy loss value of the initialized orchestration model The calculation formula is as follows (16):

[0133] (16)

[0134] In the above formula (16), min represents the minimum value function, clip represents the limit value range function, Represents the learning rate, so when the policy network updates the parameters, the parameters Updated to .

[0135] The following describes the specific process of constructing the loss value of the value function of the initialization orchestration model based on the action design data and the expected reward vector value.

[0136] Since the critic network is used to evaluate the action design data generated by the policy network, the evaluation method is to evaluate the action design data generated by the policy network at the current training time step T. Put it back into the microservice architecture system environment for execution verification, then record the current experience trajectory according to the verification process, and then calculate the total benefit value of the optimization target , used for action design data Evaluate and then construct the value function loss value .

[0137] Therefore, the calculated value function loss value The first step is to execute action design data through the microservice architecture system within the current training time step , and record the execution action design data The current experience trajectory includes action design data , preference vector w, expected reward vector value Parameters such as .

[0138] The execution process can be that each unit training time step (moment) t is based on the environment state data , then select Action Design Data , then at the next unit training time step t+1, the environment of the microservice architecture system is based on the transition probability p ( | , )∈p execution and transfer its state to the next state At the same time, execute the action design data After that, you will get immediate feedback, that is, you will get the execution action design data from the environment of the microservice architecture system based on the expected reward vector value. Total revenue after .

[0139] Calculate the loss value of the value function The second step is to determine the total revenue of the microservice architecture system for the service completion latency metric and the system load balancing metric based on the current experience trajectory, and then determine the discounted revenue value of the total revenue. The current experience trajectory contains the corresponding preference vector w, so the revenue value of the corresponding metric can be calculated based on the preference vector w for the corresponding metric.

[0140] The specific process is to first obtain the reward vector value of the service completion delay indicator and the system load balancing indicator. , this reward vector value can be preset and expressed as the following formula (17) and formula (18):

[0141] (17)

[0142] (18)

[0143] In the above formulas (17) and (18), Represents the service completion delay indicator The income value, Represents the benefit value of the system load balancing indicator O.

[0144] Next, according to the preference vector w and reward vector value included in the current experience trajectory , determine the linear reward scalar value. Similarly, here we can directly calculate the reward vector value by weighted multiplication through the preference vector. The linear reward scalar value is recorded as It is expressed as the following formula (19):

[0145] (19)

[0146] Then determine the penalty value obtained by the microservice architecture system after executing the constraint deployment action when executing the action design data corresponding to the current experience trajectory In multi-objective reinforcement learning, the benefit value of the optimization objective is used to guide learning how to generate action design data and perform execution deployment.

[0147] However, when executing the actual deployment action for the containerized microservice, some constraints (or restriction rules) may be violated. The consequences of these constraints will lead to a decrease in the corresponding benefit value. Therefore, the action that violates the constraint in the action design data becomes a constrained deployment action. Therefore, in the embodiment of the present invention, a penalty is imposed on the action design data that violates the constraint, and the penalty value is obtained. , and the penalty value Add to revenue value middle.

[0148] The present invention divides constraints into soft constraints and hard constraints. Specifically, in the process of containerized microservice scheduling, some containerized microservices must be deployed, which is called constraint 1. If the necessary containerized microservices are not deployed, constraint 1 is violated. In addition, the deployed edge node cannot exceed the resource capacity of the edge node, which is called constraint 2. If it exceeds, constraint 2 is violated. Constraints 1 and 2 are both hard constraints, that is, violations are prohibited. In theory, unnecessary containerized microservices cannot be deployed, which is called constraint (3). If unnecessary containerized microservices are deployed, constraint 3 is violated. However, in the case of violating constraint 3, the integrity of the microservice is guaranteed, which is acceptable to some extent and is called a soft constraint. Obviously, violations of hard constraints and soft constraints should have different degrees of penalties. A penalty quota is introduced in the embodiment of the present invention. , and set the penalty level for violating hard constraints Set as the penalty level for violating the soft constraint Therefore, the penalty value It can be expressed as the following formula (20):

[0149] (20)

[0150] In the above formula (20), represents the penalty level for violating hard constraints, represents the number of violations of the hard constraint, represents the penalty level for violating the soft constraint, represents the number of violations of the soft constraint, Represents the preset penalty factor.

[0151] Finally, the linear reward scalar value and penalty value The difference is used as the total benefit value of the microservice architecture system for the service completion delay index and the system load balancing index , expressed as the following formula (21):

[0152] (twenty one)

[0153] Calculate the total revenue Then determine the total revenue value The discounted revenue value is directly weighted by the discount factor, and the discounted revenue value is expressed as ,in, represents the discount parameter, Represents the discount factor, and m represents the mth optimization objective.

[0154] Calculate the loss value of the value function The third step is to determine the cumulative reward vector value in the current training time step and determine the average value of the difference between the cumulative reward vector value and the discounted return value.

[0155] Here, the cumulative reward vector value within the current training time step T can be obtained by the expected reward vector value included in the current experience trajectory To calculate, then calculate the cumulative reward vector value and discounted benefit value The difference between the two values ​​is used as the cumulative loss of the optimization target in the current training time step T. Finally, the average value of the cumulative loss of profit is calculated as the loss of profit in the unit training time step t, which is recorded as , specifically expressed as the following formula (22):

[0156] (twenty two)

[0157] Calculate the loss value of the value function The last step is to integrate the preference vector w into the profit loss to guide the importance of the critic network for different optimization goals. The integration process is to directly use the preference vector to The profit loss is weighted, so the final value function loss is expressed as the following formula (23):

[0158] (twenty three)

[0159] Then the loss value of the value function The calculation method is finally expressed as the following formula (24):

[0160] (twenty four)

[0161] In the above formulas (23) and (24), represents the discount parameter, represents the discount factor, m represents the mth optimization objective, The i-th preference vector is represented in the table, N represents the number of preference vectors, M represents the number of optimization targets, which is 2 in the embodiment of the present invention, T represents the current training time step, t represents the unit training time step, and the maximum value is T.

[0162] In some embodiments, as Figure 5 As shown, in the calculation function loss value In the first step, that is, in the current training time step, the action design data is executed through the microservice architecture system , and record the execution action design data After the current experience trajectory is obtained, the current experience trajectory will be stored in the historical experience trajectory library. The historical experience trajectory library is used to store the historical experience trajectory recorded by the initialization choreography model in the previous training time step. The previous training time step here is any training time step before the current training time step T. The historical experience trajectory includes the action design data involved in the previous training time step. , preference vector w, expected reward vector value and other parameters.

[0163] In the calculation function loss value In the second step, the embodiment of the present invention calculates the total benefit value in the sampling experience trajectory When calculating the total benefit, an empirical trajectory sampling balancing mechanism is adopted. This mechanism considers both historical and current empirical trajectories. Specifically, the total benefit of the microservice architecture system for service completion latency and system load balancing is determined based on the historical and current empirical trajectories. The subsequent steps are then performed to construct the loss value of the value function of the initialized orchestration model based on the total benefit.

[0164] It should be noted here that the historical experience trajectory includes historical environmental state data (e.g., ), the current experience trajectory includes the current environment state data (e.g., represented as ), the historical target service request in the historical environment state data is the same as the current target service request in the current environment state data, that is, both can represent , but the historical preference vector in the historical environmental state data and the current preference vector in the current environment state data different.

[0165] Here, when using the sampling balance mechanism to sample the experience trajectory to calculate the loss value of the value function, not only the current experience trajectory of the current training time step T is sampled, but also the historical experience trajectory of the previous training time step is sampled, which can make the judgment network realize that even when facing the same target service request Different preference vectors w can still generate different orchestration strategies. Therefore, through the empirical trajectory sampling balancing mechanism, the initialized orchestration model focuses on active preference vectors w to quickly achieve the focused optimization goal. This allows the model to maintain its trained strategy and prevents overfitting to the current preference vector during training, preventing past orchestration strategies from being forgotten. This improves the initialization of the orchestration model's ability to converge quickly on new preference vectors.

[0166] In step 1024, the sum of the policy loss value and the value function loss value is used as the final loss value of the initialized orchestration model in the current training time step, and the initialized orchestration model is trained using the final loss value to obtain a microservice orchestration model.

[0167] Through the above step 1023, in the initialization orchestration model, a strategy network is constructed The strategy loss value The loss value of the value function of the critic network Afterwards, if Figure 5 As shown, the strategy loss value Loss value with the value function The sum of the initialization and arrangement model is used as the final loss value in the current training time step T. , and pass the final loss value Train the initialization orchestration model to obtain the microservice orchestration model.

[0168] The final loss value during training Perform self-learning according to the set learning rate , when the model is updating the parameters, the parameters Updated to , enter the next training time step T+1, and at the next training time step T+1, continue to repeat the training steps of the current training time step T to train the policy network and the judgment network at the same time. When convergence occurs, the training process is complete. The initial encoding model after training is the microservice orchestration model.

[0169] Continue to see Figure 1 In step 103, when the microservice architecture system obtains the user service request, it obtains the request status information of the user service request.

[0170] After training, the microservice orchestration model can be deployed to the microservice architecture platform, for example, at the endpoint layer. When the microservice architecture system receives a user service request, it obtains the request status information. This information includes the target microservice to be executed and user preference characteristics. The target microservice is the containerized microservice that the microservice architecture system must deploy to process the user service request. The user preference characteristics represent the user's preference for service completion latency and system load balancing.

[0171] In step 104, the request status information is input into the microservice orchestration model for policy orchestration processing to obtain an orchestration strategy for the target microservice.

[0172] After the microservices architecture system receives the request status information for a user service request, it directly inputs it into the microservices orchestration model. The microservices orchestration model immediately parses the request status information and determines the target microservice included in the user service request. The target microservice is the containerized microservice that the microservices architecture system needs to deploy to process the user service request. From this, the microservices orchestration model can determine the containerized microservice that needs to be deployed to process the user service request.

[0173] In addition, the microservice orchestration model parses the user preference vector based on user preference characteristics and then performs policy orchestration processing for the containerized microservices and user preference vectors. Specifically, the policy network of the microservice orchestration model generates action design data based on the containerized microservices and user preference vectors, and evaluates the generated action design data through the evaluation network. In other words, it optimizes the optimization target based on the user preference characteristics, that is, it maximizes the balance of the service completion delay indicator and the system load balancing indicator after the microservice architecture module executes the action design data. When the service completion delay indicator and the system load balancing indicator reach maximum balance, the corresponding action design data at this time is output as the orchestration strategy for the target microservice and returned to the terminal layer of the microservice architecture model.

[0174] In step 105, the containerized microservices are deployed according to the orchestration strategy, and the deployed containerized microservices are run through the microservice architecture system to process user service requests.

[0175] Through step 104, after the microservice orchestration model outputs the orchestration strategy of the corresponding containerized microservice, that is, after outputting the corresponding action design data, the terminal layer of the microservice architecture system deploys the containerized microservice according to the orchestration strategy, that is, the terminal layer deploys each containerized microservice m to be deployed to the edge node n in the edge layer for execution.

[0176] After the action design data is executed, meaning deployment is complete, the information is transmitted to the base station through the edge nodes of the microservice architecture system. The deployed containerized microservices are then run to process the user service request and return it to the terminal layer. The execution of the containerized microservices is also the process of processing the user service request. When the containerized microservices are executed, the user service request is also processed.

[0177] Since the action design data output by the microservice orchestration model (i.e., the deployment action for edge nodes) is orchestrated based on user preference characteristics, this deployment action balances the two conflicting indicators of service completion latency and system load balancing. Therefore, optimizing the service completion latency indicator ensures that the process of processing user service requests can not only meet user needs and improve the user service experience, but also optimize the system load balancing indicator to improve the service performance of the microservice architecture system and reduce management costs.

[0178] Through the embodiment of the present invention, the service completion delay index and the system load balancing index of the microservice architecture system are first determined, and then the initialization orchestration model is orchestrated and trained based on these two indicators to obtain a microservice orchestration model. When there is a user request, the request status information containing the user preference characteristics is input into the microservice orchestration model, and the microservice orchestration model will generate an orchestration strategy for the containerized microservice based on the user preference characteristics, and then the microservice architecture system can deploy the containerized microservice according to the orchestration strategy. In this way, the containerized microservice orchestration model is used to formulate an orchestration strategy based on the user's preference for the two indicators to reasonably balance the service completion delay index and the system load balancing index, thereby improving the service performance of the microservice architecture system and reducing management costs.

[0179] The following describes the containerized microservices intelligent collaborative orchestration device provided by the present invention. The containerized microservices intelligent collaborative orchestration device described below and the containerized microservices intelligent collaborative orchestration method described above can be referenced to each other.

[0180] Figure 6 This is a schematic diagram of the structure of the containerized microservice intelligent collaborative orchestration device provided by the present invention. Figure 6As shown, the present invention provides a containerized microservice intelligent collaborative orchestration device, which specifically includes: a determination module 601, a training module 602, an acquisition module 603, an orchestration module 604, and a deployment module 605. Among them, the determination module 601 is used to determine the service completion delay index of the microservice architecture system, and determine the system load balancing index of the microservice architecture system based on the environmental information of the microservice architecture system. The service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; the training module 602 is used to perform orchestration training on the initialization orchestration model based on the service completion delay index and the system load balancing index to obtain a microservice orchestration model; the acquisition module 603 is used to, when the microservice architecture system obtains a user service request, Obtain request status information of the user service request, wherein the request status information includes the target microservice and user preference characteristics, the target microservice is the containerized microservice that needs to be deployed when the microservice architecture system processes the user service request, and the user preference characteristics are used to characterize the user's preference for the service completion delay indicator and the system load balancing indicator; the orchestration module 604 is used to input the request status information into the microservice orchestration model for policy orchestration processing to obtain the orchestration strategy for the target microservice; the deployment module 605 deploys the containerized microservice according to the orchestration strategy, and runs the deployed containerized microservice through the microservice architecture system to process the user service request.

[0181] Specifically, the above-mentioned containerized microservice intelligent collaborative orchestration device provided by the present invention can implement all the method steps implemented by the above-mentioned containerized microservice intelligent collaborative orchestration method embodiment, and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those of the method embodiment will not be described in detail here.

[0182] Figure 7 An example of a physical structure diagram of an electronic device is shown below. Figure 7As shown, the electronic device may include: a processor 710, a communications interface 720, a memory 730 and a communication bus 740, wherein the processor 710, the communications interface 720 and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call the logic instructions in the memory 730 to execute the containerized microservice intelligent collaborative orchestration method, the method including: determining the service completion delay index of the microservice architecture system, and determining the system load balancing index of the microservice architecture system based on the environmental information of the microservice architecture system, the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; based on the service completion delay index and the system load balancing index, the initialization orchestration model is orchestrated and trained to obtain a microservice orchestration model; when the microservice When the service architecture system obtains a user service request, it obtains request status information of the user service request, wherein the request status information includes a target microservice and user preference characteristics, the target microservice is a containerized microservice that needs to be deployed when the microservice architecture system processes the user service request, and the user preference characteristics are used to characterize the user's preference for the service completion delay indicator and the system load balancing indicator; the request status information is input into the microservice orchestration model for policy orchestration processing to obtain an orchestration strategy for the target microservice; the containerized microservice is deployed according to the orchestration strategy, and the deployed containerized microservice is run through the microservice architecture system to process the user service request.

[0183] Furthermore, the logic instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0184] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the containerized microservice intelligent collaborative orchestration method provided by the above methods, the method including: determining the service completion delay index of the microservice architecture system, and determining the system load balancing index of the microservice architecture system based on the environmental information of the microservice architecture system, the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; based on the service completion delay index and the system load balancing index , perform orchestration training on the initialization orchestration model to obtain a microservice orchestration model; when the microservice architecture system obtains a user service request, obtain request status information of the user service request, wherein the request status information includes a target microservice and a user preference feature, the target microservice is a containerized microservice that needs to be deployed when the microservice architecture system processes the user service request, and the user preference feature is used to characterize the user's preference for the service completion delay indicator and the system load balancing indicator; the request status information is input into the microservice orchestration model for policy orchestration processing to obtain an orchestration strategy for the target microservice; the containerized microservice is deployed according to the orchestration strategy, and the deployed containerized microservice is run through the microservice architecture system to process the user service request.

[0185] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the containerized microservice intelligent collaborative orchestration method provided by the above methods, the method comprising: determining a service completion delay index of the microservice architecture system, and determining a system load balancing index of the microservice architecture system based on environmental information of the microservice architecture system, the service completion delay index being the time required for the microservice architecture system to process a user service request, and the system load balancing index being used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; compiling the initialization orchestration model based on the service completion delay index and the system load balancing index; The method comprises the following steps: first, performing training on the microservices and obtaining a microservices orchestration model; when the microservices architecture system obtains a user service request, obtaining request status information of the user service request, wherein the request status information includes a target microservice and a user preference feature, the target microservice is a containerized microservice to be deployed when the microservices architecture system processes the user service request, and the user preference feature is used to characterize the user's preference for the service completion delay indicator and the system load balancing indicator; inputting the request status information into the microservices orchestration model for strategy orchestration processing to obtain an orchestration strategy for the target microservice; deploying the containerized microservice according to the orchestration strategy, and running the deployed containerized microservice through the microservices architecture system to process the user service request.

[0186] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0187] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0188] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A containerized microservice intelligent collaborative orchestration method, applied to a microservice architecture system, characterized by: The method comprises: Determine a service completion delay index of the microservice architecture system, and determine a system load balancing index of the microservice architecture system based on environmental information of the microservice architecture system, wherein the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; Based on the service completion delay indicator and the system load balancing indicator, the initialization orchestration model is orchestrated and trained to obtain a microservice orchestration model; When the microservice architecture system obtains a user service request, it obtains request status information of the user service request, wherein the request status information includes a target microservice and a user preference feature. The target microservice is a containerized microservice to be deployed when the microservice architecture system processes the user service request. The user preference feature is used to represent the user's preference for the service completion delay indicator and the system load balancing indicator. Inputting the request status information into the microservice orchestration model for strategy orchestration processing to obtain an orchestration strategy for the target microservice; The containerized microservice is deployed according to the orchestration strategy, and the deployed containerized microservice is run through the microservice architecture system to process the user service request.

2. A containerized microservice intelligent collaborative orchestration method according to claim 1, characterized in that: The microservice architecture system includes a terminal layer, an edge layer, and a mirror layer. The terminal layer includes multiple containerized microservices obtained by splitting user service requests. The edge layer includes multiple edge nodes. The mirror layer includes multiple container images. The container images are used to deploy the containerized microservices to the edge nodes for execution. Determining the service completion delay indicator of the microservice architecture system includes: When the containerized microservice is deployed to the edge node, determining, in the edge node, an image pulling time for the containerized microservice to pull a container image from the image layer; After the containerized microservice pulls the container image, determining the service processing time of the containerized microservice by the edge node, and determining the data transmission time between the edge nodes; Determine a service completion delay index of the microservice architecture system based on the image pulling time, the service processing time, and the data transmission time.

3. A containerized microservice intelligent collaborative orchestration method according to claim 1, characterized in that: The microservice architecture system includes an edge layer, the edge layer includes multiple edge nodes, the environmental information includes remaining computing resources and remaining memory of each edge node, and determining the system load balancing index of the microservice architecture system based on the environmental information of the microservice architecture system includes: For each of the edge nodes, determining a load computing resource according to the remaining computing resource, and determining a load memory according to the remaining memory; Determining a first average value of a load computing resource of the edge node, and determining a second average value of a load memory of the edge node; The total load balancing value of the edge nodes in the microservice architecture system is determined according to the first average value and the second average value, and the average of the total load balancing values ​​is used as the system load balancing indicator of the microservice architecture system.

4. A containerized microservice intelligent collaborative orchestration method according to claim 1, characterized in that: The initialization orchestration model includes a policy network and a judgment network. The training process of the initialization orchestration model is performed according to the training time step. The initialization orchestration model is orchestrated and trained based on the service completion delay index and the system load balancing index to obtain a microservice orchestration model, including: Obtaining environmental status data, wherein the environmental status data includes a target service request and a preference vector, the preference vector being constructed based on a delay weight and a load weight, the delay weight representing a preference of the target service request for the service completion delay indicator, and the load weight representing a preference of the target service request for the system load balancing indicator; At a current training time step, the environment state data is input into the policy network to generate action design data, and the evaluation network is called to predict the environment state data to generate an expected reward vector value for the action design data, wherein the action design data is used to characterize an edge node deployment action, and the edge node deployment action is a deployment action of deploying the containerized microservice of the terminal layer to the edge node in the edge layer when the microservice architecture system processes the target service request. The current training time step is any training time step in the orchestration training process; Constructing a strategy loss value of the initialized orchestration model according to the action design data, and constructing a value function loss value of the initialized orchestration model according to the action design data and the expected reward vector value; The sum of the strategy loss value and the value function loss value is used as the final loss value of the initialized orchestration model in the current training time step, and the initialized orchestration model is trained by the final loss value to obtain a microservice orchestration model.

5. A containerized microservice intelligent collaborative orchestration method according to claim 4, characterized in that: The constructing the strategy loss value of the initialization orchestration model according to the action design data includes: Obtaining the predicted probability of the policy network when generating action design data at the current training time step, and determining the policy amplitude value of the policy network between training time steps based on the predicted probability; In the current training time step, the generalized advantage estimates of the service completion delay indicator and the system load balancing indicator are determined respectively; Determining, based on the generalized advantage estimate and the preference vector included in the environmental state data, a delay advantage value of the policy network of the initialized orchestration model for the service completion delay indicator and a load advantage value of the policy network for the system load balancing indicator; Determining an advantage function value of the strategy network according to the delay advantage value and the load advantage value; A strategy loss value of the initialized orchestration model is constructed based on the advantage function value and the strategy amplitude value.

6. A containerized microservice intelligent collaborative orchestration method according to claim 4, characterized in that: Constructing a value function loss value of the initialized orchestration model according to the action design data and the expected reward vector value, including: In the current training time step, the action design data is executed by the microservice architecture system, and a current experience trajectory of executing the action design data is recorded, wherein the current experience trajectory includes the action design data, the preference vector, and the expected reward vector value; Determine, based on the current experience trajectory, a total revenue value of the microservice architecture system for a service completion latency indicator and a system load balancing indicator, and determine a discounted revenue value for the total revenue value; Determining a cumulative reward vector value in a current training time step, and determining an average value of a difference between the cumulative reward vector value and the discounted benefit value; A value function loss value of the initialized orchestration model is constructed based on the average value of the difference and the preference vector.

7. A containerized microservice intelligent collaborative orchestration method according to claim 6, characterized in that: Determining the total benefit value of the microservice architecture system for the service completion delay indicator and the system load balancing indicator based on the current experience trajectory includes: Obtain the reward vector values ​​for the service completion delay indicator and the system load balancing indicator; determining a linear reward scalar value according to the preference vector included in the current experience trajectory and the reward vector value; Determining a penalty value obtained by the microservice architecture system after executing the constrained deployment action when executing the action design data corresponding to the current experience trajectory; The difference between the linear reward scalar value and the penalty value is used as the total benefit value of the microservice architecture system for the service completion delay index and the system load balancing index.

8. A containerized microservice intelligent collaborative orchestration method according to claim 6, characterized in that: After recording the experience track of executing the action design data, the method further includes: Storing the current experience trajectory in a historical experience trajectory library, wherein the historical experience trajectory library is used to store the historical experience trajectory recorded by the initialized orchestration model at a previous training time step, wherein the previous training time step is any training time step before the current training time step; Determine the total benefit value of the microservice architecture system for the service completion delay index and the system load balancing index based on the historical experience trajectory and the current experience trajectory, and construct the value function loss value of the initialization orchestration model based on the total benefit value; Among them, the historical experience trajectory includes historical environment state data, the current experience trajectory includes current environment state data, the historical target service request in the historical environment state data is the same as the current target service request in the current environment state data, and the historical preference vector in the historical environment state data is different from the current preference vector in the current environment state data.

9. A containerized microservice intelligent collaborative orchestration device, characterized in that: The device comprises: a determination module for determining a service completion delay index of a microservice architecture system and, based on environmental information of the microservice architecture system, determining a system load balancing index of the microservice architecture system, wherein the service completion delay index is the time required for the microservice architecture system to process a user service request, and the system load balancing index is used to characterize the degree of system load imbalance of the microservice architecture system when processing a user service request; A training module is used to perform orchestration training on the initialization orchestration model based on the service completion delay indicator and the system load balancing indicator to obtain a microservice orchestration model; An acquisition module is configured to obtain request status information of a user service request when the microservice architecture system obtains the user service request, wherein the request status information includes a target microservice and a user preference feature. The target microservice is a containerized microservice to be deployed when the microservice architecture system processes the user service request. The user preference feature is used to represent the user's preference for a service completion delay indicator and a system load balancing indicator. An orchestration module, configured to input the request status information into the microservice orchestration model for strategy orchestration processing to obtain an orchestration strategy for the target microservice; A deployment module deploys the containerized microservice according to the orchestration strategy and runs the deployed containerized microservice through a microservice architecture system to process the user service request.

Images