Method, device and equipment for hybrid orchestration of AI services and microservices in cloud-edge collaboration

Abstract

A method, apparatus, and device for hybrid orchestration of AI services and microservices in cloud-edge collaboration include determining the amount of heterogeneous service resources to be allocated based on target edge user requests, a heterogeneous service resource threshold, and full-scale AI service operation intensity, and allocating resources to full-scale AI services at the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated; determining the number of to-be-deployed instances of lightweight AI services and microservices based on target edge user requests; screening out a target AMDQN corresponding to target basic information from an orchestration strategy database containing multiple basic AMDQNs having a mapping relationship with geographic location-time slot combinations; and controlling the target AMDQN to interact with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of to-be-deployed instances in the current heterogeneous service orchestration network, thereby effectively realizing hybrid orchestration of AI services and microservices in a heterogeneous environment.

Description

Technical Field

[0001] The present application relates to the field of cloud-edge collaborative network technology, and specifically to a method, device, and equipment for hybrid orchestration of AI services and microservices in cloud-edge collaboration. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI services have been widely used in areas such as smart homes, smart cities, and autonomous driving. However, these applications place extremely high demands on service performance, including service response latency, resource utilization, and system robustness. However, because existing network architectures are generally centered around centralized computing, they struggle to meet the performance requirements of diverse scenarios and user types. This is especially true in cloud-edge collaborative networks with heterogeneous computing and resources, where the hybrid orchestration of AI services and microservices faces numerous challenges.

[0003] Currently, there is little relevant work or technology focusing on making valuable research on the hybrid orchestration of AI services and microservices. Among them, existing research is more focused on the deployment optimization of a single service type, such as using microservice architecture to achieve horizontal expansion of microservice instances in mobile edge computing (MEC) scenarios, or only considering AI service deployment and task offloading for cloud-edge collaboration. However, these methods usually ignore the complex interaction characteristics between AI services and microservices in the service chain, and it is difficult to effectively deal with scenarios where delay-sensitive requests and compute-intensive requests coexist in heterogeneous environments. Therefore, the above methods cannot be used to achieve hybrid orchestration of AI services and microservices. Summary of the Invention

[0004] The present application provides a method, apparatus, and device for hybrid orchestration of AI services and microservices in cloud-edge collaboration, so as to effectively realize hybrid orchestration of AI services and microservices in a heterogeneous environment.

[0005] In a first aspect, an embodiment of the present application provides a method for hybrid orchestration of AI services and microservices in cloud-edge collaboration, comprising the following steps:

[0006] Obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot;

[0007] Determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource threshold, and the operation intensity of the full-scale AI service;

[0008] Allocate resources for full-scale AI services at the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated;

[0009] Determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request;

[0010] Filtering a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as the target AMDQN, wherein the orchestration strategy database includes a plurality of basic AMDQNs having a mapping relationship with geographic location-time slot combinations;

[0011] The control target AMDQN interacts with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network;

[0012] Among them, the method for generating the basic AMDQN is as follows: based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, network topology information, and the number of computing platforms, a state space, an action space, and an optimization objective function are respectively constructed, and the historical number of instances to be deployed corresponding to lightweight AI services and microservices is determined; a reward function is constructed according to the optimization objective function and the historical number of instances to be deployed; an initial AMDQN is constructed based on the state space, action space, and reward function and trained to generate a basic AMDQN.

[0013] In a second aspect, an embodiment of the present application provides a hybrid orchestration device for AI services and microservices in cloud-edge collaboration, including:

[0014] An information acquisition module, which is used to obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot;

[0015] A resource determination module, configured to determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource amount threshold, and the operation intensity of the full-scale AI service;

[0016] A resource allocation module, which is used to allocate resources to the full-scale AI service of the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated;

[0017] An instance calculation module, which is used to determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request;

[0018] a model screening module, configured to screen out a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as a target AMDQN, wherein the orchestration strategy database includes a plurality of basic AMDQNs having a mapping relationship with a geographic location-time slot combination;

[0019] A hybrid orchestration module, which is used to control the target AMDQN to interact with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network;

[0020] The model generation module is used to construct the state space, action space and optimization objective function based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, network topology information, and the number of computing platforms, and determine the historical number of instances to be deployed corresponding to lightweight AI services and microservices; construct a reward function based on the optimization objective function and the historical number of instances to be deployed; construct an initial AMDQN based on the state space, action space and reward function and train it to generate a basic AMDQN.

[0021] In the third aspect, an embodiment of the present application provides a hybrid orchestration device for AI services and microservices in cloud-edge collaboration, wherein the hybrid orchestration device for AI services and microservices in cloud-edge collaboration includes a processor, a memory, and a hybrid orchestration program for AI services and microservices in cloud-edge collaboration stored on the memory and executable by the processor. When the hybrid orchestration program for AI services and microservices in cloud-edge collaboration is executed by the processor, the steps of the aforementioned hybrid orchestration method for AI services and microservices in cloud-edge collaboration are implemented.

[0022] The beneficial effects of the technical solutions provided in the embodiments of the present application include:

[0023] Obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, including target geographic location and target time slot; determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, the heterogeneous service resource threshold, and the operation intensity of the full-scale AI service; allocate resources to the full-scale AI service of the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated; determine the number of instances to be deployed corresponding to the lightweight AI service and microservice based on the target edge user request; and select the AMDQN corresponding to the target basic information as the target AMDQN from an orchestration strategy database containing multiple basic AMDQNs having a mapping relationship with geographic location-time slot combinations. The control target AMDQN interacts with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network; wherein, the generation method of the basic AMDQN is as follows: based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, network topology information, and the number of computing platforms, a state space, an action space, and an optimization objective function are constructed respectively, and the number of historical instances to be deployed corresponding to the lightweight AI services and microservices is determined; a reward function is constructed according to the optimization objective function and the number of historical instances to be deployed; an initial AMDQN is constructed based on the state space, action space, and reward function and trained to generate a basic AMDQN. Through this application, dynamic scaling adjustment of heterogeneous resources or instances of different types of services can be achieved to adapt to the edge user request intensity in different time slots or cities, and network resource load balancing is achieved through deep reinforcement learning AMDQN, while minimizing service response latency, thereby effectively realizing the hybrid orchestration of AI services and microservices in a heterogeneous environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of an embodiment of the method for hybrid orchestration of AI services and microservices in cloud-edge collaboration of this application;

[0025] Figure 2 This is a schematic diagram of the heterogeneous service orchestration network architecture involved in the embodiment of the present application;

[0026] Figure 3 Schematic diagram of the interaction framework between the AMDQN agent and the environment involved in the embodiment of the present application;

[0027] Figure 4 This is a schematic diagram of the AMDQN algorithm architecture involved in the embodiment of this application;

[0028] Figure 5 This is a schematic diagram of the hybrid orchestration of AI services and microservices involved in the embodiment of this application;

[0029] Figure 6 This is a schematic diagram of the hardware structure of the hybrid orchestration device of AI services and microservices in cloud-edge collaboration involved in the embodiment of this application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0031] First, some technical terms in this application are explained to facilitate those skilled in the art to understand this application.

[0032] AI Services: Services and applications that use AI technologies.

[0033] Full-scale AI services: AI services without model compression typically require a large amount of heterogeneous service resources and are generally used to process compute-intensive requests such as LLMs (Large Language Models) inference tasks.

[0034] Lightweight AI services: These are typically derived from full-size AI services through model compression techniques (such as quantization and pruning). Deploying lightweight AI services requires fewer resources, and to ensure service stability, lightweight AI services of the same type are generally derived using the same model compression method. Therefore, it is generally assumed that lightweight AI services of the same type consume the same heterogeneous service resources and have the same inference rate.

[0035] Microservice: A service unit with an independent code base, database, and runtime environment that can be deployed and upgraded independently. Each microservice is responsible for a single function or business logic, and interacts and collaborates with other services through an API (Application Programming Interface), ultimately forming a complete application to provide services to users.

[0036] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0037] In the first aspect, an embodiment of the present application provides a method for hybrid orchestration of AI services and microservices in cloud-edge collaboration.

[0038] In one embodiment, referring to Figure 1 , Figure 1 This is a flow chart of an embodiment of the hybrid orchestration method of AI services and microservices in cloud-edge collaboration of this application. Figure 1 As shown in the figure, the hybrid orchestration method of AI services and microservices in cloud-edge collaboration includes:

[0039] Step S10: Obtain target basic information and target edge user request corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot.

[0040] For example, it should be noted that the execution subject of this embodiment can be an AI service and microservice hybrid orchestration device with a heterogeneous service orchestration network architecture with functions such as data processing, network communication and program running, or other computer devices with similar functions, etc., which are not limited here.

[0041] It is understandable that the services in this embodiment include AI services and microservices, and AI services include full-size AI services and lightweight AI services; among them, the full-size AI service set is , Indicates the total number of full-scale AI services and deploys a full-scale AI service set The minimum heterogeneous service resource threshold is , 、 and Respectively represent and full-scale AI services The corresponding computing resource threshold, bandwidth resource threshold, and memory resource threshold, in addition to the full-scale AI service set The inference efficiency (i.e., the speed of serving requests) of ;The lightweight AI service set is , Indicates the total number of lightweight AI services, and deploys 1 lightweight AI service The amount of resources required is , 、 and Respectively represent and lightweight AI services The corresponding computing resource consumption, bandwidth resource consumption and memory resource consumption, and in order to ensure the stability of lightweight AI services, its inference efficiency To determine the value; the microservice set is , Indicates the total number of microservices. Microservices are mainly network functions of the open system interconnection application layer, such as firewall, traffic shaping, deep packet inspection, etc., and complete 1 microservice The amount of resources required for instantiation is , 、 and Representation and microservices The corresponding computing resource consumption, bandwidth resource consumption and memory resource consumption, in addition, microservices Service rate It is also a fixed value and obeys the exponential distribution. It should be noted that, for the sake of brevity of description, the meanings of parameters with the same meanings in the subsequent embodiments will not be repeated.

[0042] In addition, for heterogeneous service orchestration networks, this embodiment also defines resource allocation decision variables Serving full-scale AI Available computing resources and bandwidth resources; define service deployment decision variables The number of deployments of the i-th lightweight AI service and the i-th microservice on the n-th computing platform, respectively, and a binary decision variable is defined based on the service deployment decision variable Determine whether to deploy the i-th lightweight AI service and the i-th microservice on the n-th computing platform; define request routing decision variables Indicates that edge user requests are on the computing platform Services on After processing, it is routed to the computing platform Services on The routing probability of continuing processing.

[0043] See also Figure 2 As shown, it should be understood that the current heterogeneous service orchestration network architecture is a heterogeneous service orchestration network architecture composed of four network layers, namely user access layer, edge computing layer, cloud computing layer, and orchestration control layer. This architecture supports heterogeneous user requests, service types, service resources, and computing nodes. Among them, the user access layer is constructed by the edge user request set. It should be noted that the edge user request set is , Indicates the total number of types of all requests, and each request can be represented by a triple , Express a request The binary label of Representative Request is a latency-sensitive request, and It represents the request It is a computationally intensive request; Represents the completion of the request A set of services that need to be traversed in order may be a pure microservice chain or a mixed call chain containing microservices and AI services, such as ; For request arrival rate; it can be understood that edge user requests include the type of request, the composition of the request, the arrival rate of the request, the required AI service and microservice type, the service type preference and the number of concurrent requests, etc. The specific information contained therein can be determined according to actual needs and is not limited here.

[0044] This embodiment constructs an edge computing layer and a cloud computing layer based on the communication relationship between the computing platforms. Lightweight AI services and microservices are deployed through the edge computing platform in the edge computing layer, and a perception module is embedded in the edge computing layer to perceive edge user requests through the perception module and obtain perception results corresponding to the edge user requests. It should be noted that the implementation method and principle of how to obtain the perception results of edge user requests through the perception module are common knowledge in the field, and therefore, for the sake of brevity, they are not described here. It should be noted that since full-scale AI services have high requirements for heterogeneous service resources, and the cloud computing platform in the cloud computing layer has a large-scale computing cluster and advanced parallel processing hardware equipment, only the cloud computing platform can deploy full-scale AI services, and the full-scale AI services deployed on the cloud computing platform can handle computationally intensive requests such as big data analysis and LLMs reasoning tasks. In addition, since commercial AI services have very high requirements for service accuracy, such as financial analysis and atmospheric forecasting, using lightweight AI services to handle computationally intensive requests may result in significant economic losses. Therefore, this embodiment stipulates that full-scale AI services are deployed only on the cloud computing platform in the cloud computing layer, and there is only one of each full-scale AI service. The cloud computing platform and the edge computing platform can be connected via a high-latency public network, while different edge computing platforms can be connected via high-speed dedicated lines.

[0045] The orchestration control layer in this embodiment includes a time slot control system and an orchestration policy database, and the time slot control system further includes a time slot management module, a database access module, an agent selection module, and an execution and feedback module. Among them, the time slot management module is responsible for scheduling and managing system time slots to determine the current time period and the time slot status (idle, occupied, upcoming, etc.). The database access module obtains the current system time slot from the time slot management module and queries the orchestration policy database to select an appropriate AMDQN (i.e., deep Q network based on invalid action mask) agent. The agent selection module selects an appropriate AMDQN agent based on the AMDQN agent corresponding to the current time slot queried from the orchestration policy database and passes it to the subsequent execution and feedback module. Once the AMDQN agent is selected and executed, the execution and feedback module is responsible for monitoring its execution and providing feedback on the execution results (such as whether the agent execution is successful and whether the agent needs to be switched).

[0046] In this embodiment, for a current heterogeneous service orchestration network that requires hybrid orchestration of AI services and microservices, corresponding target edge user requests and target basic information, including target geographic location and target time slot, are obtained. The target edge user requests refer to all edge user requests received in the current heterogeneous service orchestration network. The target geographic location refers to the location of the current heterogeneous service network, such as the target geographic location being City A. The target time slot refers to the time corresponding to the current heterogeneous service network, such as the target time slot being aa:bb:00 am on xx / yy / zz. It should be understood that in this embodiment, a mapping relationship exists between geographic location and time slot and AMDQN, meaning that the AMDQN corresponding to the current heterogeneous service orchestration network can be screened out using geographic location and time slot.

[0047] Step S20: Determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service according to the target edge user request, the preset heterogeneous service resource amount threshold, and the operation intensity of the full-scale AI service.

[0048] For example, it should be understood that because service providers are constrained by the dual cost thresholds of CAPEX (Capital Expenditure) and OPEX (Operating Expense), their schedulable heterogeneous resource pools within the cloud computing platform are limited. This means that the limited heterogeneous service resources within the heterogeneous resource pool, such as bandwidth and computing resources, create heterogeneous service resource constraints. Therefore, to maximize resource utilization, this embodiment proposes a heterogeneous resource allocation algorithm (RAA). This algorithm uses the RAA algorithm to rationally allocate resources for full-scale AI services deployed on the cloud computing platform, maximizing the inference efficiency of full-scale AI services within the limited heterogeneous service resources. This embodiment implements the RAA algorithm within the cloud computing layer to allocate heterogeneous service resources for full-scale AI services, maximizing resource utilization.

[0049] Specifically, for the RAA algorithm, it needs to determine the request intensity of all edge users for full-scale AI services based on the perception results corresponding to the target edge user requests, and calculate the inference efficiency of the full-scale AI service based on the request intensity and the heterogeneous service resource threshold; then determine the priority of allocating heterogeneous service resources for each full-scale AI service through the inference efficiency, request intensity and operation intensity of the full-scale AI service, and finally determine the amount of heterogeneous service resources to be allocated for each full-scale AI service based on the priority. It should be noted that the heterogeneous service resource threshold includes but is not limited to the computing resource threshold, bandwidth resource threshold and memory resource threshold, and the specific value setting of the heterogeneous service resource threshold can be determined according to actual needs, which is not limited here.

[0050] Furthermore, in one embodiment, determining the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource amount threshold, and the operation intensity of the full-scale AI service includes:

[0051] Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network;

[0052] Calculating a first request intensity of the full-scale AI service based on the target perception result;

[0053] determining a first inference efficiency of the full-scale AI service based on the first request intensity;

[0054] Calculate a first heterogeneous service resource amount based on the first inference efficiency, a gain coefficient of bandwidth resources, a gain coefficient of computing resources, and an operation intensity of the full-scale AI service;

[0055] Calculating a second inference efficiency of the full-scale AI service based on a maximum value between the first heterogeneous service resource amount and a preset heterogeneous service resource amount threshold;

[0056] Determining a first priority for allocating heterogeneous service resources to each full-scale AI service based on the second inference efficiency, the first request intensity, and the operation intensity of the full-scale AI service;

[0057] Based on the first priority, the amount of heterogeneous service resources to be allocated for each full-size AI service is determined.

[0058] For example, in this embodiment, the target perception result corresponding to the target edge user request is first obtained through the current heterogeneous service orchestration network, and then the target perception result is substituted into the following full-scale AI service request intensity calculation formula to obtain the first request intensity of all edge users for each full-scale AI service:

[0059]

[0060] Where, Indicates full-scale AI services First request strength; binary label Indicates edge user requests Type, where Request on behalf of edge users is a latency-sensitive request, and It represents the edge user request is a computationally intensive request; binary variables Indicates the completion of a complete edge user request Is it necessary to go through full-scale AI service inference, where Representatives do not need, Representative needs; Indicates edge user requests arrival rate.

[0061] Then, to make full-scale AI services To meet the edge user request intensity, the first request intensity is used as the first inference efficiency of the full-scale AI service. Under the premise of meeting the heterogeneous service resource constraints of the cloud computing platform, the RAA algorithm will allocate bandwidth resources and computing resources that meet the minimum requirements for each full-scale AI service. Specifically, the first inference efficiency, the gain coefficient of bandwidth resources, the gain coefficient of computing resources, and the operation intensity of the full-scale AI service are substituted into the following inference efficiency formula to solve the first heterogeneous service resource amount:

[0062]

[0063] Where, Indicates the first inference efficiency of full-scale AI services, Represent the gain coefficients of bandwidth resources and computing resources respectively, represent the bandwidth resource amount and computing resource amount in the first heterogeneous service resource amount respectively, Indicates full-scale AI services The operational intensity reflects the computational intensity of full-scale AI services relative to memory access.

[0064] Then, the size between the first heterogeneous service resource amount and the heterogeneous service resource amount threshold is determined, and the maximum value of the two is substituted into the above-mentioned inference efficiency formula to recalculate the second inference efficiency full_AI_infer_eff of each full-size AI service; for example, if the bandwidth resource amount in the first heterogeneous service resource amount is greater than the bandwidth resource amount threshold, the bandwidth resource amount in the first heterogeneous service resource amount is substituted into the above-mentioned inference efficiency formula; otherwise, the bandwidth resource amount threshold is substituted into the above-mentioned inference efficiency formula; similarly, if the computing resource amount in the first heterogeneous service resource amount is greater than the computing resource amount threshold, the computing resource amount in the first heterogeneous service resource amount is substituted into the above-mentioned inference efficiency formula; otherwise, the computing resource amount threshold is substituted into the above-mentioned inference efficiency formula.

[0065] Then, the difference between each second inference efficiency full_AI_infer_eff and the first request intensity is recorded as infer_eff_equal, and the difference sequence infer_eff_equal_list can be obtained. For example, assuming there are 4 full-size AI services to And the corresponding infer_eff_equal_list=[1,0,3,0]; and deduplicate the infer_eff_equal_list and sort it in ascending order to get infer_eff_line_list=[0,1,3]; then, calculate the difference between the maximum value and each value in the infer_eff_line_list to get infer_eff_lack_list=[3,2,0]; the RAA algorithm counts the full-size AI services corresponding to each value in the infer_eff_line_list and sorts them in descending order of the operation intensity of the full-size AI services as line_corr_full_AI, for example, assuming , then line_corr_full_AI= , that is, the ranking in line_corr_full_AI represents the priority of different full-size AI services.

[0066] Finally, the corresponding heterogeneous service resources in infer_eff_lack_list are allocated to the full-size AI services in line_corr_full_AI in turn until the remaining resources in the cloud computing platform are exhausted or the values ​​in infer_eff_equal_list are equal, indicating that the process of allocating resources to the full-size AI service is completed, thereby obtaining the amount of heterogeneous service resources to be allocated for each full-size AI service.

[0067] Step S30: Allocate resources for the full-scale AI service of the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated.

[0068] For example, in this embodiment, after determining the amount of heterogeneous service resources to be allocated for each full-size AI service, resources will be allocated to the full-size AI services of the cloud computing layer in the current heterogeneous service orchestration network according to the amount of heterogeneous service resources to be allocated for each full-size AI service, so as to reasonably allocate computing resources and bandwidth resources to the full-size AI services, and then realize the orchestration of full-size AI services in a heterogeneous environment, thereby ensuring the best inference efficiency.

[0069] Step S40: Determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request.

[0070] For example, it is understandable that, since lightweight AI services and microservices have relatively low demand for resources, they can be distributedly deployed on multiple edge computing platforms in a multi-instance manner. Among them, this embodiment proposes a service instance calculation algorithm ICA to determine the number of instances to be deployed for lightweight AI services and microservices; specifically, for the ICA algorithm, it needs to determine the request intensity of all edge users for lightweight AI services and microservices based on the perception results corresponding to the target edge user requests, and determine the basic number of instances of lightweight AI services and microservices through the request intensity; then determine the priority of increasing the number of instances for lightweight AI services and microservices through the basic number of instances; finally, based on the priority of increasing the number of instances, determine the number of instances to be deployed that need to be increased for lightweight AI services and microservices to meet high-concurrency user requests.

[0071] Furthermore, in one embodiment, determining the number of to-be-deployed instances corresponding to the lightweight AI service and microservice according to the target edge user request includes:

[0072] Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network;

[0073] Calculate the second request intensity of all edge users for each lightweight AI service and the third request intensity of all edge users for each microservice based on the target perception result;

[0074] Determine a first basic number of instances of each lightweight AI service based on the second request intensity, and determine a second basic number of instances of each microservice based on the third request intensity;

[0075] Determine a second priority for increasing the number of instances for each lightweight AI service based on the first basic number of instances, and determine a number of instances to be deployed for each lightweight AI service based on the first basic number of instances and the second priority;

[0076] The maximum delay reduction for adding one instance for each microservice is determined based on the second basic number of instances, and the third priority for increasing the number of instances for each microservice is determined based on the sorting result of the maximum delay reduction. The number of instances to be deployed for each microservice is determined based on the second basic number of instances and the third priority.

[0077] For example, in this embodiment, the target perception result corresponding to the target edge user request is first obtained through the current heterogeneous service orchestration network, and then the target perception result is substituted into the following lightweight AI service request intensity calculation formula and microservice request intensity calculation formula to obtain the second request intensity of all edge users for each lightweight AI service and the third request intensity for each microservice:

[0078]

[0079] Where, Represents lightweight AI services Second request strength; binary label Indicates edge user requests Type of; binary variable Indicates the completion of a complete edge user request Whether lightweight AI service reasoning is required; Representing microservices The third request strength; integer variable For a complete edge user request Need to go through microservices Number of times processed.

[0080] Then the ICA algorithm will determine the basic number of instances of each lightweight AI service and each microservice based on the service intensity in the steady-state queuing network. Second request intensity and inference efficiency Substitute the following service instance calculation formula for lightweight AI services to calculate the first basic instance number of each lightweight AI service, and then obtain the basic instance number sequence of lightweight AI services light_AI_req_list:

[0081]

[0082] Where, Represents lightweight AI services The first base instance number.

[0083] Similarly, each microservice Third request intensity and inference efficiency Substitute the following microservice service instance calculation formula to calculate the second basic instance number of each microservice, and then obtain the basic instance number sequence ms_req_list of the microservice:

[0084]

[0085] Where, Representing microservices The number of second base instances.

[0086] Next, the ICA algorithm sorts the number of basic instances in light_AI_req_list in descending order to obtain the priority list of lightweight AI services light_AI_prio_list. The order of light_AI_prio_list represents the priority of increasing the number of instances for different lightweight AI services. Assume there are four lightweight AI services to If light_AI_req_list=[3,2,5,6], then sort the service in descending order based on the number of basic instances of each lightweight AI service to obtain light_AI_prio_list= ; Then, under the premise of meeting the heterogeneous service resource constraints of the cloud computing platform, the ICA algorithm will increase the number of instances of each lightweight AI service in light_AI_prio_list by 1 in turn until the constraints of heterogeneous service resources are no longer met, thereby obtaining the number of instances to be deployed for each lightweight AI service.

[0087] To improve the effectiveness of services deployed in a multi-instance manner, the ICA algorithm calculates the maximum latency reduction for each microservice by adding one instance. The second base instance number Substitute the following microservice priority calculation formula to obtain the score corresponding to the maximum delay reduction of each microservice, and then obtain the microservice score sequence ms_score_list:

[0088]

[0089] Where, Indicates deployment in the current heterogeneous service orchestration network microservices The service queue delay corresponding to the instance of ms_score_list is shown in Figure 2. It can be understood that the larger the score in ms_score_list, the higher the priority of increasing the number of instances of the corresponding microservice. Next, the ICA algorithm will add one instance to the microservice with the highest score in ms_score_list, recalculate the score of this microservice, and repeat the above process until the resource constraint is no longer met, indicating that the calculation process of the microservice instance is completed, thereby obtaining the number of instances to be deployed for each microservice.

[0090] Specifically, assume there are 3 types of microservices And the basic number of instances of each microservice calculated according to the service instance calculation formula of the microservice is , then according to the microservice priority calculation formula, the maximum delay reduction for adding one instance to each microservice is calculated as (i.e., the score) ; It can be seen that it is a microservice Adding one instance can minimize the delay. Therefore, if the remaining resources in the current heterogeneous service orchestration network are only enough to deploy one instance, this embodiment should choose microservices. Add 1 instance, then there are 3 microservices The number of instances of ; Since in the previous step, this embodiment only updated the microservice So next we just need to recalculate the number of microservices Assuming that the maximum delay reduction when the number of instances increases from 4 to 5 is 0.25 (that is, the recalculated microservice The score of the three microservices is 0.25), then the scores of the three microservices become This means that if the remaining resources of the current heterogeneous service orchestration network can deploy another instance, microservices will be given priority. Add an instance; repeat the above process until the remaining resources in the network can no longer support the addition of any other microservice instance, and then determine the number of instances to be deployed for each microservice.

[0091] Step S50: Filtering out a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as a target AMDQN, wherein the orchestration strategy database contains multiple basic AMDQNs having a mapping relationship with geographic location-time slot combinations; wherein the basic AMDQN is generated by: constructing a state space, an action space, and an optimization objective function based on historical edge user requests, service deployment information in a basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, network topology information, and the number of computing platforms, and determining the historical number of instances to be deployed corresponding to lightweight AI services and microservices; constructing a reward function based on the optimization objective function and the historical number of instances to be deployed; constructing an initial AMDQN based on the state space, action space, and reward function and performing training to generate a basic AMDQN.

[0092] Exemplarily, in this embodiment, different basic AMDQN agents are constructed for different geographic location-time slot combinations, and the basic AMDQN agents and the mapping relationship between the basic AMDQN agents and the geographic location-time slot combinations are stored in the orchestration policy database, so that the time slot control system filters out the basic AMDQN agent corresponding to the combination of the target geographic location and the target time slot from the orchestration policy database as the target AMDQN agent, so as to maximize the cumulative return through continuous interaction and repeated iteration of the target AMDQN agent with the heterogeneous service orchestration network. The ultimate goal is to minimize the average response delay of all edge user requests without violating resource, routing, service and other constraints, while balancing the service resource load of the heterogeneous service orchestration network.

[0093] Among them, see Figure 3 As shown, the state space, action space and optimization objective function of the AMDQN agent are determined based on the service deployment information, network topology information, number of computing platforms and perception results of historical edge user requests in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and the historical number of instances to be deployed corresponding to lightweight AI services and microservices is determined through the ICA algorithm; then the reward function can be constructed according to the optimization objective function and the historical number of instances to be deployed, and then the initial AMDQN agent is constructed and trained through the state space, action space and reward function until it converges to generate the basic AMDQN agent corresponding to the preset geographic location-time slot combination and transfer it to the orchestration strategy database.

[0094] Among them, traditional methods often separate and optimize service deployment and request routing when implementing hybrid orchestration, resulting in the inability to fully utilize global resources for joint decision-making, further restricting network performance. In this embodiment, during the initial AMDQN agent training process, the request routing probability of the AMDQN agent is also determined based on the service deployment status of the underlying heterogeneous service orchestration network. Based on the communication latency between computing platforms and the degree of clustering of service instances on the computing platform, the request routing probability is weighted differently. Historical edge user requests are forwarded from the parent computing platform to the next child computing platform, so as to plan a reasonable service path for each edge user request, thereby meeting the user demand for low latency.

[0095] See also Figure 3 and Figure 4 As shown in Figure 1, the AMDQN agent continuously interacts with the heterogeneous service orchestration network to gain learning experience and then updates the network structure and weight parameters. Among them, AMDQN mainly consists of two Q networks designed based on the duel network architecture, namely the online Q network and the With the target Q network The former is responsible for generating the Q value of each action in the current state, guiding the action output and gradually updating its parameters during training; the latter provides a stable Q value estimate of the next state, which is used to calculate the TD target (Temporal Difference Target) to update the online Q network, thereby reducing fluctuations during training. It should be noted that Figure 3 and Figure 4 The network environment in this example is a network environment composed of a hybrid orchestration of AI services and microservices. Figure 5 The network environment shown is composed of a hybrid orchestration of AI services and microservices.

[0096] The following examples will be combined Figure 4 The detailed process of the invalid action mask deep Q network (AMDQN) algorithm is explained:

[0097] Before each training session, the online Q network With the target Q network The weight parameters of the heterogeneous service orchestration network will be randomly initialized, and at the initial stage of each training round, the heterogeneous service orchestration network will also be initialized; then, the AMDQN agent will execute the initial solution of service deployment, that is, deploy one instance of each service in the heterogeneous service orchestration network in turn to ensure the basic service capabilities of the heterogeneous service orchestration network; and as the heterogeneous service orchestration network changes with the training process, the invalid action mask in this state It will be calculated first; it should be understood that the invalid action mask is based on the Boolean operation mask, which significantly reduces the search space of the AMDQN agent by masking each deployment operation that does not meet the service resource constraints, making the AMDQN model converge faster; the AMDQN agent is based on probability Randomly explore the remaining valid actions and perform actions Get the transfer sample (state ,action , next state , whether the current round of training is completed ) and stored in the replay buffer Once the learning conditions are met, AMDQN will start Randomly select (Batch) size of transfer samples and their corresponding importance sampling weights , calculate their target Q values ​​respectively and the current Q value , and then perform gradient descent on the loss function to update the online Q network parameters , and finally the target Q network parameters Perform a soft update.

[0098] After multiple rounds of iterative training, it is necessary to evaluate the performance of the algorithm. This can be done by initializing the environment and selecting deployment actions determined by the greedy strategy in multiple steps of interaction. , to obtain the optimal AI service and microservice hybrid orchestration strategy that the current algorithm can obtain; if the convergence judgment conditions are met, this AMDQN pre-trained model (that is, the basic AMDQN, also known as the AMDQN agent) will be saved to the orchestration strategy database, and it will serve as a control agent to provide the optimal solution to the AI ​​service and microservice hybrid orchestration problem in the heterogeneous service orchestration network in the corresponding time slot.

[0099] Furthermore, in one embodiment, the state space, action space, and optimization objective function are constructed based on historical edge user requests, service deployment information and network topology information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and the number of computing platforms, respectively, including:

[0100] Constructing a state space based on the historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and network topology information;

[0101] Constructing an action space based on the number of computing platforms in the underlying heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination;

[0102] An optimization objective function is constructed based on historical edge user requests and service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination.

[0103] The reward function is expressed as follows:

[0104]

[0105] Where, Indicates the In the training round The reward value of each time step, Indicates the total number of instances of the service to be deployed. Indicates the In the training round A comprehensive measure of the performance of heterogeneous service orchestration networks in time steps. Represent the weight parameters of the impact of time steps and training rounds on iterative rewards; Indicates edge user requests The average service delay, Represents the load balancing index of the heterogeneous service orchestration network, They represent the weight parameters of the average service delay and load balancing index on the optimization objective function.

[0106] For example, in this embodiment, the average service delay of historical edge user requests is determined by the perception results corresponding to historical edge user requests and the service deployment information in the basic heterogeneous service orchestration network. (i.e., average response latency) and the load balancing index of the underlying heterogeneous service orchestration network ; Then based on the average service delay of historical edge user requests and the load balancing index of the underlying heterogeneous service orchestration network Determine the optimization objective function; the expression of the optimization objective function is:

[0107]

[0108] Where, Request for History Edge users The average service delay, The load balancing index for the underlying heterogeneous service orchestration network, are weight parameters of the influence of average service delay and load balancing index on the optimization objective function, and their specific values ​​can be determined according to actual needs and are not limited here.

[0109] This embodiment will define the state space ,It should be explained that the state space of AMDQN can be described as the ,sum of two parts, namely the network resource states of the ,heterogeneous service orchestration network and the ,network resource states of the heterogeneous service orchestration network. S resource Service deployment status S deployment , that is, the state space of AMDQN is defined as ; Among them, the network resource status is , which includes the remaining heterogeneous service resources of each computing platform and occupancy rate ; The service deployment status is , respectively, are the distribution of various deployed services on the computing platform And the location of the computing platform where the current service is deployed in the heterogeneous service orchestration network ; For example, suppose there are 3 computing platforms in the current network , then the communication delay matrix between computing platforms is:

[0110]

[0111] If the computing platform of the current service deployment is , then calculate The sum of the communication delays to any next computing platform is , so we can get The normalized communication delays to any next computing platform are ,therefore .

[0112] Based on the state space defined above, this embodiment will construct a state space corresponding to the basic heterogeneous service orchestration network based on historical edge user requests and service deployment information and network topology information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination.

[0113] Next, we define the action space It is understandable that each step of AMDQN is to deploy a specific service on the selected computing platform. In fact, for any service to be deployed, AMDQN always obtains a one-dimensional probability list based on the state of the environment before deploying its instance based on the state of the heterogeneous service orchestration network. Specifically, it can be:

[0114]

[0115] Where, Represents a computing platform The probability of being selected for deploying the service, AMDQN is based on The strategy ultimately determines the computing platform used to deploy the service; it is worth noting that due to the invalid action mask mechanism in AMDQN, AMDQN outputs Before the meeting Filter out computing platforms that do not have sufficient resources for service deployment and assign them to Set to 0 and evenly distribute to other computing platforms; for example, there are 3 computing platforms in the current network And the corresponding , assuming that the invalid action mask mechanism is used to determine If there are insufficient resources for service deployment, Set to 0 and the original The corresponding 0.6 is evenly divided into and , then we get a new And output.

[0116] In addition, once the service is deployed successfully, AMDQN will immediately update the routing percentage of each service in the heterogeneous service orchestration network according to the routing probability update formula and the different weights of the communication delay between computing platforms and the degree of clustering of service instances on the computing platform on the request routing probability, so as to obtain the following: Figure 5The routing probability table shown in FIG is used to forward user requests from the parent computing platform to the next child computing platform according to the routing probability table. It can be seen that this embodiment uses the AMDQN control agent designed based on deep reinforcement learning to jointly optimize service deployment and request routing, thereby balancing network resource load while minimizing service response latency.

[0117] Among them, the routing probability update formula is:

[0118]

[0119] Where, Indicates that edge user requests are on the computing platform Services on After processing, it is routed to the computing platform Services on The routing probability of continuing processing, Represents the collection of full-size AI services, lightweight AI services, and microservices, respectively. Indicates that from the computing platform arrive The communication delay, Orchestrate all deployed services in the network for heterogeneous services A collection of computing platforms, Represents a computing platform Services deployed on the number of Indicates that at time step Deployed services the number of A weight parameter representing the impact of the degree of aggregation of communication delays between computing platforms on the request routing probability; A weight parameter representing the impact of the degree of clustering of service instances on the computing platform on the request routing probability; Represents a computing platform Is there a service deploy.

[0120] Based on the action space defined above, this embodiment constructs an action space corresponding to the basic heterogeneous service orchestration network based on the number of computing platforms in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination.

[0121] Define the reward function , the reward function of AMDQN describes the feedback obtained from the environment for each interaction between the AMDQN agent and the heterogeneous service orchestration network. However, in the hybrid orchestration problem of AI services and microservices, the comprehensive performance of the heterogeneous service orchestration network can only be accurately evaluated after resource allocation and service deployment are completed. This means that the comprehensive performance of the heterogeneous service orchestration network evaluated using AMDQN's rewards is only accurate in the last step. Therefore, in order to maximize the accuracy of each step of the evaluation, this embodiment adopts a new iterative reward shaping method (i.e., sparse reward shaping), which iteratively shapes the iterative rewards of different training stages, thereby improving training efficiency; wherein, the expression of the reward function is:

[0122]

[0123] Where, Indicates the In the training round The reward value of each time step, Indicates the total number of instances of the service to be deployed. Indicates the In the training round A comprehensive measure of the performance of heterogeneous service orchestration networks in time steps. Represent the weight parameters of the impact of time steps and training rounds on iterative rewards; Indicates edge user requests The average service delay, Represents the load balancing index of the heterogeneous service orchestration network, They represent the weight parameters of the average service delay and load balancing index on the optimization objective function.

[0124] It can be seen that based on the reward function defined above, this embodiment will construct a reward function corresponding to the basic heterogeneous service orchestration network according to the optimization objective function and the historical number of instances to be deployed.

[0125] Furthermore, in one embodiment, constructing an optimization objective function based on historical edge user requests and service deployment information in a basic heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination includes:

[0126] Input historical edge user requests into the underlying heterogeneous service orchestration network to obtain historical perception results;

[0127] Calculating the average service delay corresponding to historical edge user requests based on the historical perception results;

[0128] Calculating a load balancing index corresponding to the basic heterogeneous service orchestration network through service deployment information in the basic heterogeneous service orchestration network;

[0129] An optimization objective function is determined based on the average service delay and the load balancing index.

[0130] The calculation formula for the average service delay is:

[0131]

[0132] Where, Indicates edge user requests The average service delay, Indicates edge user requests The set of all request paths, Indicates edge user requests No. service paths, Indicates the computing platforms Is there any AI service on the deploy, Indicates the computing platforms Are there microservices? deploy, Indicates edge user requests exist No. computing platforms AI services The queue delay on Indicates edge user requests exist No. computing platforms Microservices The processing delay on Indicates that edge user requests are on the computing platform AI services on After processing, it is routed to the computing platform AI services on The routing probability of continuing processing, Indicates edge user requests The arrival rate, Indicates the completion of a complete edge user request Whether it needs to go through full-scale AI service inference, Indicates the completion of a complete edge user request Is lightweight AI service reasoning required? Indicates the unit time slot length, Indicates full-scale AI services The operating strength, Represents lightweight AI services The operating strength, Indicates full-scale AI services The inference efficiency, Represents lightweight AI services The inference efficiency, Indicates edge user requests Divide into a complete service path The arrival rate on Indicates edge user requests exist No. computing platforms AI services corresponding to the service chain The arrival rate on express The average communication latency when processing across computing platforms.

[0133] The calculation formula of the load balancing index is:

[0134]

[0135] Where, The load balancing index for the underlying heterogeneous service orchestration network, They represent the sensitivity coefficients of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. They represent the variance of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. Indicates the total number of computing platforms, res Indicates computing resource utilization, bandwidth resource utilization, or memory resource utilization. represents the average utilization of heterogeneous service resources, Represents a computing platform The utilization rate of heterogeneous service resources.

[0136] For example, in this embodiment, the historical perception results corresponding to historical edge user requests are first obtained through the historical heterogeneous service orchestration network, and then the historical perception results are substituted into the following average service delay calculation formula to obtain the average service delay of historical edge user requests:

[0137]

[0138] Where, Indicates edge user requests The average service delay, Indicates edge user requests The set of all request paths, Indicates edge user requests No. service paths, Indicates the computing platforms Is there any AI service on the deploy, Indicates the computing platforms Are there microservices? deploy, Indicates edge user requests exist No. computing platforms AI services The queue delay on Indicates edge user requests exist No. computing platforms Microservices The processing delay on Indicates that edge user requests are on the computing platform AI services on After processing, it is routed to the computing platform AI services on The routing probability of continuing processing, Indicates edge user requests The arrival rate, Indicates the completion of a complete edge user request Whether it needs to go through full-scale AI service inference, Indicates the completion of a complete edge user request Is lightweight AI service reasoning required? Indicates the unit time slot length, Indicates full-scale AI services The operating strength, Represents lightweight AI services Operation intensity, which reflects the intensity of lightweight AI service computation relative to memory access. Indicates full-scale AI services The inference efficiency can be calculated by the RAA algorithm; Represents lightweight AI services The inference efficiency, Indicates edge user requests Divide into a complete service path The arrival rate on Indicates edge user requests exist No. computing platforms Corresponding service chain No. Services The arrival rate on Refers to full-scale AI services Or lightweight AI services Located in the service chain No. services; express The average communication latency when processing across computing platforms.

[0139] Next, the service deployment information in the basic heterogeneous service orchestration network is substituted into the following load balancing index calculation formula to calculate the load balancing index of the basic heterogeneous service orchestration network:

[0140]

[0141] Where, The load balancing index for the underlying heterogeneous service orchestration network, They represent the sensitivity coefficients of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. They represent the variance of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. Indicates the total number of computing platforms, res Represents heterogeneous service resources, i.e. , that is, it is the computing resource occupancy rate, bandwidth resource occupancy rate or memory resource occupancy rate, Representing heterogeneous service resources The average utilization rate, Represents a computing platform Heterogeneous service resources utilization rate.

[0142] Finally, based on the average service delay of historical edge user requests and the load balancing index of the underlying heterogeneous service orchestration network The optimization objective function can be determined.

[0143] Step S60: Control the target AMDQN to interact with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network.

[0144] Exemplarily, in this embodiment, a suitable target AMDQN agent is selected based on the time slot control system to continuously interact with the current heterogeneous service orchestration network until the hybrid orchestration of lightweight AI services and microservices is completed, that is, lightweight AI services and microservices corresponding to the number of instances to be deployed are deployed in the current heterogeneous service orchestration network, thereby effectively realizing the hybrid orchestration of AI services and microservices in a heterogeneous computing network such as user requests, service types, service resources, and computing nodes.

[0145] It can be seen that compared with existing service orchestration technologies that mostly only consider the deployment of a single microservice or AI service, and split service deployment and request routing as independent problems and then optimize them separately, this embodiment splits the hybrid orchestration process of AI services and microservices and designs a three-stage algorithm based on heuristic and reinforcement learning methods to solve the hybrid orchestration problem of AI services and microservices in heterogeneous service orchestration network architecture; wherein, the three-stage algorithm consists of heuristic algorithms including the RAA algorithm and the ICA algorithm and the AMDQN reinforcement learning algorithm; the RAA algorithm is used to complete the resource allocation of full-size AI services on the cloud computing platform, the ICA algorithm is used to determine the deployment number of lightweight AI services and microservices, and the AMDQN algorithm deploys each service in turn on different computing platforms in the network according to the deployment number determined by ICA, and updates the routing probability of each computing platform after each service is deployed. In summary, the heterogeneous service orchestration network architecture in this embodiment provides technical support for the hybrid orchestration of AI services and microservices. The RAA algorithm and the ICA algorithm realize dynamic scaling adjustment of heterogeneous resources or instances of different types of services to adapt to the request intensity of edge users in different time slots or cities. The AMDQN control agent designed based on deep reinforcement learning jointly optimizes service deployment and request routing to balance the network resource load while minimizing the service response delay, ultimately solving the problem of hybrid orchestration of AI services and microservices in the heterogeneous service orchestration network architecture.

[0146] On the second aspect, an embodiment of the present application also provides a hybrid orchestration device of AI services and microservices in cloud-edge collaboration.

[0147] In one embodiment, a device for hybrid orchestration of AI services and microservices in cloud-edge collaboration includes:

[0148] An information acquisition module, which is used to obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot;

[0149] A resource determination module, configured to determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource amount threshold, and the operation intensity of the full-scale AI service;

[0150] A resource allocation module, which is used to allocate resources to the full-scale AI service of the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated;

[0151] An instance calculation module, which is used to determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request;

[0152] a model screening module, configured to screen out a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as a target AMDQN, wherein the orchestration strategy database includes a plurality of basic AMDQNs having a mapping relationship with a geographic location-time slot combination;

[0153] A hybrid orchestration module, which is used to control the target AMDQN to interact with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network;

[0154] The model generation module is used to construct the state space, action space and optimization objective function based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, network topology information, and the number of computing platforms, and determine the historical number of instances to be deployed corresponding to lightweight AI services and microservices; construct a reward function based on the optimization objective function and the historical number of instances to be deployed; construct an initial AMDQN based on the state space, action space and reward function and train it to generate a basic AMDQN.

[0155] Furthermore, in one embodiment, the resource determination module is specifically configured to:

[0156] Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network;

[0157] Calculating a first request intensity of the full-scale AI service based on the target perception result;

[0158] determining a first inference efficiency of the full-scale AI service based on the first request intensity;

[0159] Calculate a first heterogeneous service resource amount based on the first inference efficiency, a gain coefficient of bandwidth resources, a gain coefficient of computing resources, and an operation intensity of the full-scale AI service;

[0160] Calculating a second inference efficiency of the full-scale AI service based on a maximum value between the first heterogeneous service resource amount and a preset heterogeneous service resource amount threshold;

[0161] Determining a first priority for allocating heterogeneous service resources to each full-scale AI service based on the second inference efficiency, the first request intensity, and the operation intensity of the full-scale AI service;

[0162] Based on the first priority, the amount of heterogeneous service resources to be allocated for each full-size AI service is determined.

[0163] Furthermore, in one embodiment, the instance calculation module is specifically configured to:

[0164] Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network;

[0165] Calculate the second request intensity of all edge users for each lightweight AI service and the third request intensity of all edge users for each microservice based on the target perception result;

[0166] Determine a first basic number of instances of each lightweight AI service based on the second request intensity, and determine a second basic number of instances of each microservice based on the third request intensity;

[0167] Determine a second priority for increasing the number of instances for each lightweight AI service based on the first basic number of instances, and determine a number of instances to be deployed for each lightweight AI service based on the first basic number of instances and the second priority;

[0168] The maximum delay reduction for adding one instance for each microservice is determined based on the second basic number of instances, and the third priority for increasing the number of instances for each microservice is determined based on the sorting result of the maximum delay reduction. The number of instances to be deployed for each microservice is determined based on the second basic number of instances and the third priority.

[0169] Furthermore, in one embodiment, the model generation module is specifically configured to:

[0170] Constructing a state space based on the historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and network topology information;

[0171] Constructing an action space based on the number of computing platforms in the underlying heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination;

[0172] An optimization objective function is constructed based on historical edge user requests and service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination.

[0173] Furthermore, in one embodiment, the model generation module is further configured to:

[0174] Input historical edge user requests into the underlying heterogeneous service orchestration network to obtain historical perception results;

[0175] Calculating the average service delay corresponding to historical edge user requests based on the historical perception results;

[0176] Calculating a load balancing index corresponding to the basic heterogeneous service orchestration network through service deployment information in the basic heterogeneous service orchestration network;

[0177] An optimization objective function is determined based on the average service delay and the load balancing index.

[0178] Furthermore, in one embodiment, the calculation formula for the average service delay is:

[0179]

[0180] Where, Indicates edge user requests The average service delay, Indicates edge user requests The set of all request paths, Indicates edge user requests No. service paths, Indicates the computing platforms Is there any AI service on the deploy, Indicates the computing platforms Are there microservices? deploy, Indicates edge user requests exist No. computing platforms AI services The queue delay on Indicates edge user requests exist No. computing platforms Microservices The processing delay on Indicates that edge user requests are on the computing platform AI services on After processing, it is routed to the computing platform AI services on The routing probability of continuing processing, Indicates edge user requests The arrival rate, Indicates the completion of a complete edge user request Whether it needs to go through full-scale AI service inference, Indicates the completion of a complete edge user request Is lightweight AI service reasoning required? Indicates the unit time slot length, Indicates full-scale AI services The operating strength, Represents lightweight AI services The operating strength, Indicates full-scale AI services The inference efficiency, Represents lightweight AI services The inference efficiency, Indicates edge user requests Divide into a complete service path The arrival rate on Indicates edge user requests exist No. computing platforms AI services corresponding to the service chain The arrival rate on express The average communication latency when processing across computing platforms.

[0181] Furthermore, in one embodiment, the calculation formula of the load balancing index is:

[0182]

[0183] Where, The load balancing index for the underlying heterogeneous service orchestration network, They represent the sensitivity coefficients of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. They represent the variance of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. Indicates the total number of computing platforms, res Indicates computing resource utilization, bandwidth resource utilization, or memory resource utilization. represents the average utilization of heterogeneous service resources, Represents a computing platform The utilization rate of heterogeneous service resources.

[0184] Furthermore, in one embodiment, the reward function is expressed as:

[0185]

[0186] Where, Indicates the In the training round The reward value of each time step, Indicates the total number of instances of the service to be deployed. Indicates the In the training round A comprehensive measure of the performance of heterogeneous service orchestration networks in time steps. Represent the weight parameters of the impact of time steps and training rounds on iterative rewards; Indicates edge user requests The average service delay, Represents the load balancing index of the heterogeneous service orchestration network, They represent the weight parameters of the average service delay and load balancing index on the optimization objective function.

[0187] Among them, the functional implementation of each module in the above-mentioned AI service and microservice hybrid orchestration device in cloud-edge collaboration corresponds to the steps in the above-mentioned embodiment of the AI ​​service and microservice hybrid orchestration method in cloud-edge collaboration, and its functions and implementation process will not be repeated here one by one.

[0188] On the third aspect, an embodiment of the present application provides a hybrid orchestration device for AI services and microservices in cloud-edge collaboration. The hybrid orchestration device for AI services and microservices in cloud-edge collaboration can be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0189] Reference Figure 6 , Figure 6 This is a hardware structure diagram of the hybrid orchestration device for AI services and microservices in cloud-edge collaboration involved in the embodiment of this application. In the embodiment of this application, the hybrid orchestration device for AI services and microservices in cloud-edge collaboration may include a processor, a memory, a communication interface, and a communication bus.

[0190] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0191] Communication interfaces include input / output (I / O), physical, and logical interfaces, which interconnect devices within the hybrid orchestration of AI services and microservices in cloud-edge collaboration. They also interconnect devices with other devices (such as other computing devices or user devices) in cloud-edge collaboration. Physical interfaces can include Ethernet, fiber, and ATM interfaces; user devices can include displays and keyboards.

[0192] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0193] The processor may be a general-purpose processor that can call a hybrid orchestration program for AI services and microservices in cloud-edge collaboration stored in a memory and execute the hybrid orchestration method for AI services and microservices in cloud-edge collaboration provided in an embodiment of the present application. For example, the general-purpose processor may be a central processing unit (CPU). The method executed when the hybrid orchestration program for AI services and microservices in cloud-edge collaboration is called can refer to the various embodiments of the hybrid orchestration method for AI services and microservices in cloud-edge collaboration in the present application and will not be repeated here.

[0194] It will be understood by those skilled in the art that Figure 6 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0195] In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0196] The readable storage medium of the present application stores a hybrid orchestration program of AI services and microservices in cloud-edge collaboration, wherein when the hybrid orchestration program of AI services and microservices in cloud-edge collaboration is executed by the processor, the steps of the hybrid orchestration method of AI services and microservices in cloud-edge collaboration as described above are implemented.

[0197] Among them, the method implemented when the hybrid orchestration program of AI services and microservices in cloud-edge collaboration is executed can refer to the various embodiments of the hybrid orchestration method of AI services and microservices in cloud-edge collaboration in this application, and will not be repeated here.

[0198] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0199] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0200] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0201] In some processes described in the embodiments of this application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of this application or may be executed in parallel. The sequence numbers of the operations are only used to distinguish different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0202] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of this application.

[0203] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A hybrid orchestration method for AI services and microservices in cloud-edge collaboration, characterized by: The following steps are involved: Obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot; Determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource threshold, and the operation intensity of the full-scale AI service; Allocate resources for full-scale AI services at the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated; Determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request; Filtering a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as the target AMDQN, wherein the orchestration strategy database includes a plurality of basic AMDQNs having a mapping relationship with geographic location-time slot combinations; The control target AMDQN interacts with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network; The basic AMDQN generation method is as follows: based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination, network topology information, and the number of computing platforms, a state space, an action space, and an optimization objective function are constructed respectively, and the number of historical instances to be deployed corresponding to lightweight AI services and microservices is determined; a reward function is constructed according to the optimization objective function and the number of historical instances to be deployed; An initial AMDQN is constructed based on the state space, action space, and reward function and trained to generate the base AMDQN.

2. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 1 is characterized in that: The determining, according to the target edge user request, a preset heterogeneous service resource amount threshold and the operation intensity of the full-scale AI service, the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service includes: Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network; Calculating a first request intensity of the full-scale AI service based on the target perception result; determining a first inference efficiency of the full-scale AI service based on the first request intensity; Calculate a first heterogeneous service resource amount based on the first inference efficiency, a gain coefficient of bandwidth resources, a gain coefficient of computing resources, and an operation intensity of the full-scale AI service; Calculating a second inference efficiency of the full-scale AI service based on a maximum value between the first heterogeneous service resource amount and a preset heterogeneous service resource amount threshold; Determining a first priority for allocating heterogeneous service resources to each full-scale AI service based on the second inference efficiency, the first request intensity, and the operation intensity of the full-scale AI service; Based on the first priority, the amount of heterogeneous service resources to be allocated for each full-size AI service is determined.

3. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 1 is characterized in that: Determining the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request includes: Obtain target perception results corresponding to target edge user requests based on the current heterogeneous service orchestration network; Calculate the second request intensity of all edge users for each lightweight AI service and the third request intensity of all edge users for each microservice based on the target perception result; Determine a first basic number of instances of each lightweight AI service based on the second request intensity, and determine a second basic number of instances of each microservice based on the third request intensity; Determine a second priority for increasing the number of instances for each lightweight AI service based on the first basic number of instances, and determine a number of instances to be deployed for each lightweight AI service based on the first basic number of instances and the second priority; The maximum delay reduction for adding one instance for each microservice is determined based on the second basic number of instances, and the third priority for increasing the number of instances for each microservice is determined based on the sorting result of the maximum delay reduction. The number of instances to be deployed for each microservice is determined based on the second basic number of instances and the third priority.

4. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 1 is characterized in that: The state space, action space and optimization objective function are constructed based on historical edge user requests, service deployment information and network topology information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and the number of computing platforms, including: Constructing a state space based on the historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination, and network topology information; Constructing an action space based on the number of computing platforms in the underlying heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination; An optimization objective function is constructed based on historical edge user requests and service deployment information in the basic heterogeneous service orchestration network corresponding to the preset geographic location-time slot combination.

5. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 4 is characterized in that: The optimization objective function is constructed based on historical edge user requests and service deployment information in the basic heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination, including: Input historical edge user requests into the underlying heterogeneous service orchestration network to obtain historical perception results; Calculating the average service delay corresponding to historical edge user requests based on the historical perception results; Calculating a load balancing index corresponding to the basic heterogeneous service orchestration network through service deployment information in the basic heterogeneous service orchestration network; An optimization objective function is determined based on the average service delay and the load balancing index.

6. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 5 is characterized in that: The calculation formula of the average service delay is: Where, Indicates edge user requests The average service delay, Indicates edge user requests The set of all request paths, Indicates edge user requests No. service paths, Indicates the computing platforms Is there any AI service on the deploy, Indicates the computing platforms Are there microservices? deploy, Indicates edge user requests exist No. computing platforms AI services The queue delay on Indicates edge user requests exist No. computing platforms Microservices The processing delay on Indicates that edge user requests are on the computing platform AI services on After processing, it is routed to the computing platform AI services on The routing probability of continuing processing, Indicates edge user requests The arrival rate, Indicates the completion of a complete edge user request Whether it needs to go through full-scale AI service inference, Indicates the completion of a complete edge user request Is lightweight AI service reasoning required? Indicates the unit time slot length, Indicates full-scale AI services The operating strength, Represents lightweight AI services The operating strength, Indicates full-scale AI services The inference efficiency, Represents lightweight AI services The inference efficiency, Indicates edge user requests Divide into a complete service path The arrival rate on Indicates edge user requests exist No. computing platforms AI services corresponding to the service chain The arrival rate on express The average communication latency when processing across computing platforms.

7. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 5 is characterized in that: The calculation formula of the load balancing index is: Where, The load balancing index for the underlying heterogeneous service orchestration network, They represent the sensitivity coefficients of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. They represent the variance of computing resource occupancy, bandwidth resource occupancy, and memory resource occupancy, respectively. Indicates the total number of computing platforms, res Indicates computing resource utilization, bandwidth resource utilization, or memory resource utilization. represents the average utilization of heterogeneous service resources, Represents a computing platform The utilization rate of heterogeneous service resources.

8. The hybrid orchestration method of AI services and microservices in cloud-edge collaboration according to claim 1 is characterized in that: The expression of the reward function is: Where, Indicates the In the training round The reward value of each time step, Indicates the total number of instances of the service to be deployed. Indicates the In the training round A comprehensive measure of the performance of heterogeneous service orchestration networks in time steps. Represent the weight parameters of the impact of time steps and training rounds on iterative rewards; Indicates edge user requests The average service delay, Represents the load balancing index of the heterogeneous service orchestration network, They represent the weight parameters of the average service delay and load balancing index on the optimization objective function.

9. A hybrid orchestration device for AI services and microservices in cloud-edge collaboration, characterized in that: include: An information acquisition module, which is used to obtain target basic information and target edge user requests corresponding to the current heterogeneous service orchestration network, wherein the target basic information includes a target geographic location and a target time slot; A resource determination module, configured to determine the amount of heterogeneous service resources to be allocated corresponding to the full-scale AI service based on the target edge user request, a preset heterogeneous service resource amount threshold, and the operation intensity of the full-scale AI service; A resource allocation module, which is used to allocate resources to the full-scale AI service of the cloud computing layer in the current heterogeneous service orchestration network based on the amount of heterogeneous service resources to be allocated; An instance calculation module, which is used to determine the number of instances to be deployed corresponding to the lightweight AI service and microservice according to the target edge user request; a model screening module, configured to screen out a basic invalid action mask deep Q network AMDQN corresponding to the target basic information from a preset orchestration strategy database as a target AMDQN, wherein the orchestration strategy database includes a plurality of basic AMDQNs having a mapping relationship with a geographic location-time slot combination; A hybrid orchestration module, which is used to control the target AMDQN to interact with the current heterogeneous service orchestration network to deploy lightweight AI services and microservices corresponding to the number of instances to be deployed in the current heterogeneous service orchestration network; A model generation module is used to construct a state space, an action space, and an optimization objective function based on historical edge user requests, service deployment information in the basic heterogeneous service orchestration network corresponding to a preset geographic location-time slot combination, network topology information, and the number of computing platforms. It also determines the historical number of instances to be deployed corresponding to lightweight AI services and microservices; and constructs a reward function based on the optimization objective function and the historical number of instances to be deployed. An initial AMDQN is constructed based on the state space, action space, and reward function and trained to generate the base AMDQN.

10. A hybrid orchestration device for AI services and microservices in cloud-edge collaboration, characterized by: The hybrid orchestration device for AI services and microservices in cloud-edge collaboration includes a processor, a memory, and a hybrid orchestration program for AI services and microservices in cloud-edge collaboration stored on the memory and executable by the processor. When the hybrid orchestration program for AI services and microservices in cloud-edge collaboration is executed by the processor, the steps of the hybrid orchestration method for AI services and microservices in cloud-edge collaboration as described in any one of claims 1 to 8 are implemented.

Images