AI service and microservice deployment and request routing method and system in edge scenario

Abstract

This invention relates to the field of edge computing technology, providing a method and system for deploying and routing AI services and microservices in edge scenarios. The method includes: determining a deployment and request routing model for AI services and microservices in a multi-edge collaborative network scenario based on acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data; determining a joint optimization function based on the deployment and request routing model, with end-to-end latency, system energy consumption, and resource utilization as optimization objectives; training an agent to learn the optimal deployment strategy using a deep reinforcement learning algorithm based on a potential function and piecewise rewards, guided by the joint optimization function, to obtain a deployment scheme for AI service instances and microservice instances on multiple edge nodes; and calculating the routing path for user requests to reach the network based on the deployment scheme and real-time network status. The method and system provided by this invention improve service quality and resource utilization efficiency.

Description

Technical Field

[0001] This invention relates to the field of edge computing technology, and in particular to a method and system for deploying and routing AI services and microservices in edge scenarios. Background Technology

[0002] The rapid development of artificial intelligence (AI) technology has transformed numerous manufacturing industries, significantly boosting productivity. The increasingly powerful intelligent processing capabilities of AI services have led to a vast and highly interconnected architecture, posing increasingly severe challenges to the organization and application of AI models. To address this issue, Model as a Service (MaaS) has emerged as a novel service paradigm. MaaS deploys trained models on the cloud or edge nodes using containerization, allowing service providers to access advanced functionalities of the models across various business applications via APIs.

[0003] Significant challenges exist in the practical application of the MaaS (Machine-as-a-Service) AI application framework. In the workflow of AI applications, user requests first pass through an API gateway, then through multiple preprocessing modules before entering the AI ​​service for inference. Furthermore, they must traverse post-processing modules to return a structured response. It is evident that the complete delivery of AI applications depends not only on the AI ​​service but also on deep collaboration with supporting network functions. To deliver diverse and high-performance AI applications, it is crucial to consider hybrid orchestration of AI services with the currently prevalent microservices. Moreover, fine-grained hybrid orchestration allows for precise control over the number of instances, reducing response latency and system energy consumption, ultimately providing more diverse and efficient AI services.

[0004] Existing research often discusses AI and microservice orchestration separately. For example, it only considers the orchestration of monolithic AI services (such as task offloading, resource allocation, and service deployment), while neglecting the orchestration of supporting microservices during AI application implementation. Due to the lack of consideration for the calling and competition relationships between AI services and multi-instance microservices, this orchestration often leads to performance degradation in AI applications, severely impacting service quality. Furthermore, while some works have comprehensively considered resource competition and calling relationships between microservices in resource-constrained edge networks, achieving joint optimization of microservice deployment and routing, they have failed to consider the heterogeneity and uniqueness of AI services in terms of resource requirements. Clearly, these methods are unsuitable for service networks containing emerging AI applications. Therefore, how to achieve joint deployment and request routing of AI services and microservices in multilateral collaborative scenarios is a pressing issue that needs to be addressed. Summary of the Invention

[0005] This invention provides a method and system for deploying and routing AI services and microservices in edge scenarios, in order to solve the shortcomings of existing methods for deploying and routing AI services and microservices in edge scenarios that cannot adapt to emerging service networks.

[0006] This invention provides a method for deploying AI services and microservices and routing requests in edge computing scenarios, including: Based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data, the deployment and request routing model of AI services and microservices in the multi-edge collaborative network scenario is determined. Based on the deployment and request routing model, a joint optimization function is determined with the optimization objectives of end-to-end latency, system energy consumption, and resource utilization. Guided by the joint optimization function, the agent is trained to learn the optimal deployment strategy through a deep reinforcement learning algorithm based on the potential energy function and piecewise rewards, thereby obtaining the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. Based on the deployment scheme and real-time network status, calculate the routing path when a user request reaches the network.

[0007] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the network topology data includes: the number of network nodes, the number and operating frequency of CPU and GPU cores of the nodes, and the communication latency between nodes; The service characteristic data of the AI ​​services and microservices include: the computational load, memory access load, number of cores occupied by any service on any node, and service rate of the service instance for each service in the AI ​​service and microservice set; The user request stream data includes: the service chain that each request stream needs to call sequentially, the request arrival rate of the request stream that follows a Poisson distribution, the maximum tolerable latency, and the distribution probability of each access node.

[0008] According to the AI ​​service and microservice deployment and request routing method in edge scenarios provided by the present invention, the step of determining the AI ​​service and microservice deployment and request routing model in a multi-edge collaborative network scenario includes: To address the differences in service characteristics and resource requirements between AI services and microservices, service models are established for AI services and microservices respectively. These service models are used to calculate the service rate of an instance based on the number of cores occupied by the service on a node and the core working frequency.

[0009] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the step of determining a joint optimization function based on the deployment and request routing model, with end-to-end latency, system energy consumption, and resource utilization as optimization objectives, includes: Based on the deployment and request routing model, the communication latency between different services, the queuing latency of user requests in the queuing network, and the processing latency of service instances are determined. Based on the communication latency, queuing latency, and processing latency, a functional expression for the end-to-end latency is determined. Dynamic energy consumption is determined based on the utilization rate and operating frequency of CPU and GPU cores on each node. Based on the dynamic energy consumption and the static energy consumption under no-load conditions, a functional expression for system energy consumption is determined. The average resource utilization rate is determined by a function based on the weighted sum of the average CPU and GPU resource usage on each node. The joint optimization function is formed by combining the functional expressions of end-to-end delay, system energy consumption, and average resource utilization rate according to their weights.

[0010] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the step of training the agent to learn the optimal deployment strategy by using the joint optimization function as a guide and a deep reinforcement learning algorithm based on the potential function and piecewise rewards includes: The information on microservice instances and AI service instances deployed on the node, the used resource space, and the remaining resource space are used as the state space of the intelligent agent. The probability distribution of selecting a certain node for service instance deployment is used as the action space of the agent; During the instance deployment phase, if the service instance is successfully deployed on the target node, a positive segmented reward will be given; if the deployment fails due to insufficient resources on the target node, a negative penalty will be given. Once all service instances are deployed, the potential energy function value is calculated based on the improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to the historical state, and the state potential energy difference is used as an additional reward. The agent updates its network parameters by maximizing the cumulative reward until the policy converges.

[0011] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the calculation of the potential energy function value based on the degree of improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to historical states includes: If the index value of the current deployment state is better than or equal to the historical best record or the previous state record, a positive potential energy component is assigned; otherwise, a negative potential energy component is assigned. The potential energy component includes the time delay potential energy component, the energy consumption potential energy component, and the resource utilization potential energy component. The sum of the time delay potential energy component, energy consumption potential energy component, and resource utilization potential energy component is taken as the final total potential energy function value.

[0012] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the network structure of the deep reinforcement learning algorithm includes a policy network, two value networks and a corresponding target network, and the training agent learns the optimal deployment strategy, including: Experience data is sampled from the experience pool, and each piece of experience data includes the current state, action, reward, and next state; The action value loss function is calculated based on the empirical data, and the parameters of the two value networks are updated by minimizing the action value loss function. Based on the updated value network's assessment of action value, a policy loss function is calculated, and the parameters of the policy network are updated by minimizing the policy loss function. Based on the updated policy network and value network, the parameters of the target network are updated using a soft update method until the policy network converges, thus obtaining the optimal deployment scheme.

[0013] According to the edge scenario AI service and microservice deployment and request routing method provided by the present invention, the step of calculating the routing path when a user request reaches the network based on the deployment scheme and real-time network status includes: For any user request to be routed, identify the next service type to be accessed, and the set of nodes where service instances of the service type are currently deployed; Based on the number of service instances deployed on each node in the node set and the communication latency between each node and the current node, the forwarding route probability of each node is calculated. Based on the forwarding probability, a routing path is generated when the user request reaches the network.

[0014] This invention also provides an AI service and microservice deployment and request routing system for edge computing scenarios, comprising: The model determination unit is used to determine the deployment and request routing model of AI services and microservices in a multi-edge collaborative network scenario based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data. The optimization function determination unit is used to determine a joint optimization function based on the deployment and request routing model, with the optimization objectives being end-to-end latency, system energy consumption, and resource utilization. The agent training unit is used to train the agent to learn the optimal deployment strategy by using the joint optimization function as a guide and a deep reinforcement learning algorithm based on the potential energy function and segmented rewards, so as to obtain the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. The path calculation unit is used to calculate the routing path when a user request reaches the network based on the deployment scheme and real-time network status.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the AI ​​service and microservice deployment and request routing method in any of the above-described edge scenarios.

[0016] The present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the AI ​​service and microservice deployment and request routing method in edge scenarios as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the AI ​​service and microservice deployment and request routing method in any of the above-described edge scenarios.

[0018] This invention provides a method and system for deploying and routing AI services and microservices in edge scenarios. By constructing a unified deployment and request routing model for multi-edge collaborative network scenarios, it achieves accurate characterization of heterogeneous resources and complex service dependencies, fundamentally solving the problem that existing technologies are difficult to adapt to the dynamics and complexity of emerging service networks. It innovatively introduces a joint optimization function targeting end-to-end latency, system energy consumption, and resource utilization, and based on this, designs a deep reinforcement learning training mechanism based on a potential energy function and segmented rewards. This mechanism can transform macroscopic system-level optimization objectives into refined reward signals driving agent learning through the potential energy function, while using segmented rewards to ensure the success rate of basic deployment. This efficiently guides the agent to explore a globally optimal or near-optimal deployment scheme that systematically balances business experience (low latency), operating costs (low energy consumption), and infrastructure efficiency (high utilization). Ultimately, based on this intelligent deployment scheme and combined with real-time network status to generate routing paths, it ensures that user requests can be accurately and efficiently scheduled to the most suitable service instance, thereby optimizing the overall performance of service deployment and routing scheduling in the edge network environment and significantly improving service quality and resource utilization efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the AI ​​service and microservice deployment and request routing method in edge scenarios provided by this invention.

[0021] Figure 2 This is a schematic diagram of AI service and microservice deployment under a multi-edge node collaborative network provided by the present invention.

[0022] Figure 3 This is a schematic diagram of AI service and microservice deployment training based on reinforcement learning algorithms provided by the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the AI ​​service and microservice deployment and request routing system in edge scenarios provided by the present invention.

[0024] Figure 5 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0026] To address the problem that existing methods for deploying and routing AI services and microservices in edge scenarios cannot adapt to emerging service networks, this invention provides a method for deploying and routing AI services and microservices in edge scenarios. In this method, based on acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data, a deployment and request routing model for AI services and microservices in a multi-edge collaborative network scenario is determined. Based on the deployment and request routing model, a joint optimization function is determined with end-to-end latency, system energy consumption, and resource utilization as optimization objectives. Guided by the joint optimization function, a deep reinforcement learning algorithm based on potential functions and piecewise rewards is used to train an agent to learn the optimal deployment strategy, obtaining the deployment scheme of AI service instances and microservice instances on multiple edge nodes. Based on the deployment scheme and real-time network status, the routing path for user requests to reach the network is calculated.

[0027] The method provided in this invention, by constructing a unified deployment and request routing model for multi-edge collaborative network scenarios, achieves accurate characterization of heterogeneous resources and complex service dependencies, fundamentally solving the problem that existing technologies are difficult to adapt to the dynamics and complexity of emerging service networks. It innovatively introduces a joint optimization function targeting end-to-end latency, system energy consumption, and resource utilization, and based on this, designs a deep reinforcement learning training mechanism based on a potential energy function and segmented rewards. This mechanism can transform macroscopic system-level optimization objectives into refined reward signals driving agent learning through the potential energy function, while using segmented rewards to ensure the success rate of basic deployment. This efficiently guides the agent to explore a globally optimal or near-optimal deployment scheme that systematically balances business experience (low latency), operating costs (low energy consumption), and infrastructure efficiency (high utilization). Finally, based on this intelligent deployment scheme and combined with real-time network status to generate routing paths, it ensures that user requests can be accurately and efficiently scheduled to the most suitable service instance, ultimately achieving overall performance optimization of service deployment and routing scheduling in the edge network environment, significantly improving service quality and resource utilization efficiency.

[0028] The embodiments of this invention can be applied to scenarios requiring AI service and microservice deployment and request routing in edge environments. The executing entity of this method can be an electronic device such as a terminal device, computer, server, server cluster, or specially designed AI service and microservice deployment and request routing device, or a device installed in such an electronic device, which can be implemented through software, hardware, or a combination of both.

[0029] Figure 1 This is one of the flowcharts illustrating the AI ​​service and microservice deployment and request routing method in edge scenarios provided by this invention, such as... Figure 1 As shown, the method includes the following steps 110-140.

[0030] Step 110: Based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data, determine the deployment and request routing model of AI services and microservices in the multi-edge collaborative network scenario.

[0031] Specifically, the embodiments of the present invention address the problem of joint deployment and request routing of AI services and microservices in a multi-edge node collaborative network. Figure 2 This is a schematic diagram of AI service and microservice deployment under a multi-edge node collaborative network provided by the present invention, such as... Figure 2As shown, in a multilateral collaboration scenario, there are multiple edge nodes, each with limited computing resources. If an edge node deploys only a single microservice or AI model instance, the limited CPU / GPU resources will be quickly exhausted when a sudden surge of requests arrives, leading to a surge in task queuing latency or even service interruption. Furthermore, microservices typically involve short tasks and high concurrency, requiring rapid response, while AI model tasks are long tasks and computationally intensive, requiring a stable supply of computing power. When these two are deployed together, complex mutual calls and competition relationships arise. Therefore, this paper considers a multi-instance deployment scheme for microservices and AI models. By rationally configuring the resource requirements of each instance, request response latency is reduced, and network congestion and service quality degradation caused by resource contention are avoided.

[0032] The deployment and request routing model is an abstract description and digital representation of a real multi-edge node network, the AI / microservices running on it, and user requests. Network topology data describes the number and location of edge nodes (servers) and the connections between them. Service characteristic data describes the resource requirements of each AI service and microservice, such as the required number of CPU / GPU cores, memory, bandwidth, and processing power, such as the number of requests per second. Request flow describes the arrival patterns of user requests, the request content (the service chain consisting of which services need to be called sequentially), and service quality requirements, such as the maximum tolerable latency.

[0033] Preferably, the network topology data includes: the number of network nodes, the number and operating frequency of CPU and GPU cores in the nodes, and the communication latency between nodes.

[0034] The service characteristic data for AI services and microservices includes: the computational load, memory access load, the number of cores occupied by any service on any node, and the service rate of service instances for each service in the AI ​​service and microservice set.

[0035] Specifically, in a multilateral network scenario based on a hybrid orchestration of AI models and microservices, this embodiment uses Describe the topology of the physical network, where Represents edge nodes in an edge network scenario The set, This indicates the number of edge nodes in the edge network. any element in Indicates the first Several edge servers. These servers have different numbers of cores, all of which were idle before allocation. Indicates server The number of CPU cores, Indicates server The number of GPU cores. The system uses a single core as the smallest unit of resource allocation, and each core can be used by at most one instance. Edge servers are connected by Ethernet cables. Represents edge nodes and The set of links between them. Propagation delay is considered when communication occurs between two nodes within a multilateral collaborative network. This embodiment uses Represents a node and The set of transmission delays between them.

[0036] This embodiment uses Represents a set of AI models, where Indicates the types of AI models. Indicates the first This embodiment represents a collection of microservices as follows: [AI model]. ,in Indicates the types of microservices. Indicates the first Microservice types.

[0037] This embodiment uses Representing AI models The parameters, where Indicates the first The computational cost of an AI model Indicates the first The number of visits to each AI model.

[0038] This embodiment addresses the differences between AI services and microservices in terms of service characteristics and resource requirements, and establishes service models for AI services and microservices respectively. The service model is used to calculate the service rate of an instance based on the number of cores occupied by the service on the node and the core working frequency.

[0039] Because the AI ​​model instances need to be deployed on the GPUs of edge servers, this embodiment uses integer variables. Indicates deployment decisions. When At that time, explain the basic AI model No server usage The GPU cores, i.e., the AI ​​models, are not deployed on the server. Up. When hour, Representing AI models On the server The number of GPU cores used. The following restrictions apply: ; For deployment on servers AI models on For instances, only the computing resources allocated to the instance by the server are considered. Based on the core's operating frequency, the following is obtained: ; in, Indicates the operating frequency of the GPU core. This represents the floating-point operation coefficient per clock cycle of a single GPU core.

[0040] To better evaluate the theoretical inference performance of the AI ​​service module on edge nodes, this embodiment uses the calculation method in the RoofLine model. Let... The computational intensity of the AI ​​service module is represented by the following formula: ; This reflects the computational intensity of the AI ​​service module relative to memory access. Let... This represents the ratio of computing and bandwidth resources allocated to the edge server, calculated using the following formula: ; When calculating strength When performance is limited by memory bandwidth, it is called memory-intensive. Reasoning speed for: ; When calculating strength When performance is limited by the hardware's maximum computational peak, it is called compute-intensive, and the inference speed at this time... for: ; In summary, reasoning speed It can be represented as: ; in and This represents the bandwidth and computing resource pairing, respectively. The gain coefficient of the inference rate. Therefore, the inference time It can be represented as: ; Basic processing capabilities of AI models It can be represented as: ; AI service processing capabilities With core market share and operating frequency Related, can be expressed as: ; Among them, AI services On the server Core market share The following formula can be used for calculation: ; This embodiment will use integer variables Defined as deployed on the server microservices The number of cores used is limited by the resource constraints of edge nodes. The following restrictions must be met: ; When a microservice instance is deployed on a server, the instance's service speed... It can be determined by the core's operating frequency and core occupancy rate, using the following formula: ; in Indicates the core operating frequency. This indicates the processing capacity of a microservice instance without using DVFS. This indicates the microservice On the server v The core market share can be calculated using the following formula. : ; Furthermore, the user request stream data includes: the service chain that each request stream needs to call sequentially, the request arrival rate of the request stream that follows a Poisson distribution, the maximum tolerable latency, and the distribution probability of each access node.

[0041] Specifically, considering that the arrival of edge user request flows is modeled as a Poisson distribution, this embodiment uses... Represents the set of user request streams arriving at the edge network, where Indicates the type of request stream. Indicates the first There are different types of request flows. Each type of request flow can be represented by a quadruple. ,in This represents a chain of services that needs to be traversed sequentially to complete a user request. It may be a chain of pure microservices, for example... It could also be a hybrid call chain containing microservices and AI model modules, for example... .symbol The request indicates that the distribution follows a Poisson distribution. The arrival rate, due to It is a service chain that is traversed sequentially, so Too The service arrival rate of each service. This indicates that the request stream was successfully completed. The maximum delay, if the delay is greater than If the response timeout occurs, the request has not been completed successfully. Represents the request stream The probability distribution of each access node, that is... Represents the request stream From edge nodes The probability of entering the network.

[0042] Once a request stream enters the network, it will first be routed to the nearest service instance. Therefore, this invention establishes a variable for this initial routing path. To describe. Among them Indicates arrival at the access node Request stream Whether to route to the node Obviously, for The following constraints apply: ; ; in Represents the user request stream The first service in the service chain, this constraint specifies that routing can only be done to the edge node where the corresponding service instance is deployed.

[0043] For any AI service or microservice, a request can be processed on any edge node where the corresponding instance is deployed. When a request arrives at a service, it waits in a queue until it is processed. Since the same AI model or microservice may be deployed on different edge nodes, and different request flows may also call the same AI model or microservice, therefore, for the same request flow... There may be multiple routing paths that pass through the same AI model or microservice in the same order, but the AI ​​model or microservice is deployed in different locations. Therefore, this invention uses... Indicates user request The corresponding set of routing paths, for example, for , express All possible routing paths, which need to be in accordance with the set defined above. Iterate through all microservices or AI models in sequence.

[0044] In order to make full use of network resources and improve service quality, this embodiment introduces request routing variables. This indicates a user request. At the edge node Complete the service ( It can be an AI model or microservices After that, routing is done to the edge node. services The probability. It's important to note that edge nodes can only route user requests to services deployed on them. The edge nodes, and each at the edge nodes Complete the service All requests must be routed to the next node or have their service completed. Therefore, the request routing variable is... The following constraints apply: ; ; To balance the impact of instance count and communication latency on routing paths, the routing forwarding probability... It can be obtained from the following formula: ; in , The weights representing the number of instances and communication latency, and satisfying the following conditions: . For all deployed instances The set of edge nodes Represents a node Deployment services The number of instances, Represents a node and Communication delay between them.

[0045] Requests from edge users may originate from mobile terminals, Industrial Internet of Things (IIOT) devices, smart home devices, etc., and connect to different edge computing platforms via wireless connections, edge gateways, etc. To describe the user request arrival process, this embodiment assumes that edge user requests follow a Poisson distribution. Since different request flows may also call the same AI model or microservice, according to Burke's theorem and the linear additivity of Poisson flows, requests from different users for the same service can be merged into the same queuing network for that service. Define variables. Indicates user request At the edge node Online services ( It can be an AI model or microservices The request arrival rate can be calculated using the following formula in this embodiment. : ; ; .

[0046] Step 120: Based on the deployment and request routing model, determine a joint optimization function with the optimization objectives of end-to-end latency, system energy consumption, and resource utilization.

[0047] Specifically, this step transforms the actual deployment and routing problem into a computable and optimizable mathematical problem. Its purpose is to establish a unified and quantifiable standard for scientifically evaluating and comparing the merits of different deployment and routing strategies. The optimization objective is the standard for measuring the quality of a solution. In this embodiment, three core, often conflicting, optimization directions are established.

[0048] End-to-end latency refers to the total time it takes for a user request to be sent and for a complete response to be received. This may include processing time on each service instance, queuing time for requests to be processed before service instances, and network communication latency for transmitting requests between service instances on different edge nodes. The optimization goal is to minimize end-to-end latency. Lower latency results in a better user experience and faster system response.

[0049] System energy consumption refers to the total electrical energy consumed by the entire multi-edge node network to run AI services and microservices. This mainly includes the dynamic energy consumption of computing chips such as server CPUs and GPUs during operation, as well as the static energy consumption generated even when idle. The optimization goal is to minimize system energy consumption. Lower energy consumption results in lower operating costs and a more environmentally friendly system.

[0050] Resource utilization refers to the average degree to which valuable computing resources (such as CPU cores and GPU cores) on edge nodes are effectively used. For example, it can be expressed as the ratio of "used resources" to "total resources." The optimization goal is to maximize resource utilization. Higher utilization indicates less idle resources and a higher return on investment for the infrastructure.

[0051] Since the three objectives may conflict, for example, to achieve extremely low latency, more instances may need to be deployed, which may lead to increased energy consumption and decreased resource utilization, the role of the joint optimization function is to find an optimal balance point for these conflicting objectives.

[0052] In some embodiments, step 120 specifically includes: Step 121: Based on the deployment and request routing model, determine the communication latency between different services, the queuing latency of user requests in the queuing network, and the processing latency of service instances. Based on the communication latency, queuing latency, and processing latency, determine the functional expression of the end-to-end latency. Step 122: Determine the dynamic energy consumption based on the utilization and operating frequency of the CPU and GPU cores on each node, and determine the functional expression of the system energy consumption based on the dynamic energy consumption and the static energy consumption under no-load conditions. Step 123: Determine the functional expression of average resource utilization based on the weighted sum of the average CPU and GPU resource usage on each node; Step 124: Combine the functional expressions of end-to-end delay, system energy consumption, and average resource utilization rate according to weights to form a joint optimization function.

[0053] Specifically, in the hybrid orchestration of AI models and microservices in edge network scenarios, the response latency of user requests mainly consists of three parts: communication latency between different services. User request queuing latency at edge nodes and processing latency of service instances .

[0054] Propagation delay This is primarily caused by cross-node data transfer between two service instances. This applies to service instances deployed on different edge nodes. and Its communication delay It can be represented as: ; in, Indicates in service instance Edge node With service instances Edge node Network propagation delay between them. If and If deployed on the same edge node, then .

[0055] Because AI models and microservices have different deployment characteristics in edge networks, their queuing networks need to be modeled differently. Due to the additivity of the Poisson process, the combined equivalent request arrival rate still follows a Poisson distribution for both AI models and microservices. For microservices, instances of the same microservice on the same edge node are independent, so all microservice requests arriving at that edge node will be processed on these instances. If no instance is idle, it will enter a queuing queue, waiting for the core to become available. These instances together constitute the... Queuing system. In this queuing system, the service intensity is defined as... , is represented as: ; in microservices Processing capacity microservices At the edge node The service intensity on the platform must meet the following requirements. Otherwise, the waiting queue length of the queuing system will become infinitely long, causing the request response to time out.

[0056] This embodiment adjusts the forwarding probability between services. Change the service reach rate by adjusting the number of deployed instances. Adjust the service's processing capacity to ensure the service strength is always less than 1, allowing all services to be invoked normally. When the system is stable, the average queuing time for user requests in the queuing network is [not specified]. It can be represented as: ; in This indicates that no requests are waiting for the microservice. Probability of processing: ; ; For AI services, only one integrated instance of the same service exists on the same edge node; therefore, this embodiment uses an M / M / 1 queuing system. In this queuing system, service strength... Represented as: ; in Indicates AI services At the edge node The service intensity should be less than 1. Similar to microservices, this embodiment adjusts the number of AI services deployed. and request route variables To control service intensity It is always less than 1, thus ensuring that all services can be called normally.

[0057] According to the M / M / 1 queuing system, when the system is stable, the average queuing time for user requests in the queuing network is... It can be represented as: ; Next, the processing delay is calculated. User requests at edge nodes microservices Processing latency and AI models Processing latency They can be represented as follows: ; ; In this embodiment, the user request is made through multiple routing paths. Complete. Definition Representing the The processors that the request routing path passes through. Indicates the first The access node for the request path. Therefore, the first... A request path can be represented as... This embodiment uses This indicates the average latency required to complete the request via this route: ; At the same time, for the first Request routing path This embodiment can be based on the forwarding probability. To determine the probability that a user requests to choose this path. : ; Since each user request can have multiple paths, this embodiment needs to calculate the expected value of multiple paths, which is the average response latency of the user request, to determine the response latency of a user request. : ; The energy consumed by microservices and AI model instances during computation or inference is a crucial consideration for multilateral collaborative networks. Server energy consumption consists of two main parts: static energy consumption, which refers to the energy consumed when the CPU or GPU is idle, and dynamic energy consumption, which refers to the energy consumed by the CPU or GPU during execution. Therefore, the energy consumption of a single CPU core can be expressed as: ; in , and It is the regression coefficient. It refers to the operating frequency of the CPU core. It is CPU core utilization. This refers to the energy consumption of the CPU core under no-load conditions.

[0058] The energy consumption of a single GPU can be expressed as: ; in , and It is the regression coefficient. It is the operating frequency of the GPU core. It is the GPU core utilization rate. This refers to the GPU's power consumption under no-load conditions.

[0059] Therefore, the energy consumption of this edge network system It can be represented as: ; To minimize energy consumption and resource waste across the network, the system prioritizes utilizing servers with already deployed instances; therefore, this paper considers server resource utilization. This embodiment defines variables. and To represent edge nodes respectively The resource usage of CPU and GPU is calculated using the following formula: ; ; Therefore, the average resource utilization rate of edge networks It can be represented as: ; in This represents the weight assigned to different resources. In this embodiment, it refers to the average resource utilization rate. It is a weighted sum of the average CPU and GPU resource usage of each edge node.

[0060] In summary, this embodiment establishes a multi-objective optimization model for service deployment and request routing in a hybrid orchestration of microservices and AI services, under constraints such as CPU and GPU resources, deployment limitations, and routing probabilities. This model aims to balance multiple objectives, including end-to-end latency, system energy consumption, and resource utilization, ultimately determining the optimal deployment and routing scheme in a multilateral collaborative scenario.

[0061] The joint optimization function for the hybrid orchestration problem can be expressed as: ,in , and These represent the weight coefficients for different optimization objectives.

[0062] Step 130: Guided by the joint optimization function, the agent is trained to learn the optimal deployment strategy through a deep reinforcement learning algorithm based on the potential function and piecewise rewards, thereby obtaining the deployment scheme of AI service instances and microservice instances on multiple edge nodes.

[0063] Specifically, this invention jointly optimizes the deployment and request routing of microservices and AI models in a multilateral collaborative scenario, aiming to provide efficient services to users. To address the challenges posed by the NP-hard nature of this problem, this embodiment proposes a flexible action-evaluation algorithm (PS_SAC) based on a potential function and piecewise rewards. PS_SAC is a deep reinforcement learning algorithm designed to minimize request response latency and system energy consumption while improving resource utilization. This algorithm achieves this goal through fine-grained deployment of microservice and AI service instances, and further improves request processing efficiency by implementing instance-level routing probability allocation.

[0064] In reinforcement learning (RL), the agent interacts with the environment, observing the current state s_t at each step and taking action. To receive a reward Then the state changes The agent then continued to take action. Receive rewards The state changes to This process continues until it ends. The process exhibits the Markov property, meaning that the states... Only subject to and Impact. Therefore, RL is particularly well-suited for solving decision-making problems involving sequential actions. In this invention, the deployment of microservices and AI services is completed sequentially, and each step of instance deployment affects subsequent actions. Therefore, RL is well-suited for solving the problem of this invention. RL can maximize rewards and optimize the overall objective by exploring different instance deployment sequences. Therefore, the RL-based PS_SAC algorithm proposed in this embodiment is applicable to the problem of this invention, capable of exploring optimal strategies for service deployment and request routing in a hybrid orchestration of microservices and AI services.

[0065] Figure 3 This is a schematic diagram of AI service and microservice deployment training based on reinforcement learning algorithm provided by the present invention. This embodiment is based on SAC algorithm and designs PS_SAC algorithm based on segmented reward and potential energy function for state, action, reward and agent.

[0066] The potential energy function transforms macroscopic, sparse long-term optimization objectives into instantaneous feedback that the agent can understand and obtain at each step. The potential energy function calculates a scalar value—the potential energy value—based on the system's current state (i.e., the service deployment status). A higher potential energy value indicates a lower overall performance for that state.

[0067] Segmented rewards are used to provide more granular and timely guidance in complex tasks. They divide the agent's learning process into different stages, setting clear sub-goals for each stage. In the initial deployment stage, the primary task is to ensure that instances can be successfully deployed to nodes with sufficient resources. At this point, segmented rewards provide a simple and direct signal: a small positive reward for successful deployment, and a larger negative reward for deployment failure. This ensures that the agent first learns to adhere to basic resource constraints, avoiding ineffective solutions.

[0068] Preferably, guided by a joint optimization function, the agent is trained to learn the optimal deployment strategy using a deep reinforcement learning algorithm based on a potential function and piecewise rewards. Specifically, step 130 includes: Step 131: Use the information of microservice instances and AI service instances deployed on the node, the used resource space and the remaining resource space as the state space of the intelligent agent; Step 132: Use the probability distribution of selecting a certain node for service instance deployment as the action space of the agent; Step 133: During the instance deployment phase, if the service instance is successfully deployed on the target node, a positive segmented reward will be given; if the deployment fails due to insufficient resources on the target node, a negative penalty will be given. Step 134: After all service instances are deployed, calculate the potential energy function value based on the improvement of the end-to-end latency, system energy consumption and resource utilization of the deployed system relative to the historical state, and use the state potential energy difference as an additional reward. Step 135: The agent updates its network parameters by maximizing the cumulative reward until the policy converges.

[0069] Specifically, the state space S is set as follows: The resource space of the edge node, as well as the deployment space where microservice instances and AI model instances are placed, are defined as the state space of PS_SAC. .in, It is the deployment space for microservice instances, it provides This is a matrix of size describing information about microservice instances deployed on edge nodes. The deployment state space of the AI ​​model instance is defined as follows: It represents information about AI model instances deployed on edge nodes, and the matrix size is [missing information]. Considering that resource space is divided into used resource space and remaining resource space, using... Indicates the used resource space. This represents the remaining resource space. In this model, system resources mainly consist of CPU and GPU resources, therefore they are... A matrix of size.

[0070] Motionspace A is set as follows: The probability of deploying a microservice instance or AI model instance on an edge node is defined as the action of the PS_SAC agent. The action space is defined as... ,in Indicates the selection of the first The probability of deploying an instance on each node is determined. The agent outputs the action probability distribution through the Actor network and ultimately selects the node with the highest probability to execute the deployment. If the target node lacks sufficient resources, the deployment fails and a penalty mechanism is triggered.

[0071] The reward function R is set as follows: The reward mechanism is a core component of reinforcement learning algorithms, determining the immediate feedback the agent receives at each step. PS_SAC's reward mechanism is based on a potential energy function, using the potential energy difference between states to guide the agent towards a low-latency, high-utilization, and low-energy-consumption policy. The core idea of ​​the potential energy function is to decompose the reward into the difference between the potential energy of the current state and the next state. This method ensures that the optimal policy with the original reward is consistent with the shaped reward, thus accelerating the learning process without changing the optimization objective.

[0072] Considering that metrics such as average user response latency, utilization, and energy consumption can only be calculated after deployment is complete, this invention requires ensuring successful instance deployment before designing a reward system optimized for user response latency, utilization, and energy consumption. During the instance deployment phase, a reward function is designed. Encourage agents to complete deployment quickly: ; in This represents the total number of instances. As you can see, during deployment, each successful deployment increases potential energy, providing an additional reward; otherwise, the reward is negative.

[0073] After all instances have been deployed, this embodiment of the invention designs latency potential functions with latency, energy consumption, and utilization as optimization objectives. Energy consumption potential energy function Resource utilization potential function And calculate the potential energy function value.

[0074] Preferably, the potential energy function value is calculated based on the improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to historical states, including: If the index value of the current deployment state is better than or equal to the historical best record or the previous state record, a positive potential energy component is assigned; otherwise, a negative potential energy component is assigned. The potential energy component includes the time delay potential energy component, the energy consumption potential energy component, and the resource utilization potential energy component. The sum of the time delay potential energy component, energy consumption potential energy component, and resource utilization potential energy component is taken as the final total potential energy function value.

[0075] Specifically, the time delay potential energy component is determined by the time delay potential energy function. Obtain the time-delay potential function. Expressed as a formula:

[0076] in It is a potential energy function that rewards time delay. This is the state when deployment is complete. , This represents the minimum latency among the currently deployed states. express The time delay, Indicates the status of the last deployment completed. The time delay, All of these are time delay weighting coefficients.

[0077] Similarly, with the goal of reducing energy consumption, the potential energy function is designed. If the current deployment status is completed energy consumption Less than minimum energy consumption Energy consumption in the state after the last deployment was completed. When the potential energy is positive, it increases; otherwise, it is negative. Energy dissipation potential energy function. Expressed as a formula:

[0078] in All are utilization rate weighting coefficients.

[0079] Furthermore, this invention also considers utilization rate as an optimization objective; the higher the utilization rate, the better the deployment actions of the intelligent agent. To improve resource utilization rate, a resource utilization rate potential energy function is designed. If the current deployment status is completed utilization rate Greater than maximum utilization Or the utilization rate of the state after the last deployment was completed. When the potential energy is increased, it is increased; otherwise, negative potential energy is given. Therefore, the potential energy function with respect to utilization rate is... The calculation formula is as follows:

[0080] in All of these are utilization rate weighting coefficients. Therefore, the final total potential energy function is: ; The final reward function is: ; in For the original reward, For the next state, As a reward during the deployment process. This represents the potential energy function after deployment. Through the state potential energy difference, the potential energy function provides the agent with additional rewards, effectively mitigating the sparsity problem of the original rewards. Simultaneously, by utilizing potential energy designs related to latency, energy consumption, and utilization, the complex global optimization objective is transformed into local state differences, providing the agent with a clear direction for policy optimization. Furthermore, the mathematical form of the potential energy function strictly adheres to the policy invariance theorem, ensuring that the agent does not deviate from the original optimization objective while accelerating learning, reducing the randomness of exploration, and thus accelerating algorithm convergence.

[0081] Furthermore, the network structure of the deep reinforcement learning algorithm includes a policy network, two value networks, and a corresponding target network. The agent is trained to learn the optimal deployment policy, including: Experience data is sampled from the experience pool, and each piece of experience data includes the current state, action, reward, and next state; The action value loss function is calculated based on empirical data, and the parameters of the two value networks are updated by minimizing the action value loss function. Based on the updated value network's evaluation of action value, the policy loss function is calculated, and the parameters of the policy network are updated by minimizing the policy loss function. Based on the updated policy network and value network, the parameters of the target network are updated using a soft update method until the policy network converges, thus obtaining the optimal deployment scheme.

[0082] Specifically, the PS_SAC agent adopts a "dual Critic + target network" structure, where the Actor network is the policy network. ,in This represents the parameters to be learned. The probability value of each action is output by the Actor network and can be obtained through... Calculation. The Critic network is a state-action value function, expressed as: The parameters to be learned are It is used to evaluate the value of performing a certain action in a certain state.

[0083] First, the algorithm initializes all network parameters (policy network, value network, target network), the number of service instances, rewards, and the state space. During the experience collection phase, in the state... The agent executes actions through the policy function. and get the next state. and reward value If the target node lacks sufficient resources, the deployment is marked as failed and the current loop is exited. If all instances are deployed successfully, the loop exits. If an instance is successfully deployed and is not the last instance, the state transition data is stored. To the experience pool .

[0084] After all instances are deployed, the routing forwarding probability and average user response latency are calculated. Then, the reward for the final step is calculated, and the stored state transfer data is transferred to the experience pool. .

[0085] During the network update phase, batch data is sampled from the experience pool. Perform training and calculate the target Q-value: ; in Then, the loss function is calculated using the target Q-value and the actual Q-value, and the Actor network and Critic network are updated using the loss function: ; ; Then use the parameters A soft update was performed on the target network, and the optimal deployment strategy was successfully obtained. .

[0086] The complete training steps are shown in the following pseudocode: Input: Experience Pool Soft update factor Reward Discount Factor Batch data Request stream r Network parameters.

[0087] Output: Deployment strategy .

[0088] 1. Initialize the Actor network parameter Critic Network parameter and parameter

[0089] 2. Initialize the target network , And their network parameters, making ,

[0090] 3. Initialize the number of microservices and AI model instances. ,

[0091] 4. Initialization ,

[0092] 5. for do 6. / *Round up and add one* / 7. end for 8. for do 9. / *Round up and add one* / 10. end for 11. for do 12. Initialization deployment successful flag

[0093] 13. Initialize accumulated rewards

[0094] 14. Initialize the state space

[0095] 15. for do 16. Select an action based on the current strategy.

[0096] 17. Perform the action Transition to another state Calculate the reward

[0097] 18. Calculate cumulative rewards

[0098] 19. If microservice or AI model deployment fails, then 20. Order

[0099] 21. break 22. end if 23. If this is the last instance to be deployed, do... 24. break 25. end if 26. Stored state transition data pairs To the experience pool

[0100] 27. end for 28. if do 29. Calculate the routing probability after deployment is complete.

[0101] 30. Calculate the average request response time.

[0102] 31. Calculate the reward for the final step.

[0103] 32. Stored state transition data pairs To the experience pool

[0104] 33. Update

[0105] 34. end if 35. From the experience pool Medium sampling Group data pairs

[0106] 36. For each set of data, calculate the target Q value: 37. ,

[0107] 38. Minimize the loss function ,in

[0108] 39. Using minimization of the loss function Update the two Critic networks 40. Loss Function

[0109] 41. Using loss functions Update the current Actor network 42. Update the coefficients of the entropy regularization term.

[0110] 43. Soft update target network:

[0111] 44. end for 45. return

[0112] Step 140: Based on the deployment plan and real-time network status, calculate the routing path when the user request reaches the network.

[0113] Specifically, the deployment plan clearly specifies which edge nodes(s) each AI service or microservice instance is deployed on throughout the multi-edge node network, as well as the size of each instance (e.g., the number of cores it occupies). When a request needs to access a service, it can only be routed to those nodes where an instance of that service has already been deployed.

[0114] Real-time network status is a dynamically changing snapshot of the system, primarily reflecting the current operational status of the network. Its core data includes: whether a particular edge node or its service instance is online, overloaded, or experiencing a failure; the current resource utilization (e.g., CPU / GPU utilization), request queue length, etc., of each node and its service instance; and real-time communication latency and available bandwidth between nodes, which may change due to network congestion. Real-time network status ensures the adaptability and robustness of routing decisions, enabling the system to avoid sending requests to downed or overloaded nodes, thereby achieving load balancing and high availability.

[0115] A routing path refers to the sequence of edge nodes containing all service instances that a user request must traverse from the moment it enters the network until it is finally processed. For a single request, the routing path is not calculated all at once, but rather generated gradually. After a request reaches the current node and completes processing for a service, the system selects the next node based on the forwarding probability, repeating this process until all services in the service chain have been traversed.

[0116] In some embodiments, step 140, which calculates the routing path for a user request to reach the network based on the deployment scheme and real-time network status, specifically includes: For any user request to be routed, identify the next service type to be accessed, and the set of nodes where service instances of the service type are currently deployed; Based on the number of service instances deployed on each node in the node set and the communication latency between each node and the current node, the forwarding route probability of each node is calculated. Based on the forwarding probability, a routing path is generated when the user request reaches the network.

[0117] Specifically, after a user request completes the current service at an edge node, the system needs to determine the next service the user should access. By parsing the service chain to which the request belongs, the next service to be invoked is identified. For example, after data preprocessing, the "image recognition AI service" might need to be invoked. Based on the deployment plan, the set of all edge nodes where instances of this service are deployed can be located. This set of nodes constitutes all potential feasible targets for the next jump of the current request.

[0118] Forwarding route probability takes into account the following two key factors: Service processing capacity: Requests are tended to be sent to nodes with more instances of the service deployed. A higher number of instances generally indicates a stronger service processing capacity on that node. Routing requests to these nodes helps avoid single-point overload and achieves load balancing.

[0119] Network communication efficiency: Requests are preferentially sent to nodes with lower communication latency with the current node. This directly reduces the transmission time of requests over the network, contributing to a reduction in overall end-to-end latency.

[0120] Based on the calculated probability distribution, random selection or other algorithms (such as round-robin) are used to determine the next node to which the request will actually be forwarded. After the request arrives at the new node, the above steps are repeated until all necessary services have been accessed in the order of the service chain, finally completing the request and returning the complete routing path.

[0121] Understandably, the embodiments of the present invention model the application as a call chain composed of a mixture of AI services and microservices. By designing a reinforcement learning algorithm based on potential function and segmented reward, the invention addresses the problem of mixed deployment and request routing of AI services and microservices in multi-edge node collaborative networks with the objectives of end-to-end latency, system energy consumption, and resource utilization.

[0122] The following describes the AI ​​service and microservice deployment and request routing system in edge scenarios provided by this invention. The AI ​​service and microservice deployment and request routing system in edge scenarios described below can be referred to in correspondence with the AI ​​service and microservice deployment and request routing method in edge scenarios described above.

[0123] Based on the above embodiments, Figure 4 This is a schematic diagram of the structure of the AI ​​service and microservice deployment and request routing system in edge scenarios provided by the present invention, as shown below. Figure 4 As shown, the system includes: The model determination unit 410 is used to determine the deployment and request routing model of AI services and microservices in a multi-edge collaborative network scenario based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data. The optimization function determination unit 420 is used to determine a joint optimization function based on the deployment and request routing model, with the optimization objectives being end-to-end latency, system energy consumption, and resource utilization. The agent training unit 430 is used to train the agent to learn the optimal deployment strategy by using the joint optimization function as a guide and a deep reinforcement learning algorithm based on the potential energy function and segmented rewards, so as to obtain the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. The path calculation unit 440 is used to calculate the routing path when a user request reaches the network based on the deployment scheme and real-time network status.

[0124] Based on the above embodiments, the network topology data includes: the number of network nodes, the number and operating frequency of CPU and GPU cores of the nodes, and the communication latency between nodes; The service characteristic data of the AI ​​services and microservices include: the computational load, memory access load, number of cores occupied by any service on any node, and service rate of the service instance for each service in the AI ​​service and microservice set; The user request stream data includes: the service chain that each request stream needs to call sequentially, the request arrival rate of the request stream that follows a Poisson distribution, the maximum tolerable latency, and the distribution probability of each access node.

[0125] Based on the above embodiments, the model determination unit is specifically used for: To address the differences in service characteristics and resource requirements between AI services and microservices, service models are established for AI services and microservices respectively. These service models are used to calculate the service rate of an instance based on the number of cores occupied by the service on a node and the core working frequency.

[0126] Based on the above embodiments, the optimization function determination unit is specifically used for: Based on the deployment and request routing model, the communication latency between different services, the queuing latency of user requests in the queuing network, and the processing latency of service instances are determined. Based on the communication latency, queuing latency, and processing latency, a functional expression for the end-to-end latency is determined. Dynamic energy consumption is determined based on the utilization rate and operating frequency of CPU and GPU cores on each node. Based on the dynamic energy consumption and the static energy consumption under no-load conditions, a functional expression for system energy consumption is determined. The average resource utilization rate is determined by a function based on the weighted sum of the average CPU and GPU resource usage on each node. The joint optimization function is formed by combining the functional expressions of end-to-end delay, system energy consumption, and average resource utilization rate according to their weights.

[0127] Based on the above embodiments, the agent training unit is specifically used for: The information on microservice instances and AI service instances deployed on the node, the used resource space, and the remaining resource space are used as the state space of the intelligent agent. The probability distribution of selecting a certain node for service instance deployment is used as the action space of the agent; During the instance deployment phase, if the service instance is successfully deployed on the target node, a positive segmented reward will be given; if the deployment fails due to insufficient resources on the target node, a negative penalty will be given. Once all service instances are deployed, the potential energy function value is calculated based on the improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to the historical state, and the state potential energy difference is used as an additional reward. The agent updates its network parameters by maximizing the cumulative reward until the policy converges.

[0128] Based on the above embodiments, the agent training unit is specifically used for: If the index value of the current deployment state is better than or equal to the historical best record or the previous state record, a positive potential energy component is assigned; otherwise, a negative potential energy component is assigned. The potential energy component includes the time delay potential energy component, the energy consumption potential energy component, and the resource utilization potential energy component. The sum of the time delay potential energy component, energy consumption potential energy component, and resource utilization potential energy component is taken as the final total potential energy function value.

[0129] Based on the above embodiments, the agent training unit is specifically used for: Experience data is sampled from the experience pool, and each piece of experience data includes the current state, action, reward, and next state; The action value loss function is calculated based on the empirical data, and the parameters of the two value networks are updated by minimizing the action value loss function. Based on the updated value network's assessment of action value, a policy loss function is calculated, and the parameters of the policy network are updated by minimizing the policy loss function. Based on the updated policy network and value network, the parameters of the target network are updated using a soft update method until the policy network converges, thus obtaining the optimal deployment scheme.

[0130] Based on the above embodiments, the path calculation unit is specifically used for: For any user request to be routed, identify the next service type to be accessed, and the set of nodes where service instances of the service type are currently deployed; Based on the number of service instances deployed on each node in the node set and the communication latency between each node and the current node, the forwarding route probability of each node is calculated. Based on the forwarding probability, a routing path is generated when the user request reaches the network.

[0131] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5As shown, the electronic device may include: a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a method for deploying and routing AI services and microservices in an edge scenario. This method includes: determining a deployment and routing model for AI services and microservices in a multi-edge collaborative network scenario based on acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data; determining a joint optimization function based on the deployment and routing model, with end-to-end latency, system energy consumption, and resource utilization as optimization objectives; training an agent to learn the optimal deployment strategy using a deep reinforcement learning algorithm based on potential function and piecewise reward, guided by the joint optimization function, to obtain a deployment scheme for AI service instances and microservice instances on the multiple edge nodes; and calculating the routing path when a user request reaches the network based on the deployment scheme and real-time network status.

[0132] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0133] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the network topology data of the acquired multi-edge nodes, service characteristic data of AI services and microservices, and user request flow data provided by the above methods to determine the deployment and request routing model of AI services and microservices in a multi-edge collaborative network scenario; based on the deployment and request routing model, determine a joint optimization function with end-to-end latency, system energy consumption, and resource utilization as optimization objectives; guided by the joint optimization function, train the agent to learn the optimal deployment strategy through a deep reinforcement learning algorithm based on potential function and piecewise reward to obtain the deployment scheme of AI service instances and microservice instances on the multi-edge nodes; and calculate the routing path when user requests reach the network based on the deployment scheme and real-time network status.

[0134] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program performs the above-described methods to determine the deployment and request routing model of AI services and microservices in a multi-edge collaborative network scenario based on acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data. Based on the deployment and request routing model, a joint optimization function is determined with the optimization objectives of end-to-end latency, system energy consumption, and resource utilization. Guided by the joint optimization function, an agent is trained to learn the optimal deployment strategy through a deep reinforcement learning algorithm based on potential function and piecewise reward to obtain the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. Based on the deployment scheme and real-time network status, the routing path when a user request reaches the network is calculated.

[0135] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0136] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for deploying AI services and microservices and routing requests in edge computing scenarios, characterized in that, include: Based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data, the deployment and request routing model of AI services and microservices in the multi-edge collaborative network scenario is determined. Based on the deployment and request routing model, a joint optimization function is determined with the optimization objectives of end-to-end latency, system energy consumption, and resource utilization. Guided by the joint optimization function, the agent is trained to learn the optimal deployment strategy through a deep reinforcement learning algorithm based on the potential energy function and piecewise rewards, thereby obtaining the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. Based on the deployment scheme and real-time network status, calculate the routing path when a user request reaches the network.

2. The method for deploying and routing AI services and microservices in edge scenarios according to claim 1, characterized in that, The network topology data includes: the number of network nodes, the number and operating frequency of CPU and GPU cores in each node, and the communication latency between nodes; The service characteristic data of the AI ​​services and microservices include: the computational load, memory access load, number of cores occupied by any service on any node, and service rate of the service instance for each service in the AI ​​service and microservice set; The user request stream data includes: the service chain that each request stream needs to call sequentially, the request arrival rate of the request stream that follows a Poisson distribution, the maximum tolerable latency, and the distribution probability of each access node.

3. The method for deploying and routing AI services and microservices in edge scenarios according to claim 2, characterized in that, The determination of the AI ​​service and microservice deployment and request routing model in a multi-edge collaborative network scenario includes: To address the differences in service characteristics and resource requirements between AI services and microservices, service models are established for AI services and microservices respectively. These service models are used to calculate the service rate of an instance based on the number of cores occupied by the service on a node and the core working frequency.

4. The method for deploying and routing AI services and microservices in edge scenarios according to claim 1, characterized in that, The joint optimization function, determined based on the deployment and request routing model, with the optimization objectives of end-to-end latency, system energy consumption, and resource utilization, includes: Based on the deployment and request routing model, the communication latency between different services, the queuing latency of user requests in the queuing network, and the processing latency of service instances are determined. Based on the communication latency, queuing latency, and processing latency, a functional expression for the end-to-end latency is determined. Dynamic energy consumption is determined based on the utilization rate and operating frequency of CPU and GPU cores on each node. Based on the dynamic energy consumption and the static energy consumption under no-load conditions, a functional expression for system energy consumption is determined. The average resource utilization rate is determined by a function based on the weighted sum of the average CPU and GPU resource usage on each node. The joint optimization function is formed by combining the functional expressions of end-to-end delay, system energy consumption, and average resource utilization rate according to their weights.

5. The method for deploying and request routing AI services and microservices in edge scenarios according to any one of claims 1 to 4, characterized in that, The process of training the agent to learn the optimal deployment strategy, guided by the joint optimization function and using a deep reinforcement learning algorithm based on the potential energy function and piecewise rewards, includes: The information on microservice instances and AI service instances deployed on the node, the used resource space, and the remaining resource space are used as the state space of the intelligent agent. The probability distribution of selecting a certain node for service instance deployment is used as the action space of the agent; During the instance deployment phase, if the service instance is successfully deployed on the target node, a positive segmented reward will be given; if the deployment fails due to insufficient resources on the target node, a negative penalty will be given. Once all service instances are deployed, the potential energy function value is calculated based on the improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to the historical state, and the state potential energy difference is used as an additional reward. The agent updates its network parameters by maximizing the cumulative reward until the policy converges.

6. The method for deploying and routing AI services and microservices in edge scenarios according to claim 5, characterized in that, The calculation of the potential energy function value based on the improvement of the end-to-end latency, system energy consumption, and resource utilization of the deployed system relative to historical states includes: If the index value of the current deployment state is better than or equal to the historical best record or the previous state record, a positive potential energy component is assigned; otherwise, a negative potential energy component is assigned. The potential energy component includes the time delay potential energy component, the energy consumption potential energy component, and the resource utilization potential energy component. The sum of the time delay potential energy component, energy consumption potential energy component, and resource utilization potential energy component is taken as the final total potential energy function value.

7. The method for deploying and routing AI services and microservices in edge scenarios according to any one of claims 1 to 4, characterized in that, The deep reinforcement learning algorithm's network structure includes a policy network, two value networks, and a corresponding target network. The training agent learns the optimal deployment policy, including: Experience data is sampled from the experience pool, and each piece of experience data includes the current state, action, reward, and next state; The action value loss function is calculated based on the empirical data, and the parameters of the two value networks are updated by minimizing the action value loss function. Based on the updated value network's assessment of action value, a policy loss function is calculated, and the parameters of the policy network are updated by minimizing the policy loss function. Based on the updated policy network and value network, the parameters of the target network are updated using a soft update method until the policy network converges, thus obtaining the optimal deployment scheme.

8. The method for deploying and routing AI services and microservices in edge scenarios according to any one of claims 1 to 4, characterized in that, The step of calculating the routing path for a user request to reach the network based on the deployment scheme and real-time network status includes: For any user request to be routed, identify the next service type to be accessed, and the set of nodes where service instances of the service type are currently deployed; Based on the number of service instances deployed on each node in the node set and the communication latency between each node and the current node, the forwarding route probability of each node is calculated. Based on the forwarding probability, a routing path is generated when the user request reaches the network.

9. A system for deploying and routing AI services and microservices in edge computing scenarios, characterized in that, include: The model determination unit is used to determine the deployment and request routing model of AI services and microservices in a multi-edge collaborative network scenario based on the acquired network topology data of multiple edge nodes, service characteristic data of AI services and microservices, and user request flow data. The optimization function determination unit is used to determine a joint optimization function based on the deployment and request routing model, with the optimization objectives being end-to-end latency, system energy consumption, and resource utilization. The agent training unit is used to train the agent to learn the optimal deployment strategy by using the joint optimization function as a guide and a deep reinforcement learning algorithm based on the potential energy function and segmented rewards, so as to obtain the deployment scheme of AI service instances and microservice instances on the multiple edge nodes. The path calculation unit is used to calculate the routing path when a user request reaches the network based on the deployment scheme and real-time network status.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the AI ​​service and microservice deployment and request routing method in edge scenarios as described in any one of claims 1 to 8.

Images