A low-orbit satellite micro-service deployment and request routing joint optimization method and system

Abstract

The application provides a low-orbit satellite micro-service deployment and request routing joint optimization method and system, which comprises the following steps: after a target application is divided into multiple micro-services, an optimal micro-service deployment strategy is calculated through a MAPPO algorithm based on the constraints of satellite load balancing and minimum service time delay, and each micro-service instance is deployed to a corresponding low-orbit satellite node according to the optimal micro-service deployment strategy; when a user request is received, the optimal service path and the optimal forwarding path for outputting the user request are calculated through a regional cost-aware hierarchical graph routing algorithm according to the micro-service instance deployment on the low-orbit satellite node and the real-time load information of the satellite. Through the scheme, load balancing of micro-service deployment can be realized, end-to-end service time delay is reduced, satellite resource utilization is maximized, and satellite service efficiency is ensured.

Description

Technical Field

[0001] This invention belongs to the field of low-Earth orbit satellite edge computing, and particularly relates to a method and system for joint optimization of low-Earth orbit satellite microservice deployment and request routing. Background Technology

[0002] As the vision of 6G mobile communication technology becomes clearer, building a globally covered, seamlessly connected, and intelligently integrated "space-ground integrated" information network has become an international consensus. Non-terrestrial networks, especially low-Earth orbit (LEO) satellite constellations, are considered key infrastructure for realizing this vision due to their advantages of wide coverage, low latency, and high bandwidth. At the same time, satellite networks are also gradually evolving from the traditional "transparent relay pipeline" model to an "airborne computing platform" with on-orbit computing, storage, and intelligent processing capabilities, namely "satellite computing." Its core is to extend mature terrestrial cloud computing and edge computing capabilities into space, building a programmable and virtualized on-orbit service infrastructure to support highly resilient information services globally.

[0003] However, traditional monolithic application architectures have revealed significant limitations in satellite environments: First, satellite nodes have extremely limited computing, storage, and energy resources, making it difficult to support complex, resource-intensive monolithic applications, resulting in single-satellite capability bottlenecks and wasted overall constellation potential; second, the LEO constellation topology is highly dynamic, with satellites operating at high speeds of approximately 7.8 km / s, and inter-satellite and satellite-to-ground link relationships changing frequently within minutes, requiring frequent service migrations and switching, which severely restricts monolithic applications that rely on stable operating environments; third, typical future 6G services (such as real-time remote sensing, emergency communication, and the industrial internet) often rely on multi-functional collaboration, while traditional architectures are slow to update and lack flexibility, failing to meet diverse and agile business needs.

[0004] Microservices, by breaking down complex applications into a set of lightweight, functionally independent service units, enable flexible deployment, on-demand scaling, and rapid iteration, significantly improving resource utilization and system adaptability. This characteristic aligns well with the resource constraints and heterogeneous nodes of satellite networks. By dynamically distributing different microservice instances across different satellite nodes, distributed processing of computing tasks and network-wide load balancing can be achieved. However, deploying microservices in LEO satellite networks also faces numerous challenges: LEO satellite networks and terrestrial networks have inherent differences in architecture and operating environment; the high-speed motion of satellites causes frequent switching between inter-satellite links and satellite-to-ground links, resulting in a continuously changing topology. This highly dynamic network topology poses significant challenges to service continuity and efficient resource management. Furthermore, the inherent resource limitations of LEO satellites, including limited computing power, memory, bandwidth, and storage space, further exacerbate the complexity of deployment. The heterogeneity of different satellites also requires deployment algorithms to select the most suitable satellite for each microservice. Based on the mesh topology of low-Earth orbit networks, non-convex optimization objectives, and integer constraints, this microservice deployment problem ultimately evolves into an NP-hard problem with numerous local optima.

[0005] Currently, in publicly available technical solutions, such as the patent with publication number CN116846446A, the approach involves setting a unique service requirement attribute for each microservice, defining the microservice as a virtual network request node, and defining satellite nodes as physical network nodes. Network models are then established for both virtual and physical networks. Network resource attributes are extracted from the physical network to construct a feature matrix, which is then input into a policy network built using basic neural network elements for training. This training yields a set of available satellite nodes and mapping probabilities, with the communication latency between all microservices used as a reward signal. Microservices are then deployed sequentially using the satellite nodes and mapping probabilities, and the placement strategy is adjusted based on the reward signal, thus achieving microservice deployment. However, this approach has two significant problems: first, it does not consider the load balancing issue between different satellites, potentially leading to excessive load on some satellite nodes; second, it cannot perform routing planning for user requests, making it difficult to guarantee real-time service efficiency. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method and system for joint optimization of low-Earth orbit satellite microservice deployment and request routing, which is used to solve the problem of unbalanced load in current microservice deployment and inability to achieve satellite service routing planning.

[0007] In a first aspect of the present invention, a method for joint optimization of low-Earth orbit satellite microservice deployment and request routing is provided, comprising: After dividing the target application into multiple microservices, based on the constraints of satellite load balancing and minimizing service latency, the optimal microservice deployment strategy is calculated using the MAPPO algorithm, and each microservice instance is deployed to the corresponding low-Earth orbit satellite node according to the optimal microservice deployment strategy. Upon receiving a user request, based on the deployment of microservice instances on the low-Earth orbit satellite nodes and the real-time satellite load information, the optimal service path and optimal forwarding path for the user request are calculated and output using the regional cost-aware hierarchical graph routing algorithm. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical directed acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

[0008] In a second aspect of the present invention, a joint optimization system for low-Earth orbit satellite microservice deployment and request routing is provided, comprising: The microservice deployment module is used to divide the target application into multiple microservices, and then calculate the optimal microservice deployment strategy based on the constraints of satellite load balancing and minimizing service latency using the MAPPO algorithm. Based on the optimal microservice deployment strategy, it deploys each microservice instance to the corresponding low-Earth orbit satellite node. The request routing module is used to calculate and output the optimal service path and the optimal forwarding path of the user request after receiving a user request, based on the deployment of microservice instances on the low-Earth orbit satellite nodes and the real-time satellite load information, using a regional cost-aware hierarchical graph routing algorithm. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical directed acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

[0009] In a third aspect of the present invention, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor executes the computer program to implement the steps of the method as described in the first aspect of the present invention.

[0010] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method provided in the first aspect of the present invention.

[0011] In this embodiment of the invention, under long-term constraints of satellite load balancing and minimizing service latency, the MAPPO algorithm is used to achieve optimal microservice deployment. Under short-term constraints, based on the deployment of microservice instances on low-Earth orbit satellite nodes and real-time satellite load information, a region cost-aware hierarchical graph routing algorithm is used to achieve optimal request routing. This not only achieves load balancing of satellite nodes and maximizes satellite resource utilization, but also reduces end-to-end service latency. Furthermore, it achieves joint optimization of deployment and routing, ensuring satellite service efficiency and meeting the business requests of a large number of users. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a schematic diagram of the Walker constellation configuration provided in one embodiment of the present invention; Figure 2 This is a schematic diagram of the overall architecture of a satellite edge computing network provided in one embodiment of the present invention; Figure 3 This is a flowchart illustrating a method for joint optimization of microservice deployment and request routing for low-Earth orbit satellites, provided in one embodiment of the present invention. Figure 4 This is a schematic diagram of the network architecture of the MAPPO algorithm provided in one embodiment of the present invention; Figure 5 A schematic diagram of the mapping relationship of a hierarchical logic diagram provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a joint optimization system for low-Earth orbit satellite microservice deployment and request routing, provided in one embodiment of the present invention. Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0014] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0015] It should be understood that the terms "comprising" and other similar expressions in the specification, claims, and accompanying drawings of this invention are intended to cover a non-exclusive inclusion, such as a process, method, system, or apparatus that includes a series of steps or units and is not limited to the listed steps or units. Furthermore, "first" and "second" are used to distinguish different objects and are not intended to describe a specific order.

[0016] The Walker constellation configuration consists of M×N satellites, as shown below. Figure 1 As shown, satellites are interconnected through intra-orbital inter-satellite links and inter-orbital inter-satellite links (e.g., X-links), forming a grid-like topology. To address the topology dynamics caused by the high-speed motion of satellites, this embodiment introduces the concept of time slices, where the network topology is considered to be in a relatively stable state within each time slice period.

[0017] In another embodiment, a satellite edge computing network system architecture is provided, such as Figure 2 As shown, this architecture consists of a ground station (G), a set of N low-Earth orbit satellites forming an edge node cluster, and ground users. Each satellite is treated as an edge computing server, equipped with heterogeneous on-board computing resources. The ground station, as the central control unit of the system, possesses global information about the entire satellite system, including satellite positions, resource occupancy status, and channel conditions. Given limited satellite resources, applications are broken down into a series of application microservices and deployed on the satellites to meet the specific needs of ground users. To ensure service effectiveness, the constraints of the Service Function Chain (SFC) between microservices must be considered.

[0018] The defining characteristic of a microservice architecture is the decomposition of a monolithic application into multiple independent modules with specific functionalities. These modules collaborate to fulfill requests. Furthermore, multiple instances of each microservice exist across the network. These microservice instances are arranged in a chained structure to form Service Chains (SFCs), with each service chain corresponding to a specific request. When a request arrives at the network, it is first placed into multiple input queues and then forwarded to edge servers where the corresponding microservice instances are deployed for further processing to satisfy the user's needs.

[0019] Please see Figure 3 The present invention provides a flowchart illustrating a joint optimization method for low-Earth orbit satellite microservice deployment and request routing, comprising: S301. After dividing the target application into multiple microservices, based on the constraints of satellite load balancing and minimizing service latency, the optimal microservice deployment strategy is calculated using the MAPPO algorithm, and each microservice instance is deployed to the corresponding low-Earth orbit satellite node according to the optimal microservice deployment strategy. The target application, or target application, can be deployed on low-Earth orbit satellite nodes to provide corresponding services to ground users. After dividing the target application into multiple microservices, the MAPPO (Multi-Agent Proximal Policy Optimization) algorithm is used to implement the microservice deployment.

[0020] The target application consists of different microservices with SFC (Service Function Chaining) constraints, which can be represented as follows: , where K is the total number of microservices. Represents a collection of different resource types. Indicates the number of resource types. Each satellite node The capacity of heterogeneous resources on a vector can be represented as a vector. ,in Represents a node Up Resources Available capacity.

[0021] The calling relationships of SFCs can be modeled by a directed acyclic graph, and their mathematical representation can be expressed using an adjacency matrix. express:

[0022] in, microservices After processing, its output is used as a microservice. The input, otherwise .

[0023] To achieve load balancing, parallel processing, and improved service reliability, each microservice can be deployed as multiple independent container instances. The number of containers is expressed as .

[0024] To describe the deployment strategy, a binary decision variable is defined. ,when =1 indicates microservices. The One container instance is deployed on the satellite node. If it is above, then it is 0; otherwise, it is 0. Meanwhile, the resource requirements of each microservice are represented by a vector. It means that, among them microservices Resources The demand.

[0025] Optionally, a load balancing model is established based on the utilization rates of different types of resources within each satellite node and the utilization rates of the same type of resources on each satellite node. The load balancing model is as follows:

[0026] Based on the M / M / C queuing theory, an end-to-end service latency model is established, which is as follows:

[0027] In the formula, B This indicates the load balancing degree, where α is the weighting factor. N Indicates the number of low-orbit satellites. i, j Represents a counting variable. Indicates the balance of resource utilization within the satellite. Indicates the number of resource types. Indicates the degree of balance in the utilization of inter-satellite resources. Indicates the total service latency. Indicates uplink latency. This indicates the on-board execution delay. This indicates the inter-satellite communication delay. This indicates downlink latency.

[0028] To efficiently utilize network resources and prevent some satellite nodes from becoming performance bottlenecks due to resource overload, a resource balancing model needs to be constructed. First, satellite nodes are defined. Upper utilization rate of seed resources for:

[0029] In the formula, Indicates deployment on satellite The ratio of the total resource requirements of all microservice instances on the node to the total resources of that node.

[0030] The first part of the resource balancing model is intra-satellite resource balancing, which aims to ensure that different types of resources (such as CPU and memory) can be used in a balanced manner within a single satellite node.

[0031]

[0032]

[0033] Another part of the resource balancing model describes the utilization of the same type of resources on different nodes. As the number of microservice instances increases, it is desirable to utilize all satellite resources to avoid severe task backlog on a few satellite nodes. Resources on different nodes The degree of balance can be expressed as:

[0034]

[0035] Therefore, the final resource balance formula is: .

[0036] Construct a server-side end-to-end latency model to measure the total time elapsed from when a user initiates a service request to when the final result is received. For a task initiated by user u and requesting the execution of service chain C, its end-to-end latency can be decomposed into four main parts: uplink latency ( ), downlink latency On-board execution latency ) and inter-satellite communication latency ( ).

[0037] Uplink latency refers to the time it takes for a user to request data transmission to the first access satellite, while downlink latency refers to the time it takes for the processing result to be transmitted back to the user from the last satellite. Both are composed of propagation latency and transmission latency.

[0038] In the formula, It is the distance between user u and satellite s. It's the speed of light. It refers to the amount of input or output data for a microservice. It is the communication bandwidth between the user and the satellite.

[0039] On-board execution latency is the core of the latency model, when a microservice... The processing request reached its deployed satellite The resulting latency consists of two parts: the average queuing latency due to resource waiting, and the average processing latency of the microservice task itself. Using the M / M / C queuing theory to model this process, its total execution latency can be precisely expressed as:

[0040] Processing latency is the average service rate of a single service instance. The reciprocal of the average queuing delay. The calculation is based on the analysis of the steady-state behavior of the system.

[0041] Therefore, the total on-board execution latency is:

[0042] In the M / M / C queuing model, both the request arrival process and the service process follow a Poisson distribution or an exponential distribution, and the parameter "C" represents the satellite... The above is a microservice The number of instances allocated, denoted as The service rate of a single instance is determined by the satellite's computing power. Compared to the workload of microservices themselves The decision is made based on the fundamental principles of queuing theory. When the average request arrival rate exceeds the processing capacity, it leads to an infinitely long queue, causing network congestion and consequently a decline in network service quality. Therefore, to ensure system stability, the service strength must be less than 1.

[0043]

[0044]

[0045] In the formula, This indicates a request to route to a satellite. Microservices Total arrival rate Indicates satellite The service speed.

[0046] When the service instances in the system reach a steady state, the microservice queuing latency... According to Little's Law, the calculation is as follows:

[0047]

[0048]

[0049] Inter-satellite communication latency is when microservices in a service chain... After completion, its output data It needs to be passed to subsequent microservices Based on the adjacency matrix, the inter-satellite communication latency of the microservice function chain can be expressed as:

[0050] In summary, the total communication latency of service chain c in the on-board network It equals the sum of the execution latency of all microservices along the actual routing path and the communication latency of all links. .

[0051] Optionally, the optimization objective can be to minimize the weighted sum of satellite load imbalance and service latency, and the optimization direction of microservice deployment can be controlled by adjusting the weighting coefficients.

[0052] This embodiment aims to find an optimal microservice container instance deployment strategy, which can take minimizing the weighted sum of resource load imbalance and service latency as the optimization objective. This is a multi-objective optimization problem. It can be formally expressed as: Min

[0053] st:

[0054]

[0055] In the formula, Indicates the degree of resource balance. Indicates the overall time delay. As weighting coefficients, they can be based on The overall optimization goal of control is to prioritize either achieving instance and resource load balancing or minimizing latency.

[0056] when When the load is large, the system will deploy more microservice instances across different satellites to achieve load balancing. When the latency is low, the system will deploy instances to the same or nearby satellite nodes to minimize latency as much as possible.

[0057] In this embodiment, the MAPPO algorithm for satellite microservice deployment based on load balancing and latency optimization can ensure load balancing between and within satellite nodes while minimizing service latency.

[0058] S302. Upon receiving a user request, based on the microservice instance deployment on the low-orbit satellite node and the real-time satellite load information, the optimal service path and optimal forwarding path for the user request are calculated and output using the regional cost-aware hierarchical graph routing algorithm. Upon receiving a request, the ground station searches for the optimal service path and forwarding path using a hierarchical graph routing algorithm, based on the microservice instances deployed on the low-Earth orbit satellite nodes and the real-time satellite load. Corresponding service chains can be constructed based on the deployed microservice instances, and satellite nodes can be selected based on the real-time satellite load.

[0059] The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical directed acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

[0060] Microservice instance deployment primarily achieves resource allocation over a long timescale, i.e., "which microservice instances should be deployed or migrated to which satellite nodes within which time window," with the goal of forming a globally optimal service topology. Routing strategies, on the other hand, make online routing decisions among distributed on-board instances for a large number of SFC (Service Function Chain) requests arriving in milliseconds over a short timescale. Routing path planning needs to achieve a joint optimization process of network links and compute instance load balancing while meeting end-to-end latency constraints.

[0061] Region Cost Aware Layered-Graph Routing (RCALR) is a algorithm that maps a multi-stage, multi-objective joint decision-making problem to a dynamically constructed layered logical graph to efficiently solve a single-source shortest path problem.

[0062] Specifically, the routing decisions of the service function chain of the satellite node are mapped to a hierarchical directed acyclic graph, wherein the hierarchical structure of the directed acyclic graph is the same as the service order of the service function chain. In the directed acyclic graph, the virtual link weight is calculated based on the routing delay cost, resource load cost, and network fluctuation cost, and the virtual link weight is dynamically cached. Based on virtual link weights, the optimal service path for microservices is obtained by solving the single-source shortest path with the lowest overall cost on the directed acyclic graph using a dynamic programming algorithm. Based on the optimal service path, a load-aware greedy strategy is used to make hop-by-hop forwarding decisions at each hop of data packet transmission, based on the link load of neighboring nodes and the distance to the target node, to generate the optimal forwarding path.

[0063] To structure the complex routing decision space of SFC requests, such as Figure 5 As shown, all instance-level routing paths are abstracted into a hierarchical logical graph. The hierarchical structure of this diagram is isomorphic to the service dependency order of SFC. It contains j levels, the k-th level Includes microservices All available instances.

[0064] Preferably, when the maximum resource utilization of a satellite node exceeds a preset threshold, the instances on the corresponding satellite nodes in the hierarchical directed acyclic graph are pruned, and the routing path between two microservice instances is restricted to the Manhattan distance.

[0065] During graph construction, to ensure service quality, instances on satellite nodes whose computational load has exceeded a preset critical threshold will be proactively removed. A single instance located on a satellite node... Examples on A node is pruned if and only if its maximum resource utilization is reached. Exceeding the preset threshold At the same time, the routing path between the two service instances needs to be limited to the Manhattan distance. This strategy effectively reduces the search space and ensures the stability of the routing path.

[0066] In the hierarchical logic diagram, different layer instances are connected. (Located at satellite node) )and (Located at satellite node) ) have different virtual link weights The virtual link weight consists of the following three dimensions: (1) Routing delay cost It quantifies the service quality of the routing path. It directly reflects the quality of service from the instance. Routing to instance The resulting total latency is quantized by the end-to-end latency of the SFC:

[0067] (2) Resource load cost This represents the scarcity of resources and congestion risk on the target satellite. The cost is for the target node. The non-linear function of resource load is designed to increase the cost of routing to high-load nodes.

[0068] In the formula, It is a node Upper Utilization rate of such resources This is the weighting coefficient for this type of resource. When the utilization rate approaches 1, the cost will increase dramatically, thus effectively avoiding potential computational bottlenecks.

[0069] (3) Network volatility costs : Evaluate the congestion and stability of the local routing environment. This cost affects the nodes. and The rectangular routing area formed A macro-level assessment of network health status is conducted, based on the load of all links within the region. The expected value and variance together determine:

[0070] In the formula, the expectation term represents the average congestion level of the region, while the variance term quantifies the jitter and uncertainty of network traffic. It is an adjustable risk aversion factor. This cost function gives the algorithm the ability to perceive and avoid local network congestion and systemic risks.

[0071] Preferably, a dynamic weight caching mechanism based on service dependency type is used to reuse the virtual link weights of the same microservice type within a predetermined time scale.

[0072] To improve the computational efficiency of the algorithm when handling high-concurrency SFC requests, a dynamic weight caching mechanism based on service dependency types is introduced. Since the network state is relatively stable within the same short timescale, the weights of virtual links connecting the same microservice type have high reusability. The algorithm will assign weights to each service jump type. Maintain a dynamic weight cache matrix The elements of this matrix Stored the current time interval Inside, from the example To instance Virtual link weights. When building a hierarchical graph for a new SFC request, these pre-computed weights will be reused from the cache first, and only weights that are not in the cache or have expired (due to significant changes in network conditions) will be calculated or updated.

[0073] Virtual link weights in a hierarchical logic graph By combining and weighting multidimensional costs, to ensure uniformity of all cost dimensions, all costs are first mapped to a value using a Min-Max normalization function. The interval, expressed as follows:

[0074] In the formula, , , This represents the normalized cost. Next, the RCALR algorithm dynamically adjusts the weight coefficients based on the network's overall state or service priorities. To achieve flexible adaptive adjustment, and .

[0075] Once the weights are determined, finding the optimal SFC execution path transforms into solving a Single-Source Shortest Path (SSSP) problem on the hierarchical logical graph, from the virtual source to the virtual sink. Since the hierarchical logical graph is constructed according to the service dependency order of the SFC, it naturally forms a Directed Acyclic Graph (DAG). For the shortest path problem on a DAG, dynamic programming can be used to efficiently solve for the optimal logical service path by calculating layer by layer from bottom to top (or top to bottom). .

[0076] Specifically, let's set To reach the first point from the source point Layer Instance The shortest path. Then the recurrence relation is:

[0077] Through iterative calculations layer by layer, a logical service path with the lowest overall cost from source to sink can be found. This service path represents the best choice and sequence of microservices.

[0078] However, this logical path only defines "which service instances it passes through," without specifying the actual physical forwarding path of the data packets between the satellite nodes where the two instances reside. To cope with the dynamically changing load conditions of inter-satellite links, RCALR employs a load-aware greedy strategy for hop-by-hop forwarding.

[0079] When the data packet is located from the satellite node To satellite node Any intermediate satellite node on the path At that time, the next hop node From its neighbor node set Choose the node closer to you Nodes with lower inter-satellite link load:

[0080] In the formula, From neighboring nodes to the target node Manhattan distance, It is a connection and The current load of the inter-satellite links. , This is the weighting factor.

[0081] By combining macroscopic logical path planning with local lightweight greedy search, a two-stage routing strategy can avoid high-overhead end-to-end global physical path calculation, significantly improving request routing planning efficiency. This ensures the global optimality of the SFC execution path at the service level while also providing packets with the flexibility and adaptability to avoid transient congestion at the network level.

[0082] In one embodiment, the method further includes the following steps prior to step S301: Define each microservice instance as an agent, and define the agent's state space, action space, and reward function: The state space includes satellite node resource utilization, microservice instance deployment index and satellite node location; the action space is to select visible satellites for deployment based on the observation space; and the reward function is to minimize service latency and load balancing. Historical decision data stored in the shared experience replay buffer is used as training data. The historical decision data includes satellite global state information, local observation information, and deployment decision information. The ground station uses the global state information to centrally train a centralized Critic network to evaluate the overall state of microservice deployment. Each agent trains its own Actor network based on local observation information to autonomously generate microservice deployment decisions within the visible satellite range.

[0083] Each microservice instance acts as an agent, with each agent possessing its own Actor network and sharing a Critic network. The ground station can acquire the global state and sample the global state, current action, corresponding reward, and next global state from the shared experience replay buffer to perform centralized training on the Critic network.

[0084] Microservices need to be deployed from the ground to suitable satellites as instances. The observation space includes the resource utilization and location of visible satellites. The action is to select the optimal satellite for deployment from these visible satellites. The deployment process is trained at the ground station using global state information, which consists of the observations and actions of all agents. This process can be represented as a POMDP (Partially Observable Markov Decision Process), as follows: (1) State Design: Global state information includes the resource occupancy of satellite nodes, the deployment status of instances, and the location of satellites. It is only used during the centralized training phase at the ground station and can be represented as follows: The details are as follows:

[0085]

[0086] ;

[0087] In the formula, Indicates satellite node Up Resources utilization rate This indicates the satellite index to which the microservice instance is deployed. Indicates satellite node The location.

[0088] Each agent (i.e., microservice) The decision-making basis (in the distributed execution phase) of the instance. It only contains the resource utilization and location information of nearby satellites that the agent can observe, and is a subset of the global state.

[0089] (2) Action Space: Each agent cannot access global state information. Based on its position, the microservice... Examples The observation space can be represented as This represents the resource utilization and location information of neighboring satellites. Based on its own observation space, microservices... The above example The action space can be represented as:

[0090] In the formula, It is the number of visible satellites within the service area. This indicates that the container is deployed to the node. superior.

[0091] (3) Reward Function: The optimization objective is to minimize latency and balance resource utilization, which is also considered a reward. Based on the optimization objective, the reward function can be expressed as:

[0092] In the formula, Indicates the weighting factor. Indicates the degree of resource balance. This indicates the overall time delay.

[0093] Based on the aforementioned POMDP model, an online reinforcement learning algorithm based on the Actor-Critic framework is partially adopted. Each service instance acts as an agent with partial observations. Training is conducted centrally at ground stations, using global state information to train a centralized Critic network to evaluate the overall state. Decisions are then made autonomously by each agent based on its local observations through a distributed Actor network.

[0094] The network architecture of the MAPPO algorithm is as follows: Figure 4 As shown, each agent has an Actor network with parameters as follows: And share a Critic network in the training center. The input to the Actor network is the local observations of the current agent. The output is a probability distribution of deployment actions within its visible satellite range. The Critic network is a centralized network deployed at ground stations. Its input is the global state. (A set of local observations from all agents), the output is a value assessment of the current global state. The aim is to assess the merits and demerits of the current overall deployment and resource allocation. The training process is generally conducted centrally at ground stations. Preferably, after the ground station completes the training of the Actor network, it periodically transmits the updated Actor network parameters to the satellite nodes, and synchronously updates the Actor network on the satellite nodes.

[0095] After centralized training at ground stations, the updated Actor network parameters are periodically transmitted to the agents on satellites. MAPPO training follows an online “sample-update-drop” model.

[0096] All agents use the current latest state. and Actor strategy Interact with the environment in parallel and execute deployment actions. After all agents have performed their actions, a global reward is returned. And the environment will transition to the next state. This data is then processed to form a tuple, which is sent to the ground station for storage. .

[0097] In one embodiment, the ground station collects deployment decisions generated by each agent on the satellite node using the latest Actor network, and uses a centralized Critic network to calculate the generalized advantage estimate of the deployment decision at each time step; The Critic and Actor networks of the ground station are updated using gradients based on the generalized dominance estimates at each time step.

[0098] After the ground station collects all agent data, it uses a centralized Critic network to calculate the generalized advantage estimate for each time step. :

[0099] In the formula, yes The smoothing parameter. This indicates that the action taken in this state is better than the average strategy, and vice versa. It is in time The temporal difference error represents the difference between the value estimate at the current time and the value estimate at the next time, and is calculated using the following formula:

[0100] In the formula, It is the global reward at time t (the weighted sum of the rewards of all agents). It is a discount factor.

[0101] Based on the collected data and calculated advantage values, the network parameters are updated using multiple rounds of mini-batch gradient updates. The shared Critic network minimizes its predicted state value. The true return estimated by the Monte Carlo method The update is performed using the mean squared error loss between the two values, and its loss function can be:

[0102] For each intelligent agent Each Actor network updates independently. The goal of each Actor network is to maximize its own pruning agent objective function. For the agent... Its objective function is:

[0103] In the formula, It is a small hyperparameter. The clip function will compare the probability to... Limited to Within the interval, to prevent deviation from the stable region due to excessively large single updates. The objective function is maximized using gradient ascent, with each agent... Actor Network They can steadily learn to make better decisions based on their respective local observations.

[0104] In one embodiment, part of the pseudocode for the MAPPO algorithm for deploying satellite microservices based on load balancing and latency optimization includes: Input: Global state information, initial parameters for the Actor and Critic networks, discount factor. , parameters

[0105] Output: Optimal microservice instance deployment decision Initialize Actor network parameters for each agent and centralized Critic network

[0106] for training rounds do for time step do for each agent do Based on its local observations According to the Actor strategy Get the action .

[0107] end for Perform joint actions in the environment Receive global rewards and below A global state Collected data tuples Store in replay buffer B.

[0108] end for Using data from storage area B and the Criti network Calculate the dominance function for all time steps. and returns .

[0109] for optimizing rounds do A small batch of data is randomly sampled from storage area B.

[0110] By minimizing the mean square error loss Update the Critic network.

[0111] For each agent i=1,2,...,Ndo pass Update the Actor network for each agent.

[0112] end for end for end for In one embodiment, part of the pseudocode for the region cost-aware hierarchical graph routing algorithm is as follows: Input: Service Function Chain (SFC), resource utilization and load information of each microservice instance, communication latency, load, and network volatility of each link, and virtual link weight coefficients. Output: Optimal service path and forwarding path

[0113] Initialize the hierarchical logic diagram A hierarchical directed acyclic graph (DAG) is constructed based on the service function chain (SFC), where each layer contains candidate deployment nodes for microservice instances.

[0114] for do for do if then Healthy instances Add as a node to the graph middle end if end for for do Calculate routing delay cost Resource load cost Network volatility costs

[0115] Calculate virtual link weights

[0116] end for end for for to do for each node Layer k do

[0117]

[0118] end for end for

[0119] Using the stored predecessor pointer ,from Perform backtracking to obtain the optimal service path

[0120] Based on the optimal service path By using a greedy strategy to forward the message, you can obtain...

[0121] return .

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] Figure 6 This invention provides a schematic diagram of a joint optimization system for low-Earth orbit satellite microservice deployment and request routing, comprising: The microservice deployment module 610 is used to divide the target application into multiple microservices, calculate the optimal microservice deployment strategy based on the constraints of satellite load balancing and minimum service latency using the MAPPO algorithm, and deploy each microservice instance to the corresponding low-Earth orbit satellite node according to the optimal microservice deployment strategy. Optionally, the microservice deployment module 610 includes: The load balancing model construction unit is used to establish a load balancing model based on the utilization rates of different types of resources within each satellite node and the utilization rates of the same type of resources on each satellite node. The load balancing model is as follows:

[0124] The latency model construction unit is used to establish an end-to-end service latency model based on the M / M / C queuing theory. The service latency model is as follows:

[0125] In the formula, B This indicates the load balancing degree, where α is the weighting factor. N Indicates the number of low-orbit satellites. i, j Represents a counting variable. Indicates the balance of resource utilization within the satellite. Indicates the number of resource types. Indicates the degree of balance in the utilization of inter-satellite resources. Indicates the total service latency. Indicates uplink latency. This indicates the on-board execution delay. This indicates the inter-satellite communication delay. This indicates downlink latency.

[0126] Preferably, the optimization objective is to minimize the weighted sum of satellite load imbalance and service latency, and the optimization direction of microservice deployment is controlled by adjusting the weighting coefficients.

[0127] The request routing module 620 is used to calculate and output the optimal service path and the optimal forwarding path of the user request based on the deployment of microservice instances on the low-orbit satellite nodes and the real-time load information of the satellites, using a regional cost-aware hierarchical graph routing algorithm after receiving a user request. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical directed acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

[0128] Optionally, the routing decision module 620 includes: The graph construction unit is used to map the routing decisions of the service function chain of the satellite node into a hierarchical directed acyclic graph, wherein the hierarchical structure of the directed acyclic graph is the same as the service order of the service function chain. The weight calculation unit is used to calculate the virtual link weight based on routing delay cost, resource load cost and network fluctuation cost in the directed acyclic graph, and to dynamically cache the virtual link weight. The path planning unit is used to solve the single-source shortest path with the lowest overall cost on the directed acyclic graph based on virtual link weights and through dynamic programming algorithms, so as to obtain the optimal service path of the microservice. The path search unit is used to generate the optimal forwarding path by making hop-by-hop forwarding decisions at each hop of data packet transmission based on the link load of neighboring nodes and the distance to the target node using a load-aware greedy strategy.

[0129] Preferably, when the maximum resource utilization of a satellite node exceeds a preset threshold, the instances on the corresponding satellite nodes in the hierarchical directed acyclic graph are pruned, and the routing path between two microservice instances is restricted to the Manhattan distance.

[0130] In one embodiment, the process of calculating the optimal microservice deployment strategy using the MAPPO algorithm further includes: Define each microservice instance as an agent, and define the agent's state space, action space, and reward function: The state space includes satellite node resource utilization, microservice instance deployment index and satellite node location; the action space is to select visible satellites for deployment based on the observation space; and the reward function is to minimize service latency and load balancing. Historical decision data stored in the shared experience replay buffer is used as training data. The historical decision data includes satellite global state information, local observation information, and deployment decision information. The ground station uses the global state information to centrally train a centralized Critic network to evaluate the overall state of microservice deployment. Each agent trains its own Actor network based on local observation information to autonomously generate microservice deployment decisions within the visible satellite range.

[0131] Preferably, after the ground station completes the training of the Actor network, it periodically transmits the updated Actor network parameters to the satellite nodes, and synchronously updates the Actor network on the satellite nodes.

[0132] Optionally, the ground station collects the deployment decisions generated by each agent on the satellite node using the latest Actor network, and uses a centralized Critic network to calculate the generalized advantage estimate of the deployment decision at each time step; The Critic and Actor networks of the ground station are updated using gradients based on the generalized dominance estimates at each time step.

[0133] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0134] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device is used to implement microservice deployment and request routing for satellites. Figure 7 As shown, the electronic device 70 of this embodiment includes a memory 710, a processor 720, and a system bus 730. The memory 710 includes an executable program 7101 stored thereon. As those skilled in the art will understand, Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0135] The following is combined with Figure 7 A detailed introduction to each component of the electronic device: The memory 710 can be used to store software programs and modules. The processor 720 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 710. The memory 710 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device (such as cached data), etc. In addition, the memory 710 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0136] An executable program 7101 containing optimization methods is stored in memory 710. This executable program 7101 can be divided into one or more modules / units, which are stored in memory 710 and executed by processor 720 to achieve joint optimization of microservice instance deployment and request routing for low-Earth orbit satellites. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, describing the execution process of the executable program 7101 in the electronic device 70. For example, the executable program 7101 can be divided into functional modules such as a microservice deployment module and a request routing module.

[0137] The processor 720 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 710, and by calling data stored in the memory 710, it performs various functions and processes data, thereby monitoring the overall status of the electronic device. Optionally, the processor 720 may include one or more processing units; preferably, the processor 720 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, application programs, etc., and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 720.

[0138] The system bus 730 is used to connect various functional components within the computer, transmitting data, address, and control information. Its type can be, for example, a PCI bus, an ISA bus, or a CAN bus. Instructions from the processor 720 are transmitted to the memory 710 via the bus, and the memory 710 sends data back to the processor 720. The system bus 730 handles the data and instruction exchange between the processor 720 and the memory 710. Of course, the system bus 730 can also connect to other devices, such as network interfaces and display devices.

[0139] In this embodiment of the invention, the executable program executed by the processor 720 included in the electronic device includes: After dividing the target application into multiple microservices, based on the constraints of satellite load balancing and minimizing service latency, the optimal microservice deployment strategy is calculated using the MAPPO algorithm, and each microservice instance is deployed to the corresponding low-Earth orbit satellite node according to the optimal microservice deployment strategy. Upon receiving a user request, based on the deployment of microservice instances on the low-Earth orbit satellite nodes and the real-time satellite load information, the optimal service path and optimal forwarding path for the user request are calculated and output using the regional cost-aware hierarchical graph routing algorithm. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical directed acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

[0140] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0141] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0142] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for joint optimization of microservice deployment and request routing on low-Earth orbit satellites, characterized in that, include: After dividing the target application into multiple microservices, based on the constraints of satellite load balancing and minimizing service latency, the optimal microservice deployment strategy is calculated using the MAPPO algorithm, and each microservice instance is deployed to the corresponding low-Earth orbit satellite node according to the optimal microservice deployment strategy. Upon receiving a user request, based on the deployment of microservice instances on the low-Earth orbit satellite nodes and the real-time satellite load information, the optimal service path and optimal forwarding path for the user request are calculated and output using the regional cost-aware hierarchical graph routing algorithm. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

2. The method according to claim 1, characterized in that, The process of calculating the optimal microservice deployment strategy using the MAPPO algorithm, based on constraints of satellite load balancing and minimizing service latency, includes the following: Based on the utilization rates of different types of resources within each satellite node and the utilization rates of the same type of resources on each satellite node, a load balancing model is established. The load balancing model is as follows: Based on the M / M / C queuing theory, an end-to-end service latency model is established, which is as follows: In the formula, B This indicates the load balancing degree, where α is the weighting factor. N Indicates the number of low-orbit satellites. i, j Represents a counting variable. Indicates the balance of resource utilization within the satellite. Indicates the number of resource types. Indicates the degree of balance in the utilization of inter-satellite resources. Indicates the total service latency. Indicates uplink latency. Indicates on-board execution delay. This indicates the inter-satellite communication delay. This indicates downlink latency.

3. The method according to claim 1, characterized in that, The optimal microservice deployment strategy calculated using the MAPPO algorithm, based on constraints of satellite load balancing and minimizing service latency, includes: The optimization objective is to minimize the weighted sum of satellite load imbalance and service latency, and the optimization direction of microservice deployment is controlled by adjusting the weighting coefficients.

4. The method according to claim 1, characterized in that, Before calculating the optimal microservice deployment strategy using the MAPPO algorithm, the following steps are also included: Define each microservice instance as an agent, and define the agent's state space, action space, and reward function: The state space includes satellite node resource utilization, microservice instance deployment index and satellite node location; the action space is to select visible satellites for deployment based on the observation space; and the reward function is to minimize service latency and load balancing. Historical decision data stored in the shared experience replay buffer is used as training data. The historical decision data includes satellite global state information, local observation information, and deployment decision information. The ground station uses the global state information to centrally train a centralized Critic network to evaluate the overall state of microservice deployment. Each agent trains its own Actor network based on local observation information to autonomously generate microservice deployment decisions within the visible satellite range.

5. The method according to claim 4, characterized in that, The training of each agent's respective Actor network based on local observation information also includes: After the ground station completes the training of the Actor network, it periodically transmits the updated Actor network parameters to the satellite nodes, synchronously updating the Actor network on the satellite nodes.

6. The method according to claim 4, characterized in that, The optimal microservice deployment strategy calculated using the MAPPO algorithm, based on constraints of satellite load balancing and minimizing service latency, also includes: The ground station collects deployment decisions generated by each agent on the satellite node using the latest Actor network, and uses a centralized Critic network to calculate the generalized advantage estimate of the deployment decision at each time step; The Critic and Actor networks of the ground station are updated using gradients based on the generalized dominance estimates at each time step.

7. The method according to claim 1, characterized in that, The calculation of the optimal service path and optimal forwarding path for the user request using the regional cost-aware hierarchical graph routing algorithm includes: The routing decisions of the service function chain of the satellite node are mapped to a hierarchical directed acyclic graph, wherein the hierarchical structure of the directed acyclic graph is the same as the service order of the service function chain. In the directed acyclic graph, the virtual link weight is calculated based on the routing delay cost, resource load cost, and network fluctuation cost, and the virtual link weight is dynamically cached. Based on virtual link weights, the optimal service path for microservices is obtained by solving the single-source shortest path with the lowest overall cost on the directed acyclic graph using a dynamic programming algorithm. Based on the optimal service path, a load-aware greedy strategy is used to make hop-by-hop forwarding decisions at each hop of data packet transmission, based on the link load of neighboring nodes and the distance to the target node, to generate the optimal forwarding path.

8. The method according to claim 7, characterized in that, The method of mapping the routing decisions of the service function chain of satellite nodes to a hierarchical directed acyclic graph also includes: When the maximum resource utilization of a satellite node exceeds a preset threshold, the instances on the corresponding satellite nodes in the hierarchical directed acyclic graph are pruned, and the routing path between two microservice instances is restricted to the Manhattan distance.

9. A joint optimization system for low-Earth orbit satellite microservice deployment and request routing, characterized in that, include: The microservice deployment module is used to divide the target application into multiple microservices, and then calculate the optimal microservice deployment strategy based on the constraints of satellite load balancing and minimizing service latency using the MAPPO algorithm. Based on the optimal microservice deployment strategy, it deploys each microservice instance to the corresponding low-Earth orbit satellite node. The request routing module is used to calculate and output the optimal service path and the optimal forwarding path of the user request after receiving a user request, based on the deployment of microservice instances on the low-Earth orbit satellite nodes and the real-time satellite load information, using a regional cost-aware hierarchical graph routing algorithm. The regional cost-aware hierarchical graph routing algorithm constructs a hierarchical directed acyclic graph based on the satellite service function chain, then searches the hierarchical acyclic graph for the routing path with the lowest overall cost to obtain the optimal service path, and finally searches for the optimal forwarding path based on the optimal service path using a load-aware greedy strategy.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the low-orbit satellite microservice deployment and request routing joint optimization method as described in any one of claims 1 to 8.

Images