A micro-service routing mechanism optimization method based on SDN combined with reinforcement learning technology
By combining SDN and reinforcement learning techniques, a multi-dimensional decision model is constructed to dynamically optimize microservice routing strategies. This addresses the limitations of static routing strategies in cloud-native microservice architectures, enabling adaptive optimization and network layer path adjustment, thereby improving system stability and throughput performance.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIHANG UNIV
- Filing Date
- 2026-03-18
- Publication Date
- 2026-05-29
AI Technical Summary
In existing cloud-native microservice architectures, traffic routing mechanisms rely on static rules or simple load balancing algorithms, lacking dynamic adaptive capabilities. This can lead to concentrated requests, increased response latency, or even service avalanche when the load changes.
By combining Software-Defined Networking (SDN) and reinforcement learning techniques, a multi-dimensional operational state decision model is constructed. Through reinforcement learning, the microservice status is dynamically monitored and routing strategies are optimized. Combined with the global network view and programmability of SDN, the underlying path is adjusted to achieve collaborative optimization between the application layer and the network layer.
It significantly improves the stability and resource utilization of cloud-native microservice systems under complex load conditions, reduces service response latency, and enhances overall throughput and the intelligence level of traffic scheduling.
Smart Images
Figure CN122120201A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data communication technology. Specifically, this invention relates to a microservice routing mechanism optimization method that combines software-defined networking and reinforcement learning techniques. Background Technology
[0002] With the development of cloud computing technology, cloud-native microservice architecture has gradually become the mainstream architecture for next-generation distributed systems. This architecture breaks down traditional monolithic applications into multiple functionally independent, loosely coupled microservice units. These microservices interact through lightweight communication mechanisms, are encapsulated and deployed using containerization technology, and leverage container orchestration platforms for automatic scheduling and elastic scaling. In this architecture, service instances can dynamically scale up or down based on load, service locations are dynamic and uncertain, and inter-service communication is frequent and complex, significantly improving system flexibility and scalability. However, due to the large number of microservices, complex call chains, frequent dynamic changes in the operating environment, and the fact that existing microservice routing strategies often rely on static rules or simple algorithms, such as those based on preset weights or round-robin mechanisms, they lack deep awareness and adaptive capabilities regarding real-time service performance, network congestion, and changes in business load. How to achieve efficient and intelligent inter-service communication and traffic scheduling while ensuring performance and stability has become a significant technical challenge for cloud-native microservice architecture.
[0003] In recent years, with the rapid improvement of computing power, artificial intelligence technologies, represented by reinforcement learning (RL), have flourished. Reinforcement learning is a method that continuously optimizes decision-making strategies through interaction between an agent and its environment, based on a reward feedback mechanism. Its core idea is to approximate the optimal strategy through trial and error in a dynamic environment. Compared with traditional rule-based or supervised learning methods, reinforcement learning can achieve continuous learning and adaptive optimization in unknown or complex environments, thus having broad application potential in areas such as resource scheduling, network optimization, and intelligent control. In cloud-native microservice environments, reinforcement learning can be used to dynamically adjust traffic allocation and routing strategies based on multi-dimensional indicators such as service load, response time, error rate, and network status to achieve adaptive load balancing.
[0004] Software-defined networking (SDN) decouples the network control plane from the data plane, enabling centralized programmable control of the network. This allows for dynamic configuration of network path selection and traffic scheduling via software. Compared to traditional distributed network control models, SDN technology offers greater flexibility and a global view, enabling optimization and adjustment of traffic paths based on real-time network conditions. In cloud-native microservice architectures, SDN can be used to optimize cross-node service communication paths, alleviate network congestion, and implement link-level traffic scheduling, thereby improving overall communication efficiency.
[0005] Existing cloud-native microservice traffic management solutions typically rely on static configuration or simple rule-based load balancing algorithms, such as round-robin, least-connections, or weight-based random allocation. While these methods are simple to implement, they struggle to dynamically adjust based on real-time service load changes, instance performance differences, and network link status. When system load surges or some service instances experience performance degradation, it can easily lead to concentrated requests, increased response latency, or even service avalanche. Some solutions introduce monitoring metrics for dynamic weight adjustments, but these are mostly rule-driven and lack continuous learning and adaptive optimization capabilities, making it difficult to maintain optimal long-term performance in complex and ever-changing environments.
[0006] Therefore, this solution proposes a microservice routing mechanism optimization method based on SDN and reinforcement learning, taking into account the characteristics of cloud-native microservice routing mechanisms. Combining RL and SDN enables collaborative optimization at different levels. On one hand, RL can continuously learn and optimize routing strategies based on multi-dimensional operational states (such as service load and response latency), achieving adaptive traffic allocation and capability selection at the application layer. On the other hand, SDN, through the separation of the control plane and data plane, possesses a global network view and programmability, enabling dynamic adjustment of underlying forwarding paths and flow table rules, achieving fine-grained traffic scheduling at the network layer. When the two are combined, RL is responsible for microservice policy decisions and global performance optimization, while SDN is responsible for underlying network execution. This layered collaborative approach not only overcomes the limitations of traditional single-layer optimization but also enables real-time awareness, autonomous decision-making, and continuous optimization in dynamic environments, significantly improving the stability, resource utilization, and service quality of cloud-native microservice systems under complex load conditions. Summary of the Invention
[0007] The purpose of this invention is to address the problem that existing cloud-native microservice architectures rely on static rules or simple load balancing algorithms for traffic routing, lacking dynamic adaptive capabilities. This invention proposes a cloud-native microservice intelligent routing optimization method that integrates RL and SDN. Based on the characteristics of microservices, this method constructs a decision model based on multi-dimensional operational states, dynamically monitors the microservice status, and improves overall throughput, reduces service response latency, and increases resource utilization while ensuring system stability. It achieves layered collaborative optimization at both the application and network layers, thereby enhancing the intelligence and global optimization capabilities of traffic scheduling in a cloud-native microservice environment.
[0008] The technical solution of this invention is: a microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology, comprising the following steps: Step 1) In a cloud-native microservice architecture, a microservice runtime status information collection module needs to be established to collect runtime status data of each microservice instance in real time. This runtime status data includes CPU utilization, memory usage, request queue length, and service response latency.
[0009] Step 2) For multiple optimization objectives of the cloud-native microservice architecture, construct the state space, action space, and reward function in the reinforcement learning decision module. The state space is constructed from the state information collected in Step 1) to form a multi-dimensional runtime state vector. The action space includes the target service instance selection strategy, and the reward function is a comprehensive reward function based on service response time, request queue length, and CPU utilization.
[0010] Step 3) During the operation of the microservice system, it provides input to the reinforcement learning decision module in Step 2) and outputs the corresponding routing decision results, which should be updated according to time periods.
[0011] Step 4) When the microservice is running, it accesses the routing decision results provided in Step 3) and selects the corresponding downstream to make a call. After receiving the microservice call request, the SDN controller selects the shortest path with the best link quality in the microservice call chain according to the network topology and network link quality, and sends the flow table to the corresponding node.
[0012] Step 5) When the source microservice initiates a service call request, the request message enters the switching device of the node where it is located. The switching device matches the flow table rules issued in Step 4) and forwards the message to the target node according to the corresponding action field, thereby completing the cross-node service call.
[0013] This invention proposes a microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology. Compared with existing routing methods, its advantages are as follows: This invention introduces a reinforcement learning model to construct an adaptive decision-making mechanism, enabling the routing strategy to be dynamically adjusted according to the real-time operating status of the system. This overcomes the problem that traditional static algorithms such as round-robin, weighted random, or least-connections cannot continuously optimize, achieving intelligent traffic allocation and self-learning optimization capabilities. This effectively reduces service response latency and improves the overall system throughput performance. Furthermore, this invention utilizes the centralized control and global programmability of software-defined networks to achieve coordinated control of application-layer traffic scheduling and network-layer path optimization. Compared with existing solutions that only select instances at the application layer, this further optimizes the underlying forwarding path, alleviates network congestion, and improves network resource utilization. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the structure of an optimization method for microservice routing mechanisms based on SDN and combined with reinforcement learning technology.
[0015] Figure 2 This is a comparison chart of the average time delay between the method of this invention and the random algorithm. Detailed Implementation
[0016] This invention is applied to microservice systems. The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0017] like Figure 1 As shown, this embodiment includes a reinforcement learning module, a policy table, worker nodes, and a software-defined network controller. The reinforcement learning module receives node and microservice running status reports from each worker node; makes policy decisions based on the current status; outputs the target downstream instance selection result; and updates the policy table. The policy table stores the routing decision results generated by the reinforcement learning module, providing downstream instance call information for the microservices of the worker nodes. Worker nodes include microservice instances and switching devices, and are used to periodically report node and microservice status to the reinforcement learning module; select target downstream instances for service calls according to the policy table; and send requests to the local switching device. The software-defined network controller calculates the optimal path based on network topology and link status; distributes flow table rules to each switching device along the path; and implements cross-node data forwarding.
[0018] The technical solution of the present invention will now be described in detail, and the specific implementation steps are as follows: Step 1) In a cloud-native microservice architecture, a microservice runtime status information collection module needs to be established to collect runtime status data of each microservice instance in real time. This runtime status data includes CPU utilization, memory usage, request queue length, and service response latency.
[0019] In a specific implementation, the status information collection module can be achieved through a combination of microservice exposed performance metrics, container monitoring components, and open-source monitoring systems. For example, container-level resource metrics can be collected using open-source container monitoring tools (such as cAdvisor), and these metrics can be periodically crawled and stored using open-source monitoring systems (such as Prometheus). The collected runtime status data is divided into node-level status data and service-level status data based on resource granularity. Node-level status data includes the CPU utilization and memory usage of each compute node; service-level status data includes the request queue length and service response latency of each microservice instance. The collected multidimensional runtime status data is updated at preset time intervals and constructed into a state vector format.
[0020] in:
[0021]
[0022] These represent the resource status of the j-th node and the running status of the i-th microservice instance, respectively.
[0023] Step 2) For multiple optimization objectives of the cloud-native microservice architecture, construct the state space, action space, and reward function in the reinforcement learning decision module. The state space is constructed from the state information collected in Step 1) to form a multi-dimensional runtime state vector. The action space includes the target service instance selection strategy, and the reward function is a comprehensive reward function based on service response time, request queue length, and CPU utilization.
[0024] The microservice routing process is formalized as a Markov Decision Process (MDP), consisting of a quadruple ( )definition.
[0025] For state transition probabilities, in specific implementations, it is not necessary to explicitly model the state transition probabilities; instead, samples are obtained through interaction with the environment.
[0026] The state space consists of the hierarchical running state vector constructed in step 1), including node-level resource state and service-level running state, which are used to describe the running characteristics of the current microservice system.
[0027] The action space is defined as the probability vector of the set of selectable downstream microservice instances for a given microservice. Specifically, if a microservice has N selectable downstream instances, the action is:
[0028] in, ,
[0029] The probability distribution described above is used to guide the selection of microservice instances.
[0030] The reward function is used to measure the impact of the current routing decision on system performance, and is constructed as a multi-objective weighted combination form:
[0031] in, This represents the average response time of a microservice. Indicates the length of the request queue. Indicates node CPU utilization. These are the weighting coefficients.
[0032] Each indicator is normalized before calculation to eliminate the influence of different units on the reward function.
[0033] Step 3) During the operation of the microservice system, it provides input to the reinforcement learning decision module in Step 2) and outputs the corresponding routing decision results, which should be updated according to time periods.
[0034] During system operation, the status information collection module described in step 1) collects the resource status of each node and the running status of each microservice instance according to a preset time window, and converts them into standardized status vectors, which are then input into the reinforcement learning decision module described in step 2). The time window is a fixed-period update mechanism, for example, 5 seconds as a decision cycle.
[0035] In a specific implementation, the routing decision result can be serialized into a structured data file for storage, such as in JSON format. The JSON file is used to store the allocation probability vector within the current decision cycle, and its content includes: upstream microservice identifier, a list of optional downstream microservice instances, and the allocation probability corresponding to each instance. An example structure is shown below: { "service": "svc_a1", "downstream": [ {"instance": "svc_b1", "prob": 0.25}, {"instance": "svc_b2", "prob": 0.50}, {"instance": "svc_b3", "prob": 0.25} ] } The JSON file can be stored in the control node or the local directory of each worker node, and updated once in each decision cycle. To avoid inconsistencies during policy updates, an atomic replacement method can be used after writing a temporary file, thereby ensuring the integrity and consistency of the policy file.
[0036] When processing a request, the microservice reads the probability distribution information in the current JSON file and selects the target downstream instance based on the probability sampling mechanism.
[0037] Step 4) When the microservice is running, it accesses the routing decision results provided in Step 3) and selects the corresponding downstream to make a call. After receiving the microservice call request, the SDN controller selects the shortest path with the best link quality in the microservice call chain according to the network topology and network link quality, and sends the flow table to the corresponding node.
[0038] The SDN controller should maintain network topology information and link operating status in real time, including: nodes and node connection status, link bandwidth, and link packet loss rate. Link operating status indicators can be obtained through active probing or switch statistics.
[0039] After obtaining the source node and the node where the target microservice instance resides, the SDN controller constructs a set of candidate paths based on the current network topology. In specific implementations, the following strategies can be adopted: 1. Filter several shortest paths between the source node and the target node; 2. Calculate the overall link quality index within the shortest path set; 3. Select the path with the best overall link quality as the forwarding path.
[0040] The overall link quality can be defined as:
[0041] in, For link bandwidth utilization, For link packet loss rate, These are the weighting coefficients.
[0042] Once the path is determined, the SDN controller distributes flow table rules to each node along the path to achieve targeted forwarding of microservice calls. Flow table rules should include a matching field and an action field, where the matching field includes the source IP and destination IP, and the action field includes the next-hop port.
[0043] Step 5) When the source microservice initiates a service call request, the request message enters the switching device of the node where it is located. The switching device matches the flow table rules issued in Step 4) and forwards the message to the target node according to the corresponding action field, thereby completing the cross-node service call.
[0044] This experiment builds a cloud-native microservice test environment based on a virtualization platform. The node deployment and division of labor include: one control node, which deploys the RYU SDN controller and also deploys microservice runtime information collection scripts, reinforcement learning model training and decision-making scripts, responsible for state data aggregation, routing policy generation, and SDN flow table distribution; five worker nodes, serving as microservice runtime nodes, containerizing multiple instances of microservices a, b, and c, and deploying the cAdvisor container monitoring component to collect real-time node and microservice runtime status data and report it to the control node; and one simulation node, which uses the Locust tool to simulate user-generated microservice call requests. The control node's information collection script is developed in Python, the reinforcement learning script is implemented using Python + PyTorch, and the core training script uses the PPO algorithm. The five worker nodes deploy a total of 21 instances of microservices a, b, and c across nodes. Figure 2 The variables are svc_a1~svc_a8, svc_b1~svc_b6, and svc_c1~svc_c7. This experiment only changed the routing strategy; all other experimental conditions remained completely consistent. The results were compared by conducting multiple experiments and averaging the results.
[0045] like Figure 2 As shown in the figure, the horizontal axis represents the identifiers of different microservices in the cloud-native microservice architecture, and the vertical axis represents the average response latency of microservices, in milliseconds. In the figure, `Random` represents the average response latency of the traditional random routing algorithm, and `RL` represents the average response latency of the method of this invention. The comparison results of the average response latency of microservices using the method of this invention and the traditional random routing algorithm show that, under the same experimental environment and load conditions, the routing optimization method of this invention, based on SDN and combined with reinforcement learning technology, has a lower average response latency on most microservice instances than the traditional random routing algorithm, and the average response latency on the remaining microservice instances is on par with the traditional random routing algorithm. This effectively reduces service response latency and verifies the technical effect of this invention.
[0046] In summary, this invention, through the aforementioned steps, establishes a method for optimizing microservice routing mechanisms. This proposed method can optimize microservice routing, reduce communication latency between microservices, improve the management efficiency and system stability of cloud-native microservice architectures, reduce service management complexity, and enhance the scalability and operational efficiency of enterprise software.
Claims
1. A microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology, characterized in that: Includes the following steps: Step 1) In a cloud-native microservice architecture, establish a microservice runtime status information collection module to collect runtime status data of each microservice instance in real time; The operational status data includes CPU utilization, memory usage, request queue length, and service response latency; Step 2) For multiple optimization goals of cloud-native microservice architecture, construct the state space, action space and reward function in the reinforcement learning decision module; wherein, the state information collected in Step 1) constructs a multi-dimensional running state vector of the state space, the action space includes the target service instance selection strategy, and the reward function is a comprehensive reward function based on service response time, request queue length and CPU utilization. Step 3) During the operation of the microservice system, it provides input to the reinforcement learning decision module in Step 2) and outputs the corresponding routing decision results. The decision results should be updated according to time periods. Step 4) When the microservice is running, it accesses the routing decision results provided in Step 3) and selects the corresponding downstream to make a call. After receiving the microservice call request, the SDN controller selects the shortest path with the best link quality in the microservice call chain according to the network topology and network link quality, and sends the flow table to the corresponding node. Step 5) When the source microservice initiates a service call request, the request message enters the switching device of the node where it is located. The switching device matches the flow table rules issued in Step 4) and forwards the message to the target node according to the corresponding action field, thereby completing the cross-node service call.
2. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 1, characterized in that: In step 1), the status information collection module is implemented by combining microservice exposure performance metrics, container monitoring components, and open-source monitoring systems. Container-level resource metrics are collected based on open-source container monitoring tools, and the metrics are periodically captured and stored based on the open-source monitoring system. The collected runtime status data is divided into node-level status data and service-level status data according to resource granularity. Node-level status data includes the CPU utilization and memory usage of each computing node; service-level status data includes the request queue length and service response latency of each microservice instance.
3. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 2, characterized in that: The collected multidimensional operational status data is updated at preset time intervals and constructed into a state vector form: ； in: ； ； in, It is a state-space vector. For the node's running status, This is a vector representing the running state of a microservice. Let be the CPU utilization of the j-th node. Let j be the memory usage of the j-th node. For the request queue of the i-th microservice, Let represent the response latency of the i-th microservice, and j and i represent the resource status of the j-th node and the running status of the i-th microservice instance, respectively.
4. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 1, characterized in that: In step 2), the microservice routing process is formalized as a Markov Decision Process (MDP), consisting of a quadruple ( )definition; S represents the state space, A represents the action space, and R represents the reward function. The state transition probability does not need to be explicitly modeled; samples are obtained through interaction with the environment.
5. A microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology as described in claim 1 or 4, characterized in that: The state space consists of the hierarchical running state vectors constructed in step 1), including node-level resource states and service-level running states, which are used to describe the running characteristics of the current microservice system; The action space is defined as the probability vector of the set of selectable downstream microservice instances for a given microservice; specifically, if a microservice has N selectable downstream instances, then the action is: ； in, , ; The probability distribution is used to guide the selection of microservice instances; The reward function measures the impact of the current routing decision on system performance and is constructed as a multi-objective weighted combination: ； in, This represents the average response time of a microservice. Indicates the length of the request queue. Indicates node CPU utilization. These are the weighting coefficients.
6. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 1, characterized in that: In step 3), during system operation, the status information collection module of step 1) collects the resource status of each node and the running status of each microservice instance according to a preset time window, and converts them into standardized status vectors to be input into the reinforcement learning decision module of step 2); the time window is a fixed time period update mechanism.
7. A microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology as described in claim 1 or 6, characterized in that: The routing decision results can be serialized into a structured data file for storage, and represented in JSON format; The JSON file is used to store the allocation probability vector within the current decision-making cycle, including: the upstream microservice identifier, a list of optional downstream microservice instances, and the allocation probability corresponding to each instance. An example structure is shown below: { "service": "svc_a1", "downstream": [ {"instance": "svc_b1", "prob": 0.25}, {"instance": "svc_b2", "prob": 0.50}, {"instance": "svc_b3", "prob": 0.25} ] }； Where service is the identifier of the corresponding upstream microservice, downstream is the list of downstream microservice instances, instance is the identifier of the downstream microservice, and prob represents the probability of selecting the corresponding downstream instance.
8. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 7, characterized in that: The JSON file is stored in the local directory of the control node or each worker node and is updated once in each decision cycle. To avoid inconsistencies during policy updates, an atomic replacement method is used after writing a temporary file to complete the update, ensuring the integrity and consistency of the policy file.
9. The microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 1, characterized in that: In step 4), the SDN controller should maintain network topology information and link operation status in real time, including: nodes and node connection status, link bandwidth, and link packet loss rate; link operation status indicators are obtained through active probing or switch statistics.
10. A microservice routing mechanism optimization method based on SDN and combined with reinforcement learning technology according to claim 1 or 9, characterized in that: After obtaining the source node and the node where the target microservice instance resides, the SDN controller adopts the following strategy: ① Select several shortest paths between the source node and the target node; ② Calculate the comprehensive link quality index within the shortest path set; ③ Select the path with the best overall link quality as the forwarding path; The overall link quality is defined as follows: ； in, For link bandwidth utilization, For link packet loss rate, These are the weighting coefficients; Once the path is determined, the SDN controller distributes flow table rules to each node along the path to enable targeted forwarding of microservice calls. The flow table rules include matching fields and action fields. The matching fields include the source IP and destination IP, and the action fields include the next-hop port.