Load balancing method, device and equipment for SRv6 network VPN (Virtual Private Network) service

Abstract

The invention provides a load balancing method, device and equipment for a VPN (Virtual Private Network) service of an SRv6 network, in a load balancing process, the high availability of a user side PE (Provider Edge) service can be enhanced by incorporating service dispersity, the service quality of a single service can be enhanced by incorporating service satisfaction, and node load balancing with a better effect is ensured by incorporating node load factors, so that the load balancing efficiency is improved. Link load balancing with a good effect is ensured by incorporating a link load factor, and a deep deterministic strategy gradient algorithm model is ensured to output a flow distribution proportion with better load balancing performance through multi-dimensional reward calculation. SRv6 backbone network state perception is realized through network topology and special path information, VPN flow deployment is carried out in a target path group according to a flow distribution proportion, and node load balancing and link load balancing are realized at the same time, so that the problem that node load balancing and link load balancing are separated and contradictory is solved.

Description

Technical Field

[0001] This application relates to the field of communication technologies, and in particular, to a load balancing method, apparatus, and device for SRv6 network VPN services. Background Art

[0002] With the continuous expansion of the scale of Internet service systems, the number of users has been increasing rapidly, business requirements have become gradually complex, and the service architecture of network systems has also been evolving continuously. Cluster deployment and distributed deployment have emerged as the times require. The emergence of cluster deployment and distributed deployment has brought a new challenge: how to select appropriate servers to process users' service requests, that is, how to achieve load balancing. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a load balancing method, apparatus, and device for SRv6 network VPN services, and achieve load balancing of VPN services in the SRv6 network by incorporating factors such as service satisfaction, service dispersion, load of service-side PE nodes, and VPN traffic distribution in the network into the reward scope.

[0004] Based on the above purpose, this application provides a load balancing method for SRv6 network VPN services, which is characterized by including:

[0005] Generate the network topology of the backbone network and the dedicated path information corresponding to the VPN service according to the collected backbone network information;

[0006] Determine the historical traffic allocation ratio and historical load balancing performance corresponding to the historical VPN requests, and determine the traffic matrix according to the historical traffic allocation scheme of the backbone network and the historical traffic allocation ratio;

[0007] Determine the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side according to the traffic matrix;

[0008] Determine the current load balancing performance of the current traffic allocation scheme according to the link load factor, the service satisfaction, the service dispersion, and the node load factor;

[0009] Online train the deep deterministic policy gradient algorithm model according to the current load balancing performance and the historical load balancing performance;

[0010] In response to receiving a VPN request, determine the target VPN service corresponding to the VPN request, and generate a request traffic matrix according to the current traffic allocation scheme and the VPN request;

[0011] Determine a target path group corresponding to the target VPN service according to the network topology and the proprietary path information, input the request traffic matrix into the online-trained deep deterministic policy gradient algorithm model, and determine the backbone network egress and the traffic allocation ratio of each path within the target path group in the ingress carrier edge router of the backbone network according to the model output.

[0012] Based on the same inventive concept, the present disclosure also provides a load balancing device for SRv6 network VPN services, including:

[0013] An information collection module, configured to: generate a network topology of the backbone network and proprietary path information corresponding to the VPN service according to the collected backbone network information;

[0014] A matrix generation module, configured to: determine the historical traffic allocation ratio and historical load balancing performance corresponding to the historical VPN requests, and determine a traffic matrix according to the historical traffic allocation scheme of the backbone network and the historical traffic allocation ratio;

[0015] A reward calculation module, configured to: determine the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side according to the traffic matrix;

[0016] A load calculation module, configured to: determine the current load balancing performance of the current traffic allocation scheme determined according to the link load factor, the service satisfaction, the service dispersion, and the node load factor;

[0017] An online training module, configured to: online train a deep deterministic policy gradient algorithm model according to the current load balancing performance and the historical load balancing performance;

[0018] A request response module, configured to: in response to receiving a VPN request, determine a target VPN service corresponding to the VPN request, and generate a request traffic matrix according to the current traffic allocation scheme and the VPN request;

[0019] A load balancing module, configured to: determine a target path group corresponding to the target VPN service according to the network topology and the proprietary path information, input the request traffic matrix into the online-trained deep deterministic policy gradient algorithm model, and determine the backbone network egress and the traffic allocation ratio of each path within the target path group in the ingress carrier edge router of the backbone network according to the model output.

[0020] Based on the same inventive concept, the present disclosure also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable by the processor. When the processor executes the computer program, the above-described method is implemented.

[0021] As can be seen from the above, the load balancing method, apparatus, and device for SRv6 network VPN services provided by the present application can determine the dedicated paths capable of executing VPN tasks in the backbone network and the corresponding dedicated path information by generating the network topology of the backbone network and the dedicated path information corresponding to the VPN service, providing data support for the subsequent process of the traffic allocation scheme. After generating the historical traffic allocation ratio according to the historical VPN requests, the traffic matrix is determined based on the historical traffic allocation scheme and the historical traffic allocation ratio of the backbone network, obtaining the input items for reward calculation and load balancing. Then, through the reward calculation in four dimensions, factors such as service satisfaction, service dispersion, load of the service-side PE nodes, and distribution of VPN traffic in the network are included in the reward scope. Among them, including service dispersion can enhance the high availability of the user-side PE services, including service satisfaction can enhance the service quality of a single service, including the node load factor can ensure good node load balancing, and including the link load factor can ensure good link load balancing. The load balancing ability of the deep deterministic policy gradient algorithm model is determined through multi-dimensional reward calculation, and the subsequent VPN requests are ensured to have better load balancing performance through online training of the deep deterministic policy gradient algorithm model. When a VPN request is received, the trained deep deterministic policy gradient algorithm model is used to determine the traffic allocation ratio with better load balancing performance, and the SRv6 backbone network status awareness is achieved through the network topology and the dedicated path information, and the VPN traffic is deployed within the target path group according to the traffic allocation ratio, while achieving node load balancing and link load balancing, so as to solve the problem of fragmentation and contradiction between node load balancing and link load balancing, and achieve the load balancing of VPN services in the SRv6 network. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present application or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0023] Figure 1 It is a flowchart of the load balancing method for SRv6 network VPN services in the embodiments of the present application;

[0024] Figure 2Schematic diagram of the load balancing system for the SRv6 network VPN service in the embodiment of the present application;

[0025] Figure 3 Schematic diagram of the overall network structure of the backbone network in the embodiment of the present application;

[0026] Figure 4 Schematic diagram of the learning architecture of the DDPG network in the embodiment of the present application;

[0027] Figure 5 Flowchart for determining the link load factor in the embodiment of the present application;

[0028] Figure 6 Flowchart for determining the service satisfaction degree in the embodiment of the present application;

[0029] Figure 7 Flowchart for determining the service dispersion degree in the embodiment of the present application;

[0030] Figure 8 Flowchart for determining the node load factor in the embodiment of the present application;

[0031] Figure 9 Flowchart for determining the load balancing performance in the embodiment of the present application;

[0032] Figure 10 Flowchart for determining the target path group in the embodiment of the present application;

[0033] Figure 11 Schematic diagram of the structure of the load balancing device for the SRv6 network VPN service in the embodiment of the present application;

[0034] Figure 12 Schematic diagram of the structure of the electronic device in the embodiment of the present application. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of the present application clearer and more understandable, the following further details the present application with reference to specific embodiments and the accompanying drawings.

[0036] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should have the ordinary meanings understood by those with ordinary skills in the field to which the present application belongs. The "first", "second" and similar terms used in the embodiments of the present application do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "including" or "comprising" mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connected" or "linked" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right", etc. are only used to indicate relative position relationships. When the absolute position of the object being described changes, the relative position relationship may also change accordingly.

[0037] In this document, it should be understood that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0038] Based on the above description of the background technology, the following situations also exist in the related technology:

[0039] Segment Routing over IPv6 (SRv6) based on the IPv6 forwarding plane is a new network routing protocol based on IPv6, aiming to simplify network management and improve network efficiency. SRv6 guides the forwarding path of data packets in the network by adding a series of instructions (called segment lists) to the data packet header. These instructions can be operations of local nodes or addresses specifying the next hop. SRv6 uses the extension header of IPv6 to achieve this, enabling network devices to flexibly control the transmission path of data packets according to predefined policies. Compared with traditional MPLS (Multi-Protocol Label Switching), SRv6 has better scalability and flexibility, can better support emerging network application scenarios, and also supports multiple network functions such as traffic engineering, fast rerouting, and multipath transmission, thus improving the reliability and performance of the network.

[0040] Load balancing is a network technology used to distribute the workload among multiple nodes or links to optimize resource utilization, maximize throughput, reduce response time, and avoid overload. At the node level, a load balancer is typically located between the client and the server and distributes requests to different backend servers through algorithms (such as round-robin, least connections, hashing, etc.) to ensure an even distribution of the load across each server. At the link level, load balancing distributes network traffic among multiple physical links to achieve efficient utilization of bandwidth and fault recovery. For example, through multi-link aggregation technology, the traffic path can be dynamically adjusted to avoid single points of failure and congestion, ensuring the stability and reliability of the network.

[0041] On the data forwarding plane, there are mainly two types of load balancing technologies: one is load balancing for server nodes, aiming to disperse the resource pressure on a single device and improve the overall performance of the service; the other is load balancing for network links, which optimizes the bandwidth allocation of the service traffic in the network and adjusts the traffic load on the path to solve the data congestion problem in the network.

[0042] A Virtual Private Network (VPN) is a virtual private communication network established by relying on Internet Service Providers (ISPs) and Network Service Providers (NSPs) in the public network, which can meet the enterprise's requirements for network flexibility, security, economy, scalability, etc.

[0043] VPN has the following two basic characteristics:

[0044] Private: For VPN users, using a VPN is no different from using a traditional private network. There is resource independence between the VPN and the underlying bearer network, that is, the VPN resources are not used by users outside the VPN in the network; and the VPN can provide sufficient security guarantees to ensure that the internal information of the VPN is not invaded from the outside.

[0045] Virtual: The communication within the VPN users is carried out through the public network, and this public network can also be used by other non-VPN users at the same time. What the VPN users obtain is only a private network in a logical sense. Among them, the public network is called the VPN Backbone.

[0046] By leveraging the dedicated and virtual characteristics of VPNs, existing IP networks can be decomposed into logically isolated networks. These logically isolated networks have a wide range of applications: they can be used to solve the interconnection within enterprises, the interconnection between the same or different departments; they can also be used to provide new services, such as opening up a dedicated VPN for IP telephony services to address issues such as insufficient IP network addresses, QoS guarantee, and the development of new value-added services.

[0047] From the user's perspective, VPNs have the following advantages compared to traditional private networks:

[0048] Security: Establish reliable connections between remote users, overseas offices, partners, suppliers, and the company headquarters to ensure the security of data transmission. This is particularly important for the integration of e-commerce or financial networks with communication networks.

[0049] Inexpensive: By using public networks for information communication, enterprises can connect remote offices, business travelers, and business partners at a lower cost.

[0050] Support for mobile services: Support mobile access for overseas VPN users at any time and anywhere, and can meet the growing demand for mobile services.

[0051] Service quality guarantee: Building a VPN with service quality guarantee (such as MPLS VPN) can provide different levels of service quality guarantee for VPN users.

[0052] From the operator's perspective, VPNs have the following advantages:

[0053] Operable: Improve the utilization rate of network resources and contribute to increasing the revenue of ISPs.

[0054] Flexible: VPN users can be added or deleted through software configuration without changing hardware facilities. It has great flexibility in applications.

[0055] Multi-service: Based on providing VPN interconnection, SPs can undertake multi-service operations such as network outsourcing, business outsourcing, and customized professional services.

[0056] VPNs have won the favor of more and more enterprises with their unique advantages, enabling enterprises to pay less attention to network operation and maintenance and thus focus more on achieving their business goals.

[0057] However, the emergence of cluster deployment and distributed deployment has brought new challenges to the establishment of VPNs: how to select appropriate servers through the private network address across the backbone network to handle users' service requests, how to plan the traffic distribution between users and servers, that is, how to achieve node load balancing and link load balancing for VPN services. In traditional load balancing methods, the contradiction between node load balancing and link load balancing has not been resolved, that is, node load balancing and link load balancing are relatively disjointed and lack flexibility.

[0058] Load balancing solutions in related technologies usually handle node and link load balancing separately, and it is difficult to combine the two in the scenario of VPN service load balancing, resulting in the following disadvantages: First, only focusing on node load may cause some links to be overloaded while other links are idle, and vice versa. This unbalanced resource utilization will reduce the overall network efficiency; Second, if only node load is considered, when a link fails, the traffic path may not be adjusted in time, resulting in partial service interruption; Similarly, only focusing on link load cannot effectively handle node failures. Third, separately considering node or link load balancing may form a performance bottleneck in high-concurrency scenarios, affecting the user experience. Finally, lacking a global view of the entire network, it is difficult to achieve optimal traffic scheduling and resource allocation, affecting the overall network performance and stability. In summary, the limitations of load balancing solutions in related technologies limit their application effects in complex network environments.

[0059] The load balancing method, device, and equipment for SRv6 network VPN services provided by this application can determine the dedicated paths capable of executing VPN tasks in the backbone network and the corresponding dedicated path information by generating the network topology of the backbone network and the dedicated path information corresponding to the VPN service, providing data support for the subsequent process of the traffic allocation scheme. After generating the historical traffic allocation ratio according to the historical VPN requests, a traffic matrix is determined based on the historical traffic allocation scheme and the historical traffic allocation ratio of the backbone network, obtaining the input items for reward calculation and load balancing. Then, by calculating the rewards in four dimensions, factors such as service satisfaction, service dispersion, load of the service-side PE nodes, and distribution of VPN traffic in the network are included in the scope of rewards. Among them, including service dispersion can enhance the high availability of the user-side PE services, including service satisfaction can enhance the service quality of a single service, including the node load factor can ensure good node load balancing, and including the link load factor can ensure good link load balancing. The load balancing ability of the deep deterministic policy gradient algorithm model is determined through multi-dimensional reward calculation, and the subsequent VPN requests are ensured to have better load balancing performance by online training the deep deterministic policy gradient algorithm model. When a VPN request is received, the trained deep deterministic policy gradient algorithm model is used to determine the traffic allocation ratio with better load balancing performance, and the SRv6 backbone network status awareness is realized through the network topology and dedicated path information, and the VPN traffic is deployed within the target path group according to the traffic allocation ratio, while achieving node load balancing and link load balancing at the same time, so as to solve the problem that node load balancing and link load balancing are split and contradictory, and realize the load balancing of VPN services in the SRv6 network.

[0060] The following will detail the load balancing method for SRv6 network VPN services provided by the embodiments of this application with reference to the accompanying drawings.

[0061] In some embodiments, as Figure 1 shown, the load balancing method for SRv6 network VPN services includes:

[0062] Step 101: Generate the network topology of the backbone network and the dedicated path information corresponding to the VPN service according to the collected backbone network information.

[0063] Specifically, when implemented, the load balancing system for SRv6 network VPN services is as Figure 2 shown, consisting of the backbone network of the load balancing system structure and the controller.

[0064] Among them, the overall network structure of the backbone network is as Figure 3As shown in the figure, the overall network structure is divided into a backbone intranet and a backbone extranet. User hosts (PCs) and servers (S) are located outside the backbone network. The routing devices within the backbone network include provider backbone routers (Provider, P) that are not directly connected to the outside world, and provider edge routers (Provider Edge, PE) that are directly connected to external devices. According to the traffic direction, they are divided into ingress PEs and egress PEs. The devices directly connected to the PE devices outside the backbone network are customer edge routers (Customer Edge, CE). CE devices can be routing devices, user hosts, or servers. As Figure 2 shown in orange in the figure, for the VPN tunnel, when communicating across the backbone network between users and servers, the VPN tunnel needs to be used. The SRv6 protocol can flexibly specify the path combination of the tunnel and the traffic distribution ratio between each path. Therefore, in the SRv6 backbone network, by specifying a suitable egress PE for the VPN tunnel and reasonably distributing the traffic within the VPN tunnel, the problems of server node load balancing and network link load balancing can be effectively solved.

[0065] Among them, the topology of the backbone network is represented by G=(V, E), where V is the set of nodes within the backbone network, and V = [v 1 , v 2 , …, v N , N is the number of nodes, E is the set of links within the backbone network, and E = [e 1 , e 2 , …, e M , M is the number of links, and X represents the number of VPN services in the backbone network.

[0066] Then, the prerequisite for the controller to perform traffic distribution is to understand the overall structure of the backbone network. It is necessary to obtain the backbone network information of the backbone network and generate the network topology of the backbone network and the dedicated path information corresponding to the VPN service within the controller to achieve the state awareness of the backbone network. However, not all links in the backbone network can forward VPN service data. Therefore, it is necessary to further perceive the dedicated path information corresponding to the VPN service in the backbone network to determine the paths that can forward VPN service data and the performance information of the corresponding paths.

[0067] Among them, as Figure 2As shown in the figure, the controller includes an information collection subsystem and an SRv6 VPN service load balancing subsystem (also known as the service load balancing subsystem). The information collection subsystem includes two services: topology generation and path generation. The topology generation service collects and generates backbone network information such as node information, interface information, link information, and SRv6 network information within the backbone network through the traffic matrix or SNMP protocol messages to generate a network topology corresponding to the backbone network, and delivers the backbone network information and the network topology to the path generation service. The path generation service generates dedicated path information corresponding to the links used for forwarding VPN services based on node, interface, link, SRv6 network information, etc. The information collection subsystem also delivers data such as the collected backbone network information, network topology, and dedicated path information to the upper-layer service load balancing subsystem for offline / online training of the Deep Deterministic Policy Gradient (DDPG) algorithm and business processing and query of the VPN service load balancing subsystem.

[0068] Among them, the DDPG algorithm is a deep reinforcement learning algorithm used to solve problems in continuous action spaces. It combines deterministic policy gradients and deep learning techniques to achieve effective learning in high-dimensional continuous action spaces.

[0069] As Figure 2 shown in the figure, the service load balancing subsystem includes an SRv6 VPN service interaction service, an SRv6 VPN service processing service, a traffic allocation service, and an SRv6 VPN service query service. Among them, the SRv6 VPN service interaction service is responsible for request distribution; the SRv6 VPN service processing service is responsible for adding SRv6 VPN services. Its main work content is to receive VPN requests, deliver the requests to the traffic allocation service, generate new SRv6 VPN policies according to the traffic allocation results, send them to the PE devices within the backbone network, and persist the corresponding SRv6 VPN service data; the traffic allocation service is responsible for generating a traffic matrix, inputting the traffic matrix into the DDPG algorithm, and generating a traffic allocation result according to the returned actions of the DDPG algorithm; the SRv6 VPN service query service is responsible for displaying SRv6 VPN service data within the backbone network.

[0070] The controller collects backbone network information through the information collection subsystem and generates the network topology of the backbone network and dedicated path information corresponding to VPN services based on the collected backbone network information to achieve state awareness of the backbone network, so as to ensure that the service load balancing subsystem can return accurate traffic allocation ratios.

[0071] Step 102: Determine the historical traffic allocation ratio and historical load balancing performance corresponding to the historical VPN request, and determine the traffic matrix according to the historical traffic allocation scheme and historical traffic allocation ratio of the backbone network.

[0072] During specific implementation, after determining the traffic allocation ratio according to the VPN request, the corresponding VPN request will be marked as a historical VPN request, which can be simply understood as "the previous VPN request (relative to the new VPN request)". The historical traffic allocation ratio is the traffic allocation ratio corresponding to the action made by the Deep Deterministic Policy Gradient (DDPG) model according to the VPN request. The historical load balancing performance includes the link load balancing and node load balancing of the backbone network after traffic allocation according to the historical traffic allocation ratio.

[0073] After allocating the historical traffic allocation ratio, it is necessary to determine whether this historical traffic allocation ratio has better load balancing performance. At this time, it is necessary to determine the traffic matrix according to the historical traffic allocation scheme and historical traffic allocation ratio of the backbone network, and use the traffic matrix as the output to verify whether the deep deterministic policy gradient algorithm model has a traffic allocation scheme with better load balancing performance.

[0074] Among them, the traffic matrix represents the network traffic demand from the source node to the destination node in the network, that is, the size of the traffic between the entrance and exit, and is then represented by a matrix. It plays the role of a key input parameter. The traffic matrix reflects the composition of all traffic in the entire network, describes the traffic composition in each link, and all network traffic information should be measured. When the network demand needs to change, corresponding changes can be made based on the information in the traffic matrix. In the backbone network, the SNMP protocol can be used to measure the link load of routers in the network, and the routing matrix can be obtained according to the actual network configuration information.

[0075] Exemplarily, the traffic matrix contains many elements, and each element represents a bandwidth request. For example, if the bandwidth request between node PE1 and node PE2 is k, it is simplified to the element [PE1, PE2, k]. If [PE1, P1, PE2, k] is the current traffic allocation scheme of the backbone network, and the received VPN request is sent by node PE1, and the corresponding exit node is PE2, and the required request bandwidth is j, then the VPN request can be simplified to the element [PE1, PE2, j]. At this time, the traffic matrix can be expressed as [[PE1, P1, PE2, k], [PE1, PE2, j]]. Among them, the elements in the traffic matrix can contain more information, such as all node information, VPN task information, link information, traffic allocation ratio, etc. in the current traffic allocation scheme, which will not be listed one by one here.

[0076] Step 103: Determine the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion degree of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side according to the traffic matrix.

[0077] In specific implementation, taking the traffic matrix as the input, use the DDPG algorithm to calculate the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion degree of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side in sequence, so as to achieve load balancing for VPN services on the backbone network. It is necessary to train the DDPG network in advance.

[0078] The learning architecture of the DDPG network is as Figure 4 shown. Regard the SRv6 VPN service load balancing problem as a Markov decision process. Its state includes characteristics such as the network topology structure, the load information of the egress PE, and the SRv6 VPN service requirements. These characteristic information can be provided through the northbound interface of the controller. The action is to specify the backbone network egress for the VPN service and the traffic allocation ratio within the path group.

[0079] Use the historically statistical traffic matrix for iterative training. At each time slot, select an action according to the Actor network, that is, specify the egress PE of the SRv6 VPN service and the traffic allocation ratio within the path group, and apply it to the environment. Observe the reward and the next state returned by the environment, and use this information to update the parameters of the Actor and Critic networks. The parameters of the Actor network can be updated using the policy gradient algorithm to increase the probability of selecting high-reward actions. And update the parameters of the Critic network according to the use of temporal difference, and update the value estimation of the current state by estimating the value of the next state.

[0080] Adopt the Ornstein-Uhlenbeck process to add noise to the action. The Ornstein-Uhlenbeck process can generate noise with time correlation. This characteristic helps to simulate the inertia effect in the physical system, making the exploration process more natural and suitable for control tasks.

[0081] Among them, the DDPG algorithm for SRv6 VPN service load balancing is as follows:

[0082] 1. Set the training round as Episode, the maximum number of training rounds as i, the maximum time length as T, and the reward discount factor as γ.

[0083] 2. Initialize the neural network parameters: Initialize θ μ and θ Q .

[0084] 3. Initialize the parameters of the target neural network: Initialize the local network parameters corresponding to θ μ’ and θ Q’ to be equal.

[0085] 4. Initialize the experience pool to store the experiences (current state, action, reward, next state, etc.) generated by the interaction between the agent and the environment.

[0086] 5. For Episode = 1 to i do.

[0087] 6. Obtain the initial state of the agent from the environment.

[0088] 7. For t = 1 to T do.

[0089] 8. Use the Actor neural network to generate an action a t based on the current SRv6 backbone network state s t (the egress PE and the traffic splitting ratio within the path group).

[0090] 9. Add a certain amount of Ornstein-Uhlenbeck (OU) noise to the action according to the current state and action to enhance the exploration of the agent.

[0091] 10. Input a t into the reinforcement learning environment to obtain the reward r t , and calculate s t+1 .

[0092] 11. Store the obtained experience (st, at, rt, st+1) in the experience pool.

[0093] 12. Randomly sample an experience with a quantity of N.

[0094] 13. Calculate the target Q value of the Critic neural network: y t = r t + γQ(s t+1 , a t+1 , θ Q’ ).

[0095] 14. Calculate the loss function of the Critic neural network:

[0096] 15. Update θ Q to minimize its loss function.

[0097] 16. Calculate the gradient of the Actor network using θ Q :

[0098] 17. Update θ μ to maximize its gradient.

[0099] 18. Update θ μ’ and θ q’ .

[0100] 19. End for.

[0101] 20. End for.

[0102] Incorporating service dispersion can enhance the high availability of user-side PE services. Incorporating service satisfaction can enhance the service quality of individual services. Incorporating the node load factor can ensure good node load balancing, and incorporating the link load factor can ensure good link load balancing. Multi-dimensional reward calculation is used to achieve load balancing for VPN services in the SRv6 network.

[0103] Step 104: Determine the load balancing performance of the current traffic allocation scheme based on the link load factor, service satisfaction, service dispersion, and node load factor.

[0104] Specifically, when implemented, the calculation formula for the load balancing performance Performance is:

[0105]

[0106] where Performance represents the load balancing performance, and Performance ∈ [0, 1]. The larger Performance is, the better the load balancing effect. α represents the link load factor, β represents the node load factor, represents the average satisfaction, X represents the total number of services, S represents the service satisfaction of the backbone network, represents the average dispersion, di represents the service dispersion on the user side of the backbone network, and N1 represents the number of user routes. By calculating the load balancing performance of different available traffic allocation ratios, the traffic allocation scheme with the best balancing effect is selected. Since the larger Performance is, the better the load balancing effect, when offline training the deep deterministic policy gradient algorithm model, the traffic allocation ratio corresponding to the maximum load balancing performance is determined as the model output, that is, the model outputs the traffic allocation ratio corresponding to the best traffic allocation scheme to achieve the best load balancing control of the backbone network.

[0107] Step 105: Online train the deep deterministic policy gradient algorithm model based on the current load balancing performance and historical load balancing performance.

[0108] In some embodiments, step 105 includes:

[0109] Step 1051: In response to the historical load balancing performance being better than the current load balancing performance, and the current load balancing performance being lower than the preset load balancing performance threshold, online train the deep deterministic policy gradient algorithm model according to the training data until the training end condition is met, and redeploy the new deep deterministic policy gradient algorithm model;

[0110] Step 1052: In response to the historical load balancing performance being better than the current load balancing performance, and the current load balancing performance being higher than the preset load balancing performance threshold, store the current load balancing performance, historical traffic allocation ratio, historical load balancing performance, link load factor, service satisfaction, service dispersion, and node load factor into the experience pool;

[0111] Step 1053: In response to the current load balancing performance being better than the historical load balancing performance, store the current load balancing performance, historical traffic allocation ratio, historical load balancing performance, link load factor, service satisfaction, service dispersion, and node load factor into the experience pool

[0112] In specific implementation, if the DDPG algorithm model has a good effect, it will output a traffic allocation ratio with a higher load performance. If the current load balancing performance is better than the historical load balancing performance, it means that the DDPG algorithm model has a high performance and there is no need to continue online training the DDPG algorithm model.

[0113] If the historical load balancing performance is better than the current load balancing performance, it means that the load balancing effect deteriorates after adding VPN requests. At this time, continue to compare the current load balancing performance with the preset load balancing performance threshold. If the current load balancing performance is higher than the load balancing performance threshold, it means that the entire backbone network still has good node load balancing performance and link load balancing performance. Although the load balancing performance decreases due to the increase in VPN requests, it still has a high level and there is no need to continue online training the DDPG algorithm model.

[0114] If the historical load balancing performance is better than the current load balancing performance, it means that the load balancing effect deteriorates after adding VPN requests. If the current load balancing performance is lower than the load balancing performance threshold, it means that the DDPG algorithm model has a poor performance and needs to improve the performance of the DDPG algorithm model through online training.

[0115] Step 106: In response to receiving a VPN request, determine the target VPN service corresponding to the VPN request, and generate a request traffic matrix according to the current traffic allocation scheme and the VPN request.

[0116] In specific implementation, after receiving a VPN request, it is first necessary to determine the target VPN service corresponding to the VPN request to determine the requirements of the target VPN service, such as broadband requirements, link requirements, etc. Then, a request traffic matrix corresponding to the VPN request is generated based on the current traffic allocation scheme of the backbone network and the VPN request. Among them, the traffic matrix is used to indirectly measure the network. The traffic matrix is used to represent all traffic status information between different nodes in the network, can completely record the state characteristics of the entire network, represents the traffic volume between all ingress and egress PE pairs in the entire network, and the routing information in the entire network can also clearly reflect the traffic volume of each link in the entire network. By constructing the request traffic matrix, the input for the DDPG algorithm model after receiving the VPN request is determined.

[0117] Step 107: Determine the target path group corresponding to the target VPN service according to the network topology and proprietary path information, input the request traffic matrix into the online-trained deep deterministic policy gradient algorithm model, and determine the traffic allocation ratio of the backbone network egress and each path in the target path group in the ingress carrier edge router of the backbone network according to the model output.

[0118] In specific implementation, the path group is a set of the shortest links for transmitting VPN service data, that is, a set of the shortest multiple links between the ingress PE and the egress PE. After determining the optimal traffic allocation scheme, it is necessary to determine the target path group corresponding to the target VPN service according to the network topology and proprietary path information, because the target VPN service only affects the traffic allocation in the target path group in the current traffic allocation scheme, and the traffic allocation ratio is also the traffic allocation scheme between different links in the target path group. Since both the network topology and the proprietary path information are the controller's perception information of the backbone network, through this perception information, the shortest multiple links for forwarding the target VPN service can be determined, and the set of these links is determined as the target path group.

[0119] After the target path group is determined and the data transmission link is determined, it is necessary to continue to allocate traffic to each link within the path group to achieve link load balancing and node load balancing. Then, the request traffic matrix needs to be input into the online-trained deep deterministic policy gradient algorithm model, and based on the model output, the traffic allocation ratios for the backbone network exit and each path within the target path group are determined in the ingress carrier edge router of the backbone network. This traffic allocation ratio is the optimal traffic allocation scheme determined by the DDPG algorithm model at this time. Determining the traffic allocation ratio in the ingress carrier edge router can avoid switching the ratio during the transmission process, ensuring that the traffic is allocated with the optimal allocation scheme from the beginning, so as to ensure that no secondary traffic allocation is required for different VPN tasks. Since the optimal traffic allocation scheme is obtained by balancing multi-dimensional rewards, among which, incorporating service dispersion can enhance the high availability of user-side PE services, incorporating service satisfaction can enhance the service quality of a single service, incorporating the node load factor can ensure good node load balancing, and incorporating the link load factor can ensure good link load balancing. The load balancing ability of the deep deterministic policy gradient algorithm model is determined through multi-dimensional reward calculations, and the online training of the deep deterministic policy gradient algorithm model is used to ensure better load balancing performance for subsequent VPN requests. When a VPN request is received, the trained deep deterministic policy gradient algorithm model is used to determine the traffic allocation ratio with better load balancing performance, and SRv6 backbone network status awareness is achieved through network topology and proprietary path information, and VPN traffic is deployed within the target path group according to the traffic allocation ratio, simultaneously achieving node load balancing and link load balancing, so as to solve the problem of the fragmentation and contradiction between node load balancing and link load balancing and realize the load balancing of VPN services in the SRv6 network..

[0120] In summary, the load balancing method for SRv6 network VPN services provided by this application can determine the dedicated paths in the backbone network that can execute VPN tasks and the corresponding dedicated path information by generating the network topology of the backbone network and the dedicated path information corresponding to the VPN service, providing data support for the subsequent process of the traffic allocation scheme. After generating the historical traffic allocation ratio according to the historical VPN requests, the traffic matrix is determined based on the historical traffic allocation scheme and the historical traffic allocation ratio of the backbone network, obtaining the input items for reward calculation and load balancing. Then, by calculating rewards in four dimensions, factors such as service satisfaction, service dispersion, load of the service-side PE nodes, and distribution of VPN traffic within the network are incorporated into the scope of rewards. Among them, incorporating service dispersion can enhance the high availability of the user-side PE services, incorporating service satisfaction can enhance the service quality of a single service, incorporating the node load factor can ensure good node load balancing, and incorporating the link load factor can ensure good link load balancing. The load balancing ability of the deep deterministic policy gradient algorithm model is determined through multi-dimensional reward calculation, and the subsequent VPN requests are ensured to have better load balancing performance through online training of the deep deterministic policy gradient algorithm model. When a VPN request is received, the trained deep deterministic policy gradient algorithm model is used to determine the traffic allocation ratio with better load balancing performance, and the SRv6 backbone network status awareness is achieved through the network topology and dedicated path information, and the VPN traffic is deployed within the target path group according to the traffic allocation ratio, while achieving node load balancing and link load balancing to solve the problem of fragmentation and contradiction between node load balancing and link load balancing, and realizing the load balancing of VPN services in the SRv6 network.

[0121] In some embodiments, as Figure 5 shown, determining the link load factor of the backbone network according to the traffic matrix includes:

[0122] Step 501: Determine the bandwidth capacity and actual throughput of each link according to the traffic matrix, and determine the link utilization rate of each link according to the actual throughput and bandwidth capacity of the same link.

[0123] Specifically, when implemented, C represents the bandwidth capacity of the link, then C = [c 1 , c 2 , …, c M , where M represents the number of links, U represents the actual throughput of the link, then U = [u 1 , u 2 , …, u M , and R represents the link utilization rate, then R = [r 1 , r 2 , …, r M, then the calculation process of determining the link utilization rate of each link according to the actual throughput and bandwidth capacity of the same link is shown in formula (1):

[0124]

[0125] where r i represents the link utilization rate of link i; u i represents the actual throughput of link i; c i represents the broadband capacity of link i.

[0126] Step 502: Determine the average utilization rate according to the link utilization rate of each link, and determine the link load factor according to the link utilization rate and average utilization rate of each link.

[0127] Specifically, when implemented, the calculation process of the link load factor α in the backbone network is shown in formula (2):

[0128]

[0129] where α represents the link load factor, the value range of the link load factor α is [0, 1], the smaller α is, the better the link load balancing effect on the network is, avg represents the averaging operation, and avg(R) represents the average utilization rate.

[0130] In some embodiments, as Figure 6 shown, determining the service satisfaction of the VPN service according to the traffic matrix includes:

[0131] Step 601: Determine the actual allocated throughput of the target VPN service according to the traffic matrix.

[0132] Specifically, when implemented, A represents the actual allocated throughput of all VPN services, then A = [a 1 , a 2 , …, a X , where a i represents the actual allocated throughput of VPN service i, and X represents the number of VPN services.

[0133] Step 602: Determine the maximum broadband amount corresponding to the target path group, and determine the competing VPN services corresponding to the target VPN service within the target path group according to the traffic matrix.

[0134] Step 603: Determine the required broadband amount corresponding to each competing VPN service according to the traffic matrix, and determine the available broadband amount according to the maximum broadband amount and the required broadband amount.

[0135] Specifically, when implemented, B represents the bandwidth requirements of all VPN services, then B = [b 1 , b 2 , …, bX , b i represents the required bandwidth of VPN service i; S is the satisfaction of all VPN services, then S = [s 1 , s 2 , …, s X , s i is the service satisfaction of VPN service i. re i represents the maximum bandwidth of the path group where VPN service i is located, which is all the bandwidths that the path group where VPN service i is located can provide. If the target VPN service is VPN service i, then re i represents the maximum bandwidth corresponding to the target path group. Exemplarily, for example, if the three paths in the path group of a VPN service provide bandwidths of 10 Mbps, 20 Mbps, and 30 Mbps respectively, then the maximum bandwidth re of this VPN service = 10 + 20 + 30 = 60 Mbps. The calculation process for determining the available bandwidth based on the maximum bandwidth and the required bandwidth is shown in formula (3):

[0136] min i = min{re i - ∑ j∈X’{i} b′ j , b i} (3)

[0137] where, min i represents the available bandwidth; X′ represents the number of competing services of competing VPN services in the path group that have a bandwidth competition relationship with service i, b′ j is the required bandwidth of the competing VPN service in this path group, and mini represents the minimum allocable bandwidth in the path group. When the resources in the path group are sufficient, the minimum allocable bandwidth is consistent with the requirement.

[0138] Step 604: Determine the service satisfaction based on the available bandwidth and the actual allocated throughput.

[0139] Specifically, when implemented, the process of determining the service satisfaction based on the available bandwidth and the actual allocated throughput is shown in formula (4):

[0140]

[0141] where, s i represents the service satisfaction of VPN service i; the value range of s i is [0, 1], the larger s i , the higher the service satisfaction of VPN service i. From an overall perspective, the average value of the VPN service satisfaction should be increased as much as possible, that is, increase sum(S) / X to improve the overall satisfaction.

[0142] In some embodiments, such asFigure 7 As shown in the figure, determining the service dispersion degree of the target VPN service on the user side of the backbone network according to the traffic matrix includes:

[0143] Step 701: Determine the target user-side operator edge router corresponding to the target VPN service according to the traffic matrix, and determine the number of services deployed by the target user-side operator edge router in each service-side operator edge router according to the traffic matrix.

[0144] In specific implementation, use N1 to represent the number of user-side operator edge routers (user-side PEs). For the target user-side operator edge router PE i , any service-side operator edge router (service-side PE) can provide services for PE i . The service deployment vector is expressed as de i = [num 1 , num 2 , …, num N2 , where num j is the number of services deployed by the user-side PE i on the service-side PE j , and N2 is the number of service-side PEs.

[0145] Step 702: Determine the service deployment vector and the total number of services according to the number of services, and determine the service dispersion degree according to the determined service deployment vector and the total number of services.

[0146] In specific implementation, the process of calculating the service dispersion degree di i by ordered weighted average is shown in formulas (5) and (6):

[0147]

[0148] F(x, y) = x′ · y (6)

[0149] Where di i represents the service dispersion degree of the user-side PEi, de i represents the service deployment vector, sum is the summation operation, then sum(de i ) represents the total number of services, x′ is the ascending-ordered vector of vector x, F(x, y) represents the operation relationship between vector x and vector y, then F(de i , ω) = de i ′ · ω. The service dispersion degree di i is used to measure the service dispersion degree of the user-side PEi. The value range of di i is [0, 1]. The service dispersion degree di iThe smaller it is, the higher the dispersion degree. And the higher the dispersion degree, the stronger the availability for the VPN services initiated by the user-side PEi. It will not cause all services to be unavailable due to the downtime of a single service-side PE. From an overall perspective, sum(di) / N1 should be minimized as much as possible to ensure high availability, where di = [di 1 , di 2 , …, di N1 .

[0150] In some embodiments, as Figure 8 shown, determining the node load factor according to the traffic matrix includes:

[0151] Step 801: Determine the number of processable services and the actual number of processed services of the target service-side operator edge router according to the traffic matrix.

[0152] Specifically, for the service-side PEi, the number of processable services is cpt i , and the actual processing volume is used i .

[0153] Step 802: Determine the target link load rate of the target service-side operator edge router according to the number of processable services and the actual number of processed services.

[0154] Specifically, N2 is the number of service-side PEs. If the target service-side operator edge router is service-side PEi, the number of processable services of the target service-side operator edge router is cpt i , and the actual processing volume is used i , then the target link load rate is l i , and the calculation process of the target link load rate is shown in formula (7):

[0155]

[0156] where l i is the link load rate of service-side PEi.

[0157] Step 803: Determine the load rate vector according to the link load rate of each service-side operator edge router, and determine the node load factor according to the target link load rate and the load rate vector.

[0158] Specifically, the calculation method of the node load factor β of the service-side PE is shown in formula (8):

[0159]

[0160] where avg is the mean operation, L represents the load rate vector, then L = [l 1 , l 2,…,l N2 , the value range of the node load factor β is [0, 1]. The smaller β is, the better the node load balancing effect is.

[0161] In some embodiments, such as Figure 9 shown, the load balancing performance of different traffic allocation schemes determined according to the link load factor, service satisfaction, service dispersion, and node load factor includes:

[0162] Step 901: Determine the service satisfaction and the total number of services of each VPN service, and determine the average satisfaction according to the service satisfaction and the total number of services of each VPN service.

[0163] Specifically, when implemented, represents the average satisfaction, X represents the total number of services, S represents the service satisfaction of the backbone network, and S = [s 1 , s 2 ,…, s X , s i is the service satisfaction of VPN service i.

[0164] Step 902: Determine the number of user routes of the user-side carrier edge router and the service dispersion of each user-side carrier edge router, and determine the average dispersion according to the number of user routes and the service dispersion of each user-side carrier edge router.

[0165] Specifically, when implemented, represents the average dispersion, di represents the service dispersion of the user-side carrier edge router i of the backbone network, and N1 represents the number of user routes.

[0166] Step 903: Determine the load balancing performance according to the link load factor, node load factor, average satisfaction, and average dispersion:

[0167]

[0168] Among them, Performance represents the load balancing performance, and Performance ∈ [0, 1]. The larger Performance is, the better the load balancing effect is. α represents the link load factor, β represents the node load factor, represents the average satisfaction, X represents the total number of services, S represents the service satisfaction of the backbone network, represents the average dispersion, di represents the service dispersion of the user side of the backbone network, and N1 represents the number of user routes.

[0169] Specifically, when implemented, since the smaller the node load factor β is, the better the node load balancing effect is, and the smaller the link load factor α is, the better the link load balancing effect is, the average satisfaction The larger it is, the better the business satisfaction is. The smaller the average dispersion degree is, the stronger the availability of the VPN service initiated by the user-side PEi is. Therefore, when measuring the load balancing performance, (1-α), and (1-β) are used as four dimensions to measure the load balancing performance of the corresponding traffic allocation scheme. Among them, incorporating the business dispersion degree can enhance the high availability of the user-side PE service, incorporating the business satisfaction can enhance the service quality of a single service, incorporating the node load factor can ensure good node load balancing, and incorporating the link load factor can ensure good link load balancing, resolving the contradiction between link load balancing and node load balancing, and improving business satisfaction and business availability. Since the value range of each reward is [0,1], the value range of Performance is between [0,1]. The closer Performance is to 1, the better the load balancing performance of the SRv6 VPN service. And Performance is used as the training reward of the DDPG algorithm, that is, let reward = Performance.

[0170] Determine the traffic allocation ratio corresponding to the maximum load balancing performance through multi-dimensional reward calculation, and realize the SRv6 backbone network status perception based on the network topology and proprietary path information, and deploy the VPN traffic within the target path group according to the traffic allocation ratio, while realizing node load balancing and link load balancing, so as to solve the problem of fragmentation and contradiction between node load balancing and link load balancing, and realize the load balancing of the VPN service in the SRv6 network.

[0171] In some embodiments, as Figure 10 shown, determine the target path group corresponding to the target VPN service according to the network topology and proprietary path information, including:

[0172] Step 1001: Determine the target user-side operator edge router corresponding to the target VPN service in the network topology according to the proprietary path information, and determine all paths corresponding to the target user-side operator edge router.

[0173] Specifically, in the overall network structure of the backbone network as Figure 3 shown, if the user host that issues the VPN request is PC2, the corresponding user edge router device is CE2, and the user-side operator edge router connected to the user edge router device CE2 is PE1, that is, PE1 is the ingress PE, and the corresponding service-side operator edge routers include PE2, PE3, and PE4. If the service-side operator edge router PE3 cannot execute the VPN service or rejects a new VPN request, the target service-side operator edge routers include PE2 and PE4. Among them, all paths include the path from PE1 to PE2 and the path from PE1 to PE4.

[0174] Step 1002: Determine at least one shortest path among all paths, and integrate all the shortest paths to obtain a target path group.

[0175] In specific implementation, it can be seen that the shortest path is PE1 - P1 - PE2, and there is only one, so the target path group includes PE1 - P1 - PE2.

[0176] It should be noted that the method of the embodiment of the present application can be executed by a single device, such as a computer or a server, etc. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In such a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiment of the present application, and these multiple devices will interact with each other to complete the described method.

[0177] It should be noted that some embodiments of the present application are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order than in the above embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0178] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present application further provides a load balancing device for SRv6 network VPN services.

[0179] Refer to Figure 11 , the load balancing device for SRv6 network VPN services includes:

[0180] An information collection module 10, configured to: generate a network topology of the backbone network and proprietary path information corresponding to the VPN service according to the collected backbone network information;

[0181] A matrix generation module 20, configured to: in response to receiving a VPN request, determine a target VPN service corresponding to the VPN request, and generate a traffic matrix according to the current traffic distribution scheme of the backbone network and the VPN request;

[0182] A reward calculation module 30, configured to: determine the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side according to the traffic matrix;

[0183] The load balancing module 40 is configured to: determine the load balancing performance of different traffic allocation schemes based on the link load factor, service satisfaction, service dispersion, and node load factor, and determine the traffic allocation scheme with the maximum load balancing performance as the optimal traffic allocation scheme;

[0184] The traffic allocation module 50 is configured to: determine a target path group corresponding to the target VPN service according to the network topology and proprietary path information, and determine the traffic allocation ratio of the backbone network exit and each path within the path group in the ingress provider edge router of the backbone network according to the optimal traffic allocation scheme.

[0185] For the convenience of description, when describing the above devices, various modules are described separately according to their functions. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0186] The device in the above embodiment is used to implement the load balancing method for the SRv6 network VPN service in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0187] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present application further provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the load balancing method for the SRv6 network VPN service in any of the above embodiments.

[0188] Figure 12 FIG. shows a more specific schematic diagram of the hardware structure of the electronic device provided in this embodiment. The device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. Among them, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are communicatively connected to each other inside the device through the bus 1050.

[0189] The processor 1010 may be implemented in a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0190] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1020 can store the operating system and other application programs. When implementing the technical solutions provided in the embodiments of this specification through software or firmware, the relevant program codes are stored in the memory 1020 and called and executed by the processor 1010.

[0191] The input / output interface 1030 is used to connect to the input / output module to achieve information input and output. The input / output module can be configured as a component in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Among them, the input device can include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device can include a display, a speaker, a vibrator, an indicator light, etc.

[0192] The communication interface 1040 is used to connect to the communication module (not shown in the figure) to achieve communication and interaction between this device and other devices. Among them, the communication module can achieve communication through wired means (such as USB, network cable, etc.) or through wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0193] The bus 1050 includes a path for transmitting information between various components of the device (such as the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0194] It should be noted that although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040, and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, those skilled in the art can understand that the above device may also only include the components necessary to implement the solutions of the embodiments of this specification, and do not have to include all the components shown in the figure.

[0195] The electronic device in the above embodiment is used to implement the load balancing method for the corresponding SRv6 network VPN service in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0196] Based on the same inventive concept, corresponding to the method in any of the above embodiments, the present application also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the load balancing method for the SRv6 network VPN service as described in any of the foregoing embodiments.

[0197] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0198] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the load balancing method of the SRv6 network VPN service as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0199] Based on the same concept, corresponding to the method of any of the above embodiments, the present application also provides a computer program product, including computer program instructions, which when running on a computer, cause the computer to execute the method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0200] It can be understood that before using the technical solutions of the various embodiments in the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0201] For example, in response to receiving an active request from the user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be executed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application program, server, or storage medium that executes the operation of the technical solution of the present disclosure according to the prompt message.

[0202] As an optional but non-limiting implementation manner, the way of sending a prompt message to the user in response to receiving an active request from the user can be, for example, in the form of a pop-up window, and the prompt message can be presented in text in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0203] It should be understood that the above notification and the process of obtaining user authorization are only illustrative and do not limit the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0204] Those of ordinary skill in the art should understand that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present application is limited to these examples; under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present application as described above, and they are not provided in detail for the sake of brevity.

[0205] In addition, for the sake of simplicity of explanation and discussion, and in order not to make the embodiments of the present application difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. In addition, the devices may be shown in block diagram form in order not to make the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation manners of these block diagram devices are highly dependent on the platform on which the embodiments of the present application are to be implemented (that is, these details should be completely within the understanding of those skilled in the art). In the case where specific details (such as circuits) are set forth to describe the exemplary embodiments of the present application, it will be apparent to those skilled in the art that the embodiments of the present application can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0206] Although the present application has been described in connection with specific embodiments of the present application, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. For example, other memory architectures (such as dynamic RAM (DRAM)) can be used with the embodiments discussed.

[0207] The embodiments of the present application are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the claims of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the embodiments of the present application shall be included within the protection scope of the present application.

Claims

1. A load balancing method for SRv6 network VPN service, characterized in that: include: Generate the network topology of the backbone network and the proprietary path information corresponding to the VPN service based on the collected backbone network information; Determine a historical traffic distribution ratio and a historical load balancing performance corresponding to historical VPN requests, and determine a traffic matrix according to the historical traffic distribution scheme of the backbone network and the historical traffic distribution ratio; Determine, according to the traffic matrix, a link load factor of the backbone network, a service satisfaction level of the target VPN service, a service dispersion degree of the target VPN service on the user side of the backbone network, and a node load factor of the backbone network on the service side; a current load balancing performance of a current traffic distribution scheme determined according to the link load factor, the service satisfaction, the service dispersion, and the node load factor; Online training of a deep deterministic policy gradient algorithm model according to the current load balancing performance and the historical load balancing performance; In response to receiving a VPN request, determining a target VPN service corresponding to the VPN request, and generating a request traffic matrix according to the current traffic allocation scheme and the VPN request; A target path group corresponding to the target VPN service is determined according to the network topology and the proprietary path information, the request traffic matrix is ​​input into a deep deterministic policy gradient algorithm model after online training, and the traffic allocation ratio of the backbone network exit and each path in the target path group is determined in the ingress operator edge router of the backbone network according to the model output.

2. The method according to claim 1, characterized in that Determining a link load factor of a backbone network according to the traffic matrix includes: Determining the bandwidth capacity and the actual throughput of each link according to the traffic matrix, and determining the link utilization of each link according to the actual throughput and the bandwidth capacity of the same link; An average utilization is determined according to the link utilization of each link, and the link load factor is determined according to the link utilization of each link and the average utilization.

3. The method according to claim 1, characterized in that: Determining the service satisfaction of the VPN service according to the traffic matrix includes: Determine the actual allocated throughput of the target VPN service according to the traffic matrix; Determine a maximum bandwidth corresponding to the target path group, and compete VPN services corresponding to the target VPN services in the target path group according to the traffic matrix; Determine the required bandwidth amount corresponding to each competing VPN service according to the traffic matrix, and determine the available bandwidth amount according to the maximum bandwidth amount and the required bandwidth amount; The service satisfaction level is determined according to the available bandwidth amount and the actual allocated throughput.

4. The method according to claim 1, characterized in that: Determining the service dispersion of the target VPN service on the user side of the backbone network according to the traffic matrix includes: Determine a target user-side operator edge router corresponding to the target VPN service according to the traffic matrix, and determine the number of services deployed by the target user-side operator edge router in each service-side operator edge router according to the traffic matrix; A service deployment vector and a total service quantity are determined according to the service quantity, and the service dispersion is determined according to the determined service deployment vector and the total service quantity.

5. The method according to claim 1, characterized in that Determining a node load factor according to the traffic matrix includes: Determine the number of services that can be processed and the number of services actually processed by the target service-side operator edge router according to the traffic matrix; Determine a target link load rate of the target service-side operator edge router according to the processable service quantity and the actually processed service quantity; A load rate vector is determined according to the link load rate of each service-side operator edge router, and the node load factor is determined according to the target link load rate and the load rate vector.

6. The method according to claim 1, characterized in that The load balancing performance of different traffic distribution schemes determined according to the link load factor, the service satisfaction, the service dispersion and the node load factor includes: Determine the service satisfaction and the total number of services for each VPN service, and determine the average satisfaction according to the service satisfaction and the total number of services for each VPN service; Determine the number of user routes of the user-side operator edge router and the service dispersion of each user-side operator edge router, and determine the average dispersion according to the number of user routes and the service dispersion of each user-side operator edge router; Determine the load balancing performance according to the link load factor, the node load factor, the average satisfaction and the average dispersion: Among them, Performance represents the load balancing performance, and Performance∈[0,1], where the larger the Performance is, the better the load balancing effect is, α represents the link load factor, and β represents the node load factor. represents the average satisfaction, X represents the total number of services, S represents the service satisfaction of the backbone network, represents the average dispersion, di represents the service dispersion on the user side of the backbone network, and N1 represents the number of user routes.

7. The method according to claim 1, characterized in that The determining, according to the network topology and the proprietary path information, a target path group corresponding to the target VPN service includes: Determine a target user-side operator edge router corresponding to the target VPN service in the network topology according to the proprietary path information, and determine all paths corresponding to the target user-side operator edge router; At least one shortest path is determined among all the paths, and all the shortest paths are integrated to obtain the target path group.

8. The method according to claim 1, characterized in that: The online training of the deep deterministic policy gradient algorithm model according to the current load balancing performance and the historical load balancing performance includes: In response to the historical load balancing performance being better than the current load balancing performance, and the current load balancing performance being lower than a preset load balancing performance threshold, the online deep deterministic policy gradient algorithm model is trained according to the training data until a training end condition is met, and a new deep deterministic policy gradient algorithm model is redeployed; In response to the historical load balancing performance being better than the current load balancing performance, and the current load balancing performance being higher than a preset load balancing performance threshold, storing the current load balancing performance, the historical traffic distribution ratio, the historical load balancing performance, the link load factor, the service satisfaction, the service dispersion, and the node load factor into an experience pool; In response to the current load balancing performance being better than the historical load balancing performance, the current load balancing performance, the historical traffic distribution ratio, the historical load balancing performance, the link load factor, the service satisfaction, the service dispersion and the node load factor are stored in an experience pool.

9. A load balancing device for SRv6 network VPN service, characterized in that: include: The information collection module is configured to: generate a network topology of the backbone network and proprietary path information corresponding to the VPN service according to the collected backbone network information; The matrix generation module is configured to: determine the historical traffic distribution ratio and the historical load balancing performance corresponding to the historical VPN request, and determine the traffic matrix according to the historical traffic distribution scheme of the backbone network and the historical traffic distribution ratio; The reward calculation module is configured to: determine the link load factor of the backbone network, the service satisfaction of the target VPN service, the service dispersion of the target VPN service on the user side of the backbone network, and the node load factor of the backbone network on the service side according to the traffic matrix; A load calculation module, configured to: determine the current load balancing performance of the current traffic distribution scheme according to the link load factor, the service satisfaction, the service dispersion and the node load factor; An online training module is configured to: online train a deep deterministic policy gradient algorithm model according to the current load balancing performance and the historical load balancing performance; The request response module is configured to: in response to receiving a VPN request, determine a target VPN service corresponding to the VPN request, and generate a request traffic matrix according to the current traffic allocation scheme and the VPN request; The load balancing module is configured to: determine a target path group corresponding to the target VPN service according to the network topology and the proprietary path information, input the request traffic matrix into a deep deterministic policy gradient algorithm model after online training, and determine the traffic distribution ratio of the backbone network exit and each path in the target path group in the ingress operator edge router of the backbone network according to the model output.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

Images