Transmission optimization method and related equipment for network cluster system

Abstract

The embodiments of the present application propose a transmission optimization method and related equipment for a network cluster system. The network cluster system includes multiple network clusters, and the network cluster includes multiple network nodes. The method includes: first, obtaining the link status of the communication link between each two network nodes, and obtaining a link connection matrix, which is used to characterize the connection status of each communication link; then, obtaining a network cluster scheduling model, inputting all link statuses and link connection matrices into the network cluster scheduling model, and obtaining an output link optimized transmission rate sequence and optimized traffic distribution ratio for each network cluster; finally, adjusting the transmission rate of each communication link in the corresponding network cluster according to each link optimized transmission rate sequence, and adjusting the traffic distribution of the corresponding network cluster according to each optimized traffic distribution ratio. When information is transmitted between network nodes in the network cluster system, the transmission resource utilization rate can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a transmission optimization method and related equipment for a network cluster system. Background Art

[0002] With the rapid growth of modern computing demands, large-scale network clusters have become increasingly sophisticated. With the continuous improvement of hardware capabilities, large-scale networks with multi-network convergence and heterogeneous coexistence are becoming a mainstream development trend. However, due to the large number of network nodes in large-scale networks, appropriate resource scheduling in such networks is a relatively complex issue.

[0003] In related technologies, appropriate communication link parameters are typically preconfigured between each two nodes in a large-scale network cluster to improve data transmission reliability. However, due to the highly dynamic nature of the communication links between each two nodes in a large-scale network cluster, this resource configuration method can lead to low transmission resource utilization when transmitting information in the cluster. Summary of the Invention

[0004] The embodiments of the present application provide a transmission optimization method and related equipment for a network cluster system, which can improve the utilization rate of transmission resources when a large-scale network cluster performs information transmission.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a transmission optimization method for a network cluster system, wherein the network cluster system includes multiple network clusters, each of which includes multiple network nodes. The method includes:

[0006] Acquire a link state of a communication link between every two of the network nodes, and acquire a link connection matrix, wherein the link connection matrix is ​​used to represent the connection state of each of the communication links;

[0007] Obtaining a network cluster scheduling model, inputting all the link states and the link connection matrix into the network cluster scheduling model, and outputting a link optimized transmission rate sequence and an optimized traffic distribution ratio for each network cluster;

[0008] The transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic distribution of the corresponding network cluster is adjusted according to each optimized traffic distribution ratio.

[0009] In some embodiments, obtaining the network cluster scheduling model includes:

[0010] Obtaining a load balancing optimization model for each of the network clusters;

[0011] Generating a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model;

[0012] The network cluster scheduling model of the network cluster system is generated based on the collaborative load balancing optimization model.

[0013] In some embodiments, obtaining the load balancing optimization model of each of the network clusters includes:

[0014] Obtaining a transmission rate parameter and a maximum bandwidth of each of the communication links, and obtaining a transmission rate parameter range constraint of the transmission rate parameter;

[0015] Obtaining a throughput model for each of the network clusters based on a cumulative sum of the plurality of transmission rate parameters;

[0016] Obtaining ratios of a plurality of the transmission rate parameters to the maximum bandwidth, and accumulating the ratios to obtain an energy utilization model for each of the network clusters;

[0017] Accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function;

[0018] The load balancing optimization model is generated based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint.

[0019] In some embodiments, generating the collaborative load balancing optimization model of the network cluster based on the load balancing optimization model includes:

[0020] Generate a cluster state space based on the link connection matrix and a link bandwidth parameter, a link delay parameter, a queue packet loss parameter, and a queue depth generation parameter of each of the communication links;

[0021] generating a cluster action space based on the transmission rate parameter, and generating a cluster reward model based on the target load balancing optimization function;

[0022] Obtaining initial graph convolutional network parameters of a graph convolutional neural network, and generating an initial collaborative load balancing optimization model based on the cluster state space, the cluster action space, the cluster reward model, and the initial graph convolutional network parameters;

[0023] The initial collaborative load balancing optimization model is trained for multiple rounds, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

[0024] In some embodiments, the training process of the initial collaborative load balancing optimization model includes:

[0025] Obtaining a cluster training state and a training strategy for a current iteration, determining a cluster training action from the training strategy based on the cluster training state, and obtaining a cluster training update state based on the cluster training action;

[0026] Calculating a training loss value of the cluster training action based on the cluster reward model, the cluster training update state, the cluster training state, and the cluster training action;

[0027] Obtaining a fitting function of the graph convolutional neural network, and updating the initial graph convolutional network parameters based on the training loss value and the fitting function;

[0028] The training strategy is updated based on the updated graph convolutional neural network.

[0029] In some embodiments, generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model includes:

[0030] Based on the collaborative load balancing optimization model of each network cluster, obtaining an optimized cluster training action for each network cluster;

[0031] Based on all the optimized cluster training actions in each of the network clusters, obtaining a training traffic allocation ratio for each network cluster;

[0032] Obtaining an update status of each of the network clusters based on the training traffic allocation ratio, and updating the training traffic allocation ratio based on the update status;

[0033] The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0034] In some embodiments, the updated state includes an updated value of a target load balancing optimization function and an updated value of a packet loss rate, and updating the training traffic distribution ratio based on the updated state includes:

[0035] Selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a first target adjustment cluster, and updating the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate;

[0036] Based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value are obtained;

[0037] When the updated target load balancing optimization function update value is a negative number and the updated packet loss rate update value is a positive number, selecting the network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as the second target adjustment cluster;

[0038] The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

[0039] To achieve the above-mentioned objectives, a second aspect of an embodiment of the present application provides a transmission optimization device for a network cluster system, wherein the network cluster system includes multiple network clusters, each of which includes multiple network nodes. The device includes:

[0040] a state acquisition module, configured to acquire the link state of the communication link between each two network nodes, and to acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize the connection state of each communication link;

[0041] an optimization parameter acquisition module, configured to acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and output a link optimized transmission rate sequence and an optimized traffic distribution ratio for each network cluster, wherein the link optimized transmission rate sequence and the optimized traffic distribution ratio are used to optimize the throughput and resource utilization within the network cluster system;

[0042] a parameter adjustment module, configured to adjust the transmission rate of each communication link in the corresponding network cluster one by one according to the optimized transmission rate sequence of each link, and to adjust the traffic distribution of the corresponding network cluster one by one according to each optimized traffic distribution ratio;

[0043] The information transmission module is used to transmit information in at least two of the network nodes after the transmission rate of at least one of the communication links and the traffic distribution of at least one of the network clusters are adjusted.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the transmission optimization method of the network cluster system as described in the first aspect.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, it implements the transmission optimization method of the network cluster system described in the first aspect above.

[0046] The transmission optimization method and related equipment of a network cluster system proposed in an embodiment of the present application, the network cluster system includes multiple network clusters, and the network cluster includes multiple network nodes. The method includes: first, obtaining the link status of the communication link between each two network nodes, and obtaining the link connection matrix, which is used to characterize the connection status of each communication link; then, obtaining a network cluster scheduling model, inputting all link statuses and link connection matrices into the network cluster scheduling model, and obtaining the output link optimized transmission rate sequence and optimized traffic distribution ratio of each network cluster; finally, adjusting the transmission rate of each communication link in the corresponding network cluster according to each link optimized transmission rate sequence, and adjusting the traffic distribution of the corresponding network cluster according to each optimized traffic distribution ratio. The embodiment of the present application utilizes the link status of the changing communication links in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicationally connected, and combines it with a network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, so as to obtain the link optimization transmission rate sequence and optimized traffic distribution ratio of multiple network clusters that are appropriately matched with the link status and link connection matrix, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, thereby effectively improving the transmission resource utilization when information is transmitted between network nodes in the network cluster system.

[0047] Other features and advantages of the present application will be described in the following description, and in part will become apparent from the description, or will be understood by practicing the present application. The purposes and other advantages of the present application can be achieved and obtained through the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a structural diagram of a network cluster system provided in one embodiment of the present application.

[0049] Figure 2 This is a flowchart of a transmission optimization method for a network cluster system provided in another embodiment of the present application.

[0050] Figure 3 yes Figure 2 Flowchart of step 202 in FIG.

[0051] Figure 4 yes Figure 3 Flowchart of step 301 in FIG.

[0052] Figure 5 This is a schematic diagram of a multi-network collaborative load balancing scheduling framework provided by another embodiment of the present application.

[0053] Figure 6 yes Figure 3 Flowchart of step 302 in FIG.

[0054] Figure 7 This is a flowchart of generating an initial collaborative load balancing optimization model provided by another embodiment of the present application.

[0055] Figure 8 This is a training process diagram of an initial collaborative load balancing optimization model provided by another embodiment of the present application.

[0056] Figure 9 This is a schematic diagram of a training algorithm for an initial collaborative load balancing optimization model provided in another embodiment of the present application.

[0057] Figure 10 Figure 3 Flowchart of step 303 in FIG.

[0058] Figure 11 This is a schematic diagram of a generation process of a network cluster scheduling model provided in another embodiment of the present application.

[0059] Figure 12 This is a schematic diagram of a generation algorithm of a network cluster scheduling model provided in another embodiment of the present application.

[0060] Figure 13 yes Figure 10 Flowchart of step 1003 in FIG.

[0061] Figure 14 It is a structural diagram of a transmission optimization device for a network cluster system provided in one embodiment of the present application.

[0062] Figure 15 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0066] First, let’s analyze some of the terms used in this application:

[0067] Deep reinforcement learning is a branch of machine learning that combines the powerful representation learning capabilities of deep learning with the goal-oriented mechanisms of reinforcement learning, enabling intelligent agents to learn how to make optimal decisions in an uncertain environment. Specifically, deep reinforcement learning uses deep neural networks as function approximators to estimate the value of actions or directly output action policies, allowing intelligent agents to learn the optimal path to achieve specific goals through trial and error while interacting with the environment. This approach significantly improves the model's learning and generalization capabilities, especially when processing high-dimensional input data such as images and natural language.

[0068] With the rapid growth of modern computing demands, large-scale network clusters have become increasingly sophisticated. With the continuous improvement of hardware capabilities, large-scale networks with multi-network convergence and heterogeneous coexistence are becoming a mainstream development trend. However, due to the large number of network nodes in large-scale networks, appropriate resource scheduling in such networks is a relatively complex issue.

[0069] In related technologies, appropriate communication link parameters are typically preconfigured between each two nodes in a large-scale network cluster to improve data transmission reliability. However, due to the highly dynamic nature of the communication links between each two nodes in a large-scale network cluster, this resource configuration method can lead to low transmission resource utilization when transmitting information in the cluster.

[0070] In order to improve the utilization rate of transmission resources when information is transmitted in a large-scale network cluster, an embodiment of the present application utilizes the link status of the changing communication links in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicationally connected, and combines it with a network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, thereby obtaining a link optimization transmission rate sequence and an optimized traffic distribution ratio of multiple network clusters that are appropriately matched with the link status and the link connection matrix, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, thereby effectively improving the utilization rate of transmission resources when information is transmitted between network nodes in the network cluster system.

[0071] In order to better describe the transmission optimization method of the network cluster system provided by this application, the following first describes the network cluster system applied to the transmission optimization method of the network cluster system. Figure 1 , is a schematic diagram of the structure of a network cluster system provided by an embodiment of the present application. Figure 1As shown in , it includes multiple network clusters, each network cluster includes multiple network nodes. Wherein, every two network nodes can directly communicate with each other, and every two network clusters can also communicate with each other.

[0072] In conjunction with the above-mentioned network cluster system, the transmission optimization method and related equipment of the network cluster system provided in the embodiments of the present application will be further described below. The transmission optimization method of the network cluster system provided in the embodiments of the present application can be applied to any network node with computing capabilities in the network cluster system, or a processing controller connected to the network cluster system.

[0073] The following will describe in detail the transmission optimization method of the network cluster system in the embodiment of the present application. Figure 2 , which is an optional flow chart of the transmission optimization method of the network cluster system provided in an embodiment of the present application, Figure 2 The method may include but is not limited to steps 201 to 203. It is also understood that this embodiment is for Figure 2 The order of step 201 to step 203 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0074] Step 201: Obtain the link status of the communication link between every two network nodes, and obtain a link connection matrix.

[0075] Step 201 is described in detail below.

[0076] In some embodiments, in order to improve the transmission resource utilization when large-scale network clusters are transmitting information, it is necessary to conduct preliminary research and analysis on large-scale network clusters. Figure 1 As shown in the figure, all the network nodes participating in the cooperative transmission are regarded as a fusion network, and a fixed undirected network can be obtained. At this point, the clustering idea is introduced to divide the entire network, and M network clusters can be obtained to form a network cluster set. At this point, define the network cluster k m The link set between every two network nodes in l (m,j) represents the network cluster k m The jth communication link. m represents the network cluster k m The number of network nodes in n. (m,p) represents the network cluster k m There are p communication links. It can be understood that any network cluster k m The total number of links p is arbitrary.

[0077] In order to improve the utilization of transmission resources when large-scale network clusters are transmitting information, it is necessary to obtain the link status S of all communication links between every two network nodes in each network cluster before each resource management configuration of the network cluster system. m (t)∈R p×fs , where fs represents the link status including: link bandwidth, link transmission delay, packet loss rate during queuing during transmission, and queue length during queuing during transmission in the communication link. m represents the mth network cluster, and p represents the number of communication links in the network cluster.

[0078] Since in practical applications, usually not every two network nodes need to transmit information in the communication link, it is also necessary to obtain the link connection matrix ε of all network clusters in advance. m,ij ) nm×nm , where l m,ij For any two network nodes n in the mth network cluster (m,i) With n (m,j) Connectivity; nm is equivalent to n in the above m , that is, the same network cluster k m The number of network nodes in the network. (m,ij) =1 means the two network nodes are connected; otherwise l (m,ij) =0, indicating that the link is disconnected.

[0079] Step 202: Obtain a network cluster scheduling model, input all link states and link connection matrices into the network cluster scheduling model, and output a link optimized transmission rate sequence and an optimized traffic distribution ratio for each network cluster.

[0080] Step 202 is described in detail below.

[0081] In some embodiments, a deep reinforcement learning framework and a network cluster system are pre-built to effectively output the most appropriate resource configuration parameters within the network cluster system (i.e., the optimized transmission rate sequence and optimized traffic distribution ratio for each network cluster) for different link states and link connection matrices. This further improves transmission resource utilization when large-scale network clusters transmit information. The following further describes how to build this network cluster scheduling model.

[0082] Reference Figure 3 , obtaining a network cluster scheduling model, including the following steps 301 to 303.

[0083] Step 301: Obtain a load balancing optimization model for each network cluster.

[0084] Step 301 is described in detail below.

[0085] In some embodiments, to construct a suitable network cluster scheduling model, mathematical modeling must be performed based on the network cluster system in advance to construct a mathematical optimization model for improving transmission resource utilization. This mathematical optimization model can then be used to build a reinforcement learning model. The following further describes how to construct a mathematical optimization model corresponding to the network cluster system.

[0086] Reference Figure 4 , obtaining a load balancing optimization model for each network cluster, including the following steps 401 to 405.

[0087] Step 401: Acquire the transmission rate parameter and maximum bandwidth of each communication link, and acquire the transmission rate parameter range constraint of the transmission rate parameter.

[0088] Step 402: Obtain a throughput model for each network cluster based on the cumulative sum of multiple transmission rate parameters.

[0089] Step 403: Obtain ratios of multiple transmission rate parameters and maximum bandwidth, and accumulate the ratios to obtain an energy utilization model for each network cluster.

[0090] Step 404: Accumulate the throughput model and the energy utilization model to generate a target load balancing optimization function.

[0091] Step 405: Generate a load balancing optimization model based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint.

[0092] Steps 401 to 405 are described in detail below.

[0093] In some embodiments, in order to clearly understand the network node transmission control process in the network cluster system, the traffic scheduling process in the network cluster system is divided into continuous discrete time slots t∈{0,1,2,…}. Define the network cluster k m Communication link within (m,j) The arrival rate, transmission rate, and packet loss rate are α (m,j) (t), b (m,j) (t), d (m,j) (t). After the clustering is performed in the network cluster system, each network cluster k m There will be an internal corresponding transmission control strategy π m Execute transfer action a m (t) Control the transmission rate of each link, that is, Define network cluster k m When all communication links in the t time slot, the link state matrix corresponding to the link state is S m (t)∈Rp×fs Considering the heterogeneity of heterogeneous network node resources in the network cluster system, fs is defined as the maximum state attribute dimension of all links. At this time, the traffic transmission control problem in each network cluster is transformed into: according to the link state S m (t), the transmission control strategy π within the scheduling network cluster m Execute control action a m (t), maximize the network throughput within the network cluster T m (t) and maximize resource utilization U m (t)Target.

[0094] That is, for each network cluster, first determine the transmission rate parameter (ie, transmission rate b) of each communication link (such as the jth communication link) in each network cluster (such as the mth network cluster). (m,j) (t)) and the maximum bandwidth B m,j , and determine the transmission rate parameter b (m,j) (t) Transmission rate parameter range constraint b (m,j) (t)≤B m,j .

[0095] Next, based on multiple transmission rate parameters b (m,j) The cumulative sum of (t) is used to obtain the throughput model of each network cluster Furthermore, the ratios of multiple transmission rate parameters and maximum bandwidth are obtained, and the ratios are accumulated to obtain the energy utilization model U of each network cluster. m (t)=Σ (m,j) b (m,j) (t) / B m,j Then, the throughput model and energy utilization model are accumulated to generate the target load balancing optimization function

[0096] Finally, based on the transmission rate parameter b (m,j) (t), target load balancing optimization function ξ e1 As well as the transmission rate parameter range constraints, the load balancing optimization model for each network cluster is generated as shown in the following optimization problem (P1).

[0097]

[0098] α (m,j) (t)≤B m,j ,b (m,j) (t)≤B m,j , (4)

[0099] a m (t)=π m (S m (t)). (5)

[0100] Among them, ST is the scheduling time interval. is the network cluster k m All communication links l in time slot t (m,j) The transmission rate b (m,j) (t). β is the balance weight between the objectives of maximizing throughput and maximizing resource utilization in each network cluster. m,j is the network cluster k m The maximum bandwidth of the jth link. Formula (1) is the overall goal (i.e., the target load balancing optimization function), which strives to achieve a trade-off between maximizing throughput and maximizing resource utilization within the ST time interval. Formula (2) is calculated by network cluster k m The throughput in time slot t is calculated based on the transmission rate of the network cluster k. m The bandwidth resource utilization rate in time slot t is calculated based on the transmission rate of the network cluster k. m Communication Link (m,j) The transmission rate and arrival rate are constrained and cannot be greater than the maximum transmission bandwidth B of the communication link. m,j Formula (5) is the network cluster k m The execution strategy constraint in time slot t. This load balancing optimization model can be understood as: under the constraint of limited bandwidth resources, the network cluster k m Finding the optimal transmission control strategy π m . Then π m According to the network status S in time slot t m (t) Conduct transmission control strategy guidance m (t), achieving the trade-off goal of maximizing network throughput and maximizing resource utilization.

[0101] Through the above steps 401 to 405, the relevant mathematical modeling parameters in the network cluster system are used to adjust the transmission rate of the communication link between the network nodes in each network cluster to achieve the purpose of improving the throughput and bandwidth resource utilization within the network cluster, and construct a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model. The collaborative load balancing optimization model can be effectively used to improve the resource utilization in the network cluster system.

[0102] In some embodiments, in an actual network cluster system In the process, due to the randomness of the transmission traffic and the dynamic nature of the communication link state, the load capacity of the communication link will fluctuate. Therefore, a queuing model is further defined to temporarily store the extra data that exceeds the instantaneous load of the network, such as in the time slot t of the network cluster k m The queue model in the jth link is Q(m,j) (t). And further based on the queuing model, transmission rate and packet loss rate of time slot t, the queuing model update model is generated as shown in the following formula (6).

[0103]

[0104] At this time, the network cluster k m The set of all link queuing vectors in is Taking into account the overall transmission stability of the network cluster system, assuming that the network cluster k m In control strategy and the maximum network capacity Λ, any communication link l (m,j) The long-term queue depth Q (m,j) (t) are subject to the long-term stability constraint as shown in the following formula (7).

[0105]

[0106] Based on the above long-term stability constraint analysis and the load balancing optimization model of each network cluster, the load balancing optimization model can be further transformed into a collaborative load balancing model for the network cluster system. In this case, assuming that the access network node of network cluster m needs to send large-scale data traffic to the network node of another network cluster g, according to the connectivity E of each network cluster, it is necessary to select multiple adjacent network clusters to form a collaborative transmission group. Perform load balancing transmission control. At this point, based on the load balancing optimization model P1 of a single network cluster, the cluster collaborative load balancing model P2 can be further obtained as shown below.

[0107]

[0108] in, The goal of the cluster collaboration load balancing model P2 is to transform the internal collaborative transmission of each network cluster into the collaborative transmission between multiple network clusters in the network cluster system. Formula (8) is the goal of this model, which aims to achieve the joint goal of maximizing the throughput and maximizing the bandwidth utilization during the cluster collaboration process. Formula (9) calculates the total throughput of the collaboration between network clusters in time slot t. Formula (10) calculates the total bandwidth utilization of the collaboration between network clusters in time slot t. Formula (11) represents the collaboration strategy between network clusters. Under control, according to the control strategy of each network cluster m Control the traffic distribution ratio of the corresponding network cluster m (t). Formula (12) is the stability constraint of the cooperative transmission queue.

[0109] Step 302: Generate a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model.

[0110] Step 302 is described in detail below.

[0111] In some embodiments, after obtaining the load balancing optimization model of each network cluster, in order to achieve accurate multi-network cluster collaborative load balancing scheduling, taking into account the problems of large network scale, perception and control difficulties in the process of multi-network cluster collaborative transmission within the network cluster system, a multi-network collaborative load balancing scheduling framework is studied and designed. Figure 5 , is a schematic diagram of a multi-network collaborative load balancing scheduling framework provided by an embodiment of the present application. Figure 5 As shown in [1], a distributed architecture concept is introduced to manage the entire network cluster system in clusters. Each cluster has an independent perception module, policy update and iteration module, and policy execution control module, which can accelerate the perception and control process. However, distributed cluster management of the network is difficult to achieve globally optimal scheduling. Therefore, this embodiment also introduces a distributed asynchronous learning method to perform distributed asynchronous learning and update of the scheduling policy of each cluster, enhancing the global average optimal scheduling capability.

[0112] exist Figure 5 The multi-network collaborative load balancing scheduling framework shown here primarily includes network awareness, network clustering management, policy update, and asynchronous policy learning. Network awareness primarily utilizes the INT framework to perceive network node status and traffic; network clustering management is responsible for clustering networks with similar transmission properties; policy update primarily utilizes awareness data and the actor-critic framework of reinforcement learning to iteratively generate policies; and asynchronous policy learning utilizes distributed asynchronous learning methods to share policies between algorithms with the same model structure and attribute inputs. The framework is based on a three-layer design consisting of an intelligent service layer, a resource adaptation layer, and a network component layer. Five components are designed to implement network awareness, policy generation, and policy delivery. These components are primarily distributed within the network component layer and the resource adaptation layer. Specific component functions are described below.

[0113] Perception Observer: Mainly responsible for perceiving and detecting all network component resources and traffic. This chapter mainly combines the INT framework to perform detection tasks.

[0114] Cluster manager: performs clustering and grouping control of nodes according to clustering control strategy.

[0115] Cross-layer parser: This is primarily responsible for parsing and extracting perception data, as well as encapsulating and delivering dynamic policy data. Specifically, when the cross-layer parser receives INT perception data packets from the perception observer, it parses the INT packets to extract the perception data. Conversely, when the cross-layer parser receives policy actions from the policy generator or policy parameters from the asynchronous learner, it generates a policy package that encapsulates the corresponding actions and parameters and delivers them.

[0116] Data analyzer: Mainly responsible for further formatting of perception data and providing the required formatted data input for the algorithm.

[0117] Policy generator: generates and updates policies based on embedded algorithms and formatted perception data.

[0118] Asynchronous learner: It mainly performs periodic learning and updates based on the strategy of each cluster, and sends the strategy parameters to each cluster head for traffic load balancing and scheduling.

[0119] Adaptive scheduler: controls the traffic transmission rate of all links according to the scheduling strategy.

[0120] Based on Figure 5 The multi-network collaborative load balancing scheduling framework shown below further describes how to generate a collaborative load balancing optimization model for a network cluster based on the load balancing optimization model.

[0121] Reference Figure 6 , generating a collaborative load balancing optimization model of a network cluster based on the load balancing optimization model, including the following steps 601 to 604.

[0122] Step 601: Generate a cluster state space based on the link connection matrix and the link bandwidth parameter, link delay parameter, queue packet loss parameter and queue depth generation parameter of each communication link.

[0123] Step 602: Generate a cluster action space based on the transmission rate parameter, and generate a cluster reward model based on the target load balancing optimization function.

[0124] Step 603: Obtain initial graph convolutional network parameters of the graph convolutional neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model and the initial graph convolutional network parameters.

[0125] Step 604: Perform multiple rounds of training on the initial collaborative load balancing optimization model, and obtain a collaborative load balancing optimization model of the network cluster based on the trained initial collaborative load balancing optimization model.

[0126] Steps 601 to 604 are described in detail below.

[0127] In some embodiments, taking into account the heterogeneity of network node resources within the network cluster system and the different load capacities of each communication link, a graph convolutional neural network (GCN) is introduced. The "barrel" short board idea is used to cluster adjacent nodes / links to realize the joint load capacity analysis of multiple nodes, and then generate strategies for transmission control.

[0128] Then, combined with the framework of reinforcement learning, the link state S of the communication link between each two network nodes in each time slot network cluster system is calculated. m (t)∈R p×fs , where fs includes link bandwidth parameters, link delay parameters, queue packet loss parameters, and queue depth generation parameters, and is further combined with the link connection matrix ε=(l m,ij ) nm×nm Generate a cluster state space in each time slot network cluster system.

[0129] Then, the transmission rate parameter of the communication link between each two network nodes in each time slot network cluster system is used to generate the cluster action space

[0130] In the reinforcement learning framework, the reward value is the feedback after the action is executed. The policy parameters are adjusted according to the quality of the feedback, and then the control action is adjusted to meet the current S m Therefore, according to the target load balancing optimization function (1) in the load balancing optimization model P1, the cluster reward model is generated as shown in the following formula (13).

[0131]

[0132] In addition, according to the above definition, the input layer of the graph convolutional neural network is set to {S m (t), A}. The hidden layer iteration process of the graph convolutional neural network is shown in the following formula (14).

[0133]

[0134] Among them, the input of the l+1 layer network is H l ∈R P×fs , in the initial output layer H 0 =S m (t) P×fs fs is the dimension of the network state feature vector. P is the number of links. To add a self-connected connected adjacency matrix, I P is the unit matrix. A is the current network cluster k mThe adjacency matrix E of . is the degree matrix, W l is the graph neural network parameter. σ is the activation function. At this time, the polynomial fitting function of the graph convolutional neural network is defined as shown in the following formula (15).

[0135] Z=f(X,A) (15)

[0136] X is the input of each layer, and in the initial output layer X = H 0 =S m (t) P×fs At this point, based on the above cluster reward model (13), the loss function of the graph convolutional neural network can be further generated as shown in the following formula (16).

[0137] Loss=(ξ r (t)-ξ r (t-1))lnZ. (16)

[0138] Based on the above description and combined with the reinforcement learning framework, we first determine the initial graph convolutional network parameters of the graph convolutional neural network (which include the relevant iteration parameters in the hidden layer, i.e., the relevant parameters in formula (14)); then, we generate the initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model, and the initial graph convolutional network parameters.

[0139] Next, based on a reinforcement learning algorithm framework, we conduct multiple rounds of training on the initial collaborative load balancing optimization model. Based on this trained initial collaborative load balancing optimization model, we develop a collaborative load balancing optimization model for the network cluster. This effectively improves the transmission resource utilization of the network cluster system in practical applications. The following section further describes how to train the initial collaborative load balancing optimization model.

[0140] Reference Figure 7 The training process of the initial collaborative load balancing optimization model includes the following steps 701 to 704.

[0141] Step 701: Obtain the cluster training state and training strategy of the current iteration, determine the cluster training action from the training strategy based on the cluster training state, and obtain the cluster training update state based on the cluster training action.

[0142] Step 702: Based on the cluster reward model, the cluster training update state, the cluster training state, and the cluster training action, calculate the training loss value of the cluster training action.

[0143] Step 703: Obtain a fitting function of the graph convolutional neural network, and update the initial graph convolutional network parameters based on the training loss value and the fitting function.

[0144] Step 704: Update the training strategy based on the updated graph convolutional neural network.

[0145] Steps 701 to 704 are described in detail below.

[0146] In some embodiments, based on the above analysis, the graph neural network (GCN) is combined with the deep reinforcement learning method, and the internal parameters of the graph neural network are trained through multiple parameter iterations to obtain the transmission control strategy π of each network cluster. m . Reference Figure 8 , is a schematic diagram of the training process of an initial collaborative load balancing optimization model provided by an embodiment of the present application. The training process framework is mainly divided into two parts: online scheduling process and offline training process. The online scheduling process collects the link status in the network cluster system through perception, and then combines the link status and link connection matrix into a graph network state as the input of the scheduler. Transmission control strategy π m According to the input information, the transmission rate control action of the corresponding communication link is output to control the flow transmission in the network cluster system. The offline training process mainly uses the latest experience data of the network cluster system to carry out the transmission control strategy π m Training is performed. Formatted empirical data is generated based on link state perception and then centrally stored. Two neural networks in the trainer perform action predictions and update the neural network policy parameters based on the loss function. Finally, the trainer updates the updated neural network policy parameters to the scheduler for a new round of scheduling control. Details are described below.

[0147] Reference Figure 9 , is a schematic diagram of a training algorithm for an initial collaborative load balancing optimization model provided in an embodiment of the present application. Figure 9 In the algorithm shown, the link connection matrix E is first input. The network and framework components are initialized, and network perception and transmission control based on INT are performed (i.e., step 1 in Algorithm 1). Then, the graph convolutional neural network scheduling strategy parameters are randomly initialized. (i.e., step 2 in Algorithm 1). Next, perform network perception and control, collect state information, and perform strategy training and update (i.e., steps 3 to 11 in Algorithm 1). Specifically, obtain the network cluster k of the current iteration. m Cluster training state S m (t) (i.e., step 4 in Algorithm 1) and the training strategy π m , then based on the cluster training state S m (t) Determine the cluster training action a from the training strategy m (t) (i.e., step 5 in Algorithm 1), and obtain the cluster training update state S based on the cluster training action m(t+1), and calculate the training loss value of the cluster training action through the above formula (16), and form the test array {S m (t),a m (t),R loss ,S m (t+1)} (i.e., steps 6 to 9 in Algorithm 1). Next, m pieces of data are randomly selected from the experience pool, and the graph convolutional neural network gradient solution and strategy update are performed according to the above fitting function (15) and the training loss value (line 10 in Algorithm 1). When the ST round strategy update or loss value R loss <ω, output strategy

[0148] The overall overhead of Algorithm 1 includes the storage of empirical values ​​and the offline policy training process, which increases as the policy update interval decreases. Therefore, during the experimental phase, the offline training phase was run on a separate server. The generation and transmission of other perception and policy data packets is primarily related to the policy update frequency and is relatively inexpensive. Furthermore, the algorithm's multi-path coordinated transmission can cause data out-of-order at the receiving end, resulting in additional data storage and packet out-of-order processing overhead.

[0149] Through the above steps 601 to 604, and steps 701 to 704, based on the reinforcement learning framework, the link state and link connection matrix in the load balancing optimization model generated after mathematical modeling of the network cluster system, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the graph convolutional neural network are used to construct an initial collaborative load balancing optimization model, and the reinforcement learning framework is used to train the initial collaborative load balancing optimization model to obtain a suitable collaborative load balancing optimization model, thereby utilizing the "barrel" short board idea to realize the joint load capacity analysis of multiple nodes through clustering of adjacent nodes / links, and then generating a strategy for transmission control to effectively improve the transmission resource utilization during information transmission within each network cluster in the network cluster system.

[0150] Step 303: Generate a network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model.

[0151] Step 303 is described in detail below.

[0152] In some embodiments, after obtaining the collaborative load balancing optimization model of a single network cluster, it is necessary to further consider the collaborative load balancing problem of multiple network clusters in the global network cluster system. When a network cluster m accesses any network node and needs to send large-scale data traffic to any network node of another network cluster g, multiple adjacent network clusters are selected to form a collaborative transmission group based on the connectivity E of each network cluster. And perform corresponding flow distribution ratio control Realize cooperative transmission group The global load balancing goal is achieved. Based on this, it is necessary to develop a collaborative load balancing optimization model for multiple network clusters and further generate a network cluster scheduling model for collaborative optimization of multiple network clusters in the network cluster system. This can improve transmission resource utilization when transmitting information in large-scale network clusters. The following will further describe how to generate the network cluster scheduling model.

[0153] Reference Figure 10 , generating a network cluster scheduling model of a network cluster system based on a collaborative load balancing optimization model, including the following steps 1001 to 1004.

[0154] Step 1001: Based on the collaborative load balancing optimization model of each network cluster, an optimized cluster training action of each network cluster is obtained.

[0155] Step 1002: Based on all optimized cluster training actions in each network cluster, obtain the training traffic allocation ratio of each network cluster.

[0156] Step 1003: Obtain an update status of each network cluster based on the training traffic allocation ratio, and update the training traffic allocation ratio based on the update status.

[0157] Step 1004: Obtain a network cluster scheduling model based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0158] Steps 1001 to 1004 are described in detail below.

[0159] In some embodiments, multiple network cluster systems in a network cluster system coordinate transmission groups To solve the load balancing problem, it is necessary to optimize the transmission rate strategy of a single network cluster and the traffic distribution of multiple network clusters at the same time. Figure 11 , is a schematic diagram of the generation process of a network cluster scheduling model provided in an embodiment of the present application.

[0160] like Figure 11As shown in , first, due to the unbalanced deployment of network components, the number of network nodes and connectivity of each network cluster in the original network cluster system are mostly inconsistent, that is, it is necessary to supplement the most complete graph structure of all original groups and format them into the same network structure, that is, the number of graph nodes and connectivity are consistent. At this time, execute Algorithm 2 to perform load balancing training for each single cluster group. In order to achieve collaborative load balancing among multiple cluster groups, a distributed asynchronous learning method is introduced to share and learn parameters for the same formatted graph structure. Finally, according to the transmission requirements of the sending and receiving ends of the traffic, the graph structure is formatted into a multi-branch tree structure, and according to the strategy of each network cluster Predict the transmission rate weight of each link, and introduce multi-branch tree and simulated annealing ideas to dynamically adjust the traffic distribution ratio of each group Realize dynamic adaptive scheduling of traffic and network resources.

[0161] Reference Figure 12 , is a schematic diagram of a generation algorithm of a network cluster scheduling model provided by an embodiment of the present application. Based on the above model and problem transformation, a multi-cluster collaborative load balancing scheduling algorithm based on distributed asynchronous learning is proposed, such as Figure 12 The algorithm mainly consists of two parts: the calculation of the collaborative traffic distribution ratio between network clusters (steps 3 to 12 in Algorithm 2) and the asynchronous learning of the group output strategy (steps 13 to 17 in Algorithm 2). First, initialize the components of the framework and perform network perception and control based on the INT framework (step 1 in Algorithm 2). Then, call Algorithm 1, which is the collaborative load balancing optimization model based on each network cluster, to obtain the optimized cluster training action a for each network cluster. m (t) and execute (step 3 in Algorithm 2). Then, according to the traffic sending and receiving end addresses, each network cluster is multi-tree path planned, and based on all the optimized cluster training actions in each network cluster, the cumulative transmission rate a m (t) Calculate the root node weight of the network cluster, that is, the training traffic allocation ratio w m (Line 3 in Algorithm 2). Then the root node weights w of all network clusters are m Normalization is performed and traffic distribution scheduling is performed (lines 5 to 6 in Algorithm 2) to obtain the updated state of each network cluster. Then, the training traffic distribution ratio of each network cluster is updated based on the updated state (lines 7 to 17 in Algorithm 2), as described below.

[0162] Reference Figure 13 , updating the training traffic allocation ratio based on the updated status, including the following steps 1301 to 1304.

[0163] Step 1301: Select a network cluster whose target load balancing optimization function update value is negative and whose packet loss rate update value is positive as the first target adjustment cluster, and update the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate.

[0164] Step 1302: Based on the updated training traffic distribution ratio, obtain an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value.

[0165] Step 1303: When the updated target load balancing optimization function update value is negative and the updated packet loss rate update value is positive, select the network cluster whose target load balancing optimization function update value is negative and whose packet loss rate update value is positive as the second target adjustment cluster.

[0166] Step 1304: Select the network cluster with the largest updated target load balancing optimization function value as the benchmark cluster, and update the training traffic allocation ratio of the second target adjustment cluster based on the training traffic allocation ratio of the benchmark cluster.

[0167] Steps 1301 to 1304 are described in detail below.

[0168] In some embodiments, reference Figure 12 As shown in Algorithm 2 in

[15] , after executing traffic distribution scheduling to obtain the updated state of each network cluster, the updated state of each network cluster includes the updated value of the target load balancing optimization function ξ e1 (t+1)-ξ e1 (t) and packet loss rate update value D m (t+1)-D m (t), detect the network status of each group and perform performance analysis before and after traffic execution, define the overall throughput and bandwidth resource utilization of the network cluster as joint objectives, when the performance of time slot t is less than the performance of time slot t+1 (ξ e1 (t)<ξ e1 (t+1), that is, the updated value of the target load balancing optimization function ξ e1 (t+1)-ξ e1 (t) is a negative number), and the total packet loss in time slot t+1 increases (D m (t) <D m (t+1), which is the updated value of packet loss rate D m (t+1)-D m (t) is a negative number), the network cluster k is determined m In the case of traffic overload, the network cluster k m As the first target adjustment cluster, the first target adjustment cluster k is then adjusted by a ratio of one minus the traffic degradation rate ζ. mThe training traffic distribution ratio is adjusted (lines 7 to 11 in Algorithm 2). After the ratio adjustment is completed, the traffic distribution ratio is adjusted again (step 12 in Algorithm 2). At this time, all network clusters are traversed again, that is, based on the updated training traffic distribution ratio, the updated target load balancing optimization function update value, the updated packet loss rate update value and the updated target load balancing optimization function value are obtained again. When the overall performance weakens and the packet loss increases (ξ e1 (t)<ξ e1 (t+1) and D m (t) <D m (t+1), that is, the updated value of the target load balancing optimization function ξ e1 (t+1)-ξ e1 (t) is negative and the packet loss rate is updated to D m (t+1)-D m (t) is a negative number), determine the current group k m The strategy has low performance and the network is clustered into k m As the second target adjustment cluster, select the target load balancing optimization function value ξ from the multiple network clusters at this time e1 (t) The best network cluster k j As the benchmark cluster, then based on the training traffic allocation ratio of the benchmark cluster, the second target is updated with probability ε to adjust the training traffic allocation ratio θ of the cluster m =εθ m +(1-ε)θ j . Finally output the adjusted flow ratio The overhead of Algorithm 2 is mainly the parameter sharing additive update of the adjacent strategies, which has a small overhead.

[0169] Through the above steps 1001 to 1004, and steps 1301 to 1304, the collaborative load balancing optimization model of each network cluster is integrated, and the obtained optimized cluster training action is used to plan the training traffic allocation ratio of each network cluster, and then the training traffic allocation ratio of the network cluster is gradually updated based on different update states to obtain a suitable network cluster scheduling model for the network cluster system. In actual applications, in complex large-scale networks, based on the traffic states and link connection matrices of different network clusters, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio can be generated for each network cluster, so as to improve the transmission resource utilization when information is transmitted in large-scale network clusters.

[0170] Step 203: adjusting the transmission rate of each communication link in the corresponding network cluster according to the optimized transmission rate sequence of each link, and adjusting the traffic distribution of the corresponding network cluster according to each optimized traffic distribution ratio.

[0171] Step 203 is described in detail below.

[0172] In some embodiments, after obtaining the optimal link optimized transmission rate sequence and optimized traffic distribution ratio for each network cluster based on the link status and link connection matrix of all communication links in the current network cluster system based on a network cluster scheduling model, the transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic distribution of the corresponding network cluster is adjusted according to each optimized traffic distribution ratio. After the parameter adjustments, data transmission is then performed on each network node within the network cluster system to improve transmission resource utilization.

[0173] The transmission optimization method and related equipment of a network cluster system proposed in an embodiment of the present application, the network cluster system includes multiple network clusters, and the network cluster includes multiple network nodes, the method includes: first, obtaining the link status of the communication link between each two network nodes, and obtaining the link connection matrix, the link connection matrix is ​​used to characterize the connection status of each communication link; then, obtaining the transmission rate parameter and maximum bandwidth of each communication link, and obtaining the transmission rate parameter range constraint of the transmission rate parameter, based on the cumulative sum of multiple transmission rate parameters, obtaining the throughput model of each network cluster, obtaining the ratio of multiple transmission rate parameters to the maximum bandwidth, and accumulating the comparison values ​​to obtain the energy utilization model of each network cluster, accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function, based on the transmission rate parameter, the target load balancing optimization function and the transmission rate parameter range constraint, generating a load balancing optimization model, based on the link connection matrix and the link bandwidth parameter, link delay parameter, queuing packet loss parameter and queue depth generation parameter of each communication link to generate a cluster. state space, generate a cluster action space based on the transmission rate parameter, and generate a cluster reward model based on the target load balancing optimization function, obtain the initial graph convolutional network parameters of the graph convolutional neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model and initial graph convolutional network parameters, perform multiple rounds of training on the initial collaborative load balancing optimization model, and obtain the collaborative load balancing optimization model of the network cluster based on the trained initial collaborative load balancing optimization model. The training process of the initial collaborative load balancing optimization model includes: obtaining the cluster training state and training strategy of the current iteration, determining the cluster training action from the training strategy based on the cluster training state, and obtaining the cluster training update state based on the cluster training action, calculating the training loss value of the cluster training action based on the cluster reward model, cluster training update state, cluster training state and cluster training action, obtaining the fitting function of the graph convolutional neural network, and updating the initial graph convolutional network parameters based on the training loss value and the fitting function, and updating the training strategy based on the updated graph convolutional neural network;And, based on the collaborative load balancing optimization model of each network cluster, the optimization cluster training action of each network cluster is obtained, based on all the optimization cluster training actions in each network cluster, the training traffic distribution ratio of each network cluster is obtained, the network cluster with a negative target load balancing optimization function update value and a positive packet loss rate update value is selected as the first target adjustment cluster, the training traffic distribution ratio of the first target adjustment cluster is updated based on a minus traffic degradation rate, based on the updated training traffic distribution ratio, the updated target load balancing optimization function update value, the updated packet loss rate update value and the updated target load balancing optimization function value are obtained, when the updated target load balancing optimization function update value is negative and the updated packet loss rate update value is positive, the target load balancing optimization function update value is selected. The network cluster with a negative value and a positive packet loss rate update value is selected as the second target adjustment cluster. The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster. The training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster. The network cluster scheduling model is derived based on the collaborative load balancing optimization model of the network cluster and the updated training traffic allocation ratio. All link states and link connection matrices are input into the network cluster scheduling model to obtain the link optimized transmission rate sequence and optimized traffic allocation ratio for each network cluster. Finally, the transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic allocation of the corresponding network cluster is adjusted according to each optimized traffic allocation ratio.

[0174] The embodiment of the present application utilizes the link status of the changing communication links in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicationally connected, and combines it with the network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, so as to obtain the link optimization transmission rate sequence and optimized traffic distribution ratio of multiple network clusters that match the link status and link connection matrix appropriately, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, thereby effectively improving the transmission resource utilization when information is transmitted between network nodes in the network cluster system; and, utilizing the relevant mathematical modeling parameters in the network cluster system, so as to adjust the transmission rate of the communication link between the network nodes in each network cluster to achieve the purpose of improving the throughput and bandwidth resource utilization in the network cluster, and construct a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model, and the collaborative load balancing optimization model can be effectively used to improve the resource utilization in the network cluster system; in addition, based on the reinforcement learning framework, the load balancing optimization model generated after mathematical modeling of the network cluster system is utilized. The link state and link connection matrix, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the graph convolutional neural network are combined to construct an initial collaborative load balancing optimization model, and the reinforcement learning framework is used to train the initial collaborative load balancing optimization model to obtain a suitable collaborative load balancing optimization model. The "barrel" short board idea is used to cluster adjacent nodes / links to realize the joint load capacity analysis of multiple nodes, and then a strategy is generated for transmission control to effectively improve the transmission resource utilization of each network cluster during information transmission in the network cluster system. In addition, by integrating the collaborative load balancing optimization model of each network cluster and using the obtained optimized cluster training action to plan the training traffic allocation ratio of each network cluster, and then gradually updating the training traffic allocation ratio of the network cluster based on different update states, a suitable network cluster scheduling model for the network cluster system is obtained. In practical applications, based on the traffic state and link connection matrix of different network clusters in complex large-scale networks, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster can be generated, so as to improve the transmission resource utilization when large-scale network clusters perform information transmission.

[0175] The embodiment of the present application also provides a transmission optimization device for a network cluster system, which can implement the transmission optimization method of the network cluster system. Figure 14 , the apparatus 1400 comprises:

[0176] A state acquisition module 1410 is configured to acquire the link state of a communication link between each two network nodes and to acquire a link connection matrix, wherein the link connection matrix is ​​used to represent the connection state of each communication link;

[0177] Optimization parameter acquisition module 1420 is used to obtain a network cluster scheduling model, input all link states and link connection matrices into the network cluster scheduling model, and output a link optimized transmission rate sequence and optimized traffic distribution ratio for each network cluster. The link optimized transmission rate sequence and optimized traffic distribution ratio are used to optimize the throughput and resource utilization within the network cluster system.

[0178] a parameter adjustment module 1430 for adjusting the transmission rate of each communication link in the corresponding network cluster one by one according to the optimized transmission rate sequence of each link, and adjusting the traffic distribution of the corresponding network cluster one by one according to each optimized traffic distribution ratio;

[0179] The information transmission module 1440 is configured to transmit information between at least two network nodes after the transmission rate of at least one communication link and the traffic distribution of at least one network cluster are adjusted.

[0180] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0181] Obtain the load balancing optimization model for each network cluster;

[0182] Generate a collaborative load balancing optimization model for network clusters based on the load balancing optimization model;

[0183] A network cluster scheduling model for a network cluster system is generated based on a collaborative load balancing optimization model.

[0184] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0185] Obtaining a transmission rate parameter and a maximum bandwidth of each communication link, and obtaining a transmission rate parameter range constraint of the transmission rate parameter;

[0186] Based on the cumulative sum of multiple transmission rate parameters, a throughput model of each network cluster is obtained;

[0187] Obtain the ratios of multiple transmission rate parameters and maximum bandwidth, and accumulate the ratios to obtain the energy utilization model of each network cluster;

[0188] The throughput model and energy utilization model are accumulated to generate the target load balancing optimization function;

[0189] A load balancing optimization model is generated based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint.

[0190] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0191] Generate a cluster state space based on the link connection matrix and the link bandwidth parameter, link delay parameter, queue packet loss parameter and queue depth generation parameter of each communication link;

[0192] Generate a cluster action space based on the transmission rate parameter, and generate a cluster reward model based on the target load balancing optimization function;

[0193] Obtain the initial graph convolutional network parameters of the graph convolutional neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model, and the initial graph convolutional network parameters;

[0194] The initial collaborative load balancing optimization model is trained for multiple rounds, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

[0195] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0196] Obtain the cluster training state and training strategy of the current iteration, determine the cluster training action from the training strategy based on the cluster training state, and obtain the cluster training update state based on the cluster training action;

[0197] Based on the cluster reward model, cluster training update state, cluster training state, and cluster training action, the training loss value of the cluster training action is calculated;

[0198] Get the fitting function of the graph convolutional neural network and update the initial graph convolutional network parameters based on the training loss value and the fitting function;

[0199] Update the training strategy based on the updated graph convolutional neural network.

[0200] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0201] Based on the collaborative load balancing optimization model of each network cluster, the optimized cluster training action of each network cluster is obtained;

[0202] Based on all the optimized cluster training actions in each network cluster, the training traffic allocation ratio of each network cluster is obtained;

[0203] Obtain the updated status of each network cluster based on the training traffic allocation ratio, and update the training traffic allocation ratio based on the updated status;

[0204] The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0205] In some embodiments, the optimization parameter acquisition module 1420 is further configured to:

[0206] Selecting a network cluster whose target load balancing optimization function update value is negative and whose packet loss rate update value is positive as the first target adjustment cluster, and updating the training traffic distribution ratio of the first target adjustment cluster based on a minus traffic degradation rate;

[0207] Based on the updated training traffic distribution ratio, the updated target load balancing optimization function update value, the updated packet loss rate update value and the updated target load balancing optimization function value are obtained;

[0208] When the updated value of the target load balancing optimization function is negative and the updated value of the packet loss rate is positive, the network cluster with the negative updated value of the target load balancing optimization function and the positive updated value of the packet loss rate is selected as the second target adjustment cluster;

[0209] The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

[0210] In the above embodiments, the description of each embodiment has different emphases. For the parts not described in detail in a certain embodiment, the specific implementation of the transmission optimization device of the network cluster system is basically the same as the specific implementation of the transmission optimization method of the above network cluster system, and will not be repeated here.

[0211] In an embodiment of the present application, a transmission optimization device of a network cluster system utilizes the link state of a changing communication link in the network cluster system and a link connection matrix for characterizing whether network nodes are connected for communication, and constructs a network cluster scheduling model based on a communication process performed by network nodes in the network cluster in advance, thereby obtaining a link optimization transmission rate sequence and an optimized traffic distribution ratio of multiple network clusters that match the link state and the link connection matrix, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, thereby effectively improving the transmission resource utilization rate when information is transmitted between network nodes in the network cluster system; and, utilizing relevant mathematical modeling parameters in the network cluster system, so as to adjust the transmission rate of the communication link between network nodes in each network cluster to achieve the purpose of improving the throughput and bandwidth resource utilization rate within the network cluster, and construct a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model, and the collaborative load balancing optimization model can be effectively used to improve the resource utilization rate in the network cluster system; in addition, based on the reinforcement learning framework, the load balancing optimization model generated by mathematical modeling of the network cluster system is utilized. The link state and link connection matrix in the load balancing optimization model, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the graph convolutional neural network are combined to construct an initial collaborative load balancing optimization model. The initial collaborative load balancing optimization model is trained using a reinforcement learning framework to obtain a suitable collaborative load balancing optimization model. The "barrel" short board idea is used to cluster adjacent nodes / links to realize the joint load capacity analysis of multiple nodes. Then, a strategy is generated for transmission control to effectively improve the transmission resource utilization during information transmission within each network cluster in the network cluster system. In addition, by integrating the collaborative load balancing optimization model of each network cluster and using the obtained optimized cluster training action to plan the training traffic allocation ratio of each network cluster, and then gradually updating the training traffic allocation ratio of the network cluster based on different update states, a suitable network cluster scheduling model for the network cluster system is obtained. In practical applications, based on the traffic state and link connection matrix of different network clusters in complex large-scale networks, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster can be generated to improve the transmission resource utilization during information transmission in large-scale network clusters.

[0212] An embodiment of the present application further provides an electronic device, including:

[0213] at least one memory;

[0214] at least one processor;

[0215] at least one program;

[0216] The program is stored in the memory, and the processor executes the at least one program to implement the transmission optimization method of the network cluster system implemented in the present application. The electronic device can be any smart terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), an in-vehicle computer, etc.

[0217] See also Figure 15 , Figure 15 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0218] The processor 1501 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0219] The memory 1502 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device, or RAM (Random Access Memory). The memory 1502 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1502, and the processor 1501 calls and executes the transmission optimization method of the network cluster system of the embodiment of this application;

[0220] Input / output interface 1503, used to implement information input and output;

[0221] Communication interface 1504, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0222] Bus 1505 , which transmits information between various components of the device (e.g., processor 1501 , memory 1502 , input / output interface 1503 , and communication interface 1504 );

[0223] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0224] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the transmission optimization method of the network cluster system described above is implemented.

[0225] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0226] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0227] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0228] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0229] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0230] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0236] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A transmission optimization method for a network cluster system, characterized in that: The network cluster system includes multiple network clusters, each of which includes multiple network nodes. The method includes: Acquire a link state of a communication link between every two of the network nodes, and acquire a link connection matrix, wherein the link connection matrix is ​​used to represent the connection state of each of the communication links; Obtaining a network cluster scheduling model, inputting all the link states and the link connection matrix into the network cluster scheduling model, and outputting a link optimized transmission rate sequence and an optimized traffic distribution ratio for each network cluster; Adjusting the transmission rate of each communication link in the corresponding network cluster according to each link optimized transmission rate sequence, and adjusting the traffic distribution of the corresponding network cluster according to each optimized traffic distribution ratio; The obtaining of the network cluster scheduling model includes: Obtaining a load balancing optimization model for each of the network clusters; Generating a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model; Generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model; The obtaining of the load balancing optimization model of each network cluster includes: Obtaining a transmission rate parameter and a maximum bandwidth of each of the communication links, and obtaining a transmission rate parameter range constraint of the transmission rate parameter; Obtaining a throughput model for each of the network clusters based on a cumulative sum of the plurality of transmission rate parameters; Obtaining ratios of a plurality of the transmission rate parameters to the maximum bandwidth, and accumulating the ratios to obtain an energy utilization model for each of the network clusters; Accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function; generating the load balancing optimization model based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint; Generating the collaborative load balancing optimization model of the network cluster based on the load balancing optimization model includes: Generate a cluster state space based on the link connection matrix and a link bandwidth parameter, a link delay parameter, a queue packet loss parameter, and a queue depth generation parameter of each of the communication links; generating a cluster action space based on the transmission rate parameter, and generating a cluster reward model based on the target load balancing optimization function; Obtaining initial graph convolutional network parameters of a graph convolutional neural network, and generating an initial collaborative load balancing optimization model based on the cluster state space, the cluster action space, the cluster reward model, and the initial graph convolutional network parameters; The initial collaborative load balancing optimization model is trained for multiple rounds, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

2. The transmission optimization method of the network cluster system according to claim 1, characterized in that: The training process of the initial collaborative load balancing optimization model includes: Obtaining a cluster training state and a training strategy for a current iteration, determining a cluster training action from the training strategy based on the cluster training state, and obtaining a cluster training update state based on the cluster training action; Calculating a training loss value of the cluster training action based on the cluster reward model, the cluster training update state, the cluster training state, and the cluster training action; Obtaining a fitting function of the graph convolutional neural network, and updating the initial graph convolutional network parameters based on the training loss value and the fitting function; The training strategy is updated based on the updated graph convolutional neural network.

3. The transmission optimization method of a network cluster system according to claim 1, characterized in that: Generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model includes: Based on the collaborative load balancing optimization model of each network cluster, obtaining an optimized cluster training action for each network cluster; Based on all the optimized cluster training actions in each of the network clusters, obtaining a training traffic allocation ratio for each network cluster; Obtaining an update status of each of the network clusters based on the training traffic allocation ratio, and updating the training traffic allocation ratio based on the update status; The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

4. The transmission optimization method of the network cluster system according to claim 3, characterized in that: The updated state includes an updated value of a target load balancing optimization function and an updated value of a packet loss rate, and updating the training traffic distribution ratio based on the updated state includes: Selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a first target adjustment cluster, and updating the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate; Based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value are obtained; When the updated target load balancing optimization function update value is a negative number and the updated packet loss rate update value is a positive number, selecting the network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as the second target adjustment cluster; The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

5. A transmission optimization device for a network cluster system, characterized in that: The network cluster system includes multiple network clusters, each of which includes multiple network nodes. The device includes: a state acquisition module, configured to acquire the link state of the communication link between each two network nodes, and to acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize the connection state of each communication link; an optimization parameter acquisition module, configured to acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and output a link optimized transmission rate sequence and an optimized traffic distribution ratio for each network cluster, wherein the link optimized transmission rate sequence and the optimized traffic distribution ratio are used to optimize the throughput and resource utilization within the network cluster system; a parameter adjustment module, configured to adjust the transmission rate of each communication link in the corresponding network cluster one by one according to the optimized transmission rate sequence of each link, and to adjust the traffic distribution of the corresponding network cluster one by one according to each optimized traffic distribution ratio; an information transmission module, configured to transmit information between at least two of the network nodes after the transmission rate of at least one of the communication links and the traffic distribution of at least one of the network clusters are adjusted; The obtaining of the network cluster scheduling model includes: Obtaining a load balancing optimization model for each of the network clusters; Generating a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model; Generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model; The obtaining of the load balancing optimization model of each network cluster includes: Obtaining a transmission rate parameter and a maximum bandwidth of each of the communication links, and obtaining a transmission rate parameter range constraint of the transmission rate parameter; Obtaining a throughput model for each of the network clusters based on a cumulative sum of the plurality of transmission rate parameters; Obtaining ratios of a plurality of the transmission rate parameters to the maximum bandwidth, and accumulating the ratios to obtain an energy utilization model for each of the network clusters; Accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function; generating the load balancing optimization model based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint; Generating the collaborative load balancing optimization model of the network cluster based on the load balancing optimization model includes: Generate a cluster state space based on the link connection matrix and a link bandwidth parameter, a link delay parameter, a queue packet loss parameter, and a queue depth generation parameter of each of the communication links; generating a cluster action space based on the transmission rate parameter, and generating a cluster reward model based on the target load balancing optimization function; Obtaining initial graph convolutional network parameters of a graph convolutional neural network, and generating an initial collaborative load balancing optimization model based on the cluster state space, the cluster action space, the cluster reward model, and the initial graph convolutional network parameters; The initial collaborative load balancing optimization model is trained for multiple rounds, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the transmission optimization method for the network cluster system according to any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transmission optimization method of the network cluster system according to any one of claims 1 to 4 is implemented.

Images