Transmission optimization method of network cluster system and related equipment

Abstract

According to the transmission optimization method of the network cluster system and the related equipment, the network cluster system comprises a plurality of network clusters, each network cluster comprises a plurality of network nodes, and the method comprises the steps that firstly, the link state of a communication link between every two network nodes is acquired, and a link connection matrix is acquired; the link connection matrix is used for representing the connection state of each communication link; then, obtaining a network cluster scheduling model, inputting all link states and the link connection matrix into the network cluster scheduling model, and obtaining an output link optimization transmission rate sequence and an optimization flow distribution proportion of each network cluster; and finally, the transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the flow distribution of the corresponding network cluster is adjusted according to each optimized flow distribution proportion, so that the utilization rate of transmission resources can be effectively improved when information transmission between network nodes is performed in the network cluster system.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a transmission optimization method and related equipment for a network cluster system. Background Art

[0002] With the rapid growth of modern computing needs, large-scale network clusters have been built more and more perfectly. With the continuous improvement of hardware level, large-scale networks with multi-network integration and heterogeneous coexistence are also a mainstream development trend. However, since the number of network nodes in large-scale networks is usually large, it is a relatively complex problem to perform appropriate resource scheduling in large-scale networks.

[0003] In the related art, suitable communication link parameters are usually pre-configured between each two nodes in order to improve the reliability of data transmission according to the communication transmission data requirements between each two nodes in a large-scale network cluster. However, due to the high dynamic changes of the communication link between each two nodes in a large-scale network cluster, after using this resource configuration method, when information is transmitted in a large-scale network cluster, there is a situation where the transmission resource utilization rate is low. Summary of the invention

[0004] The embodiments of the present application provide a transmission optimization method and related equipment for a network cluster system, which can improve the utilization rate of transmission resources when a large-scale network cluster performs information transmission.

[0005] To achieve the above object, a first aspect of an embodiment of the present application proposes a transmission optimization method for a network cluster system, wherein the network cluster system includes a plurality of network clusters, and the network cluster includes a plurality of network nodes. The method includes:

[0006] Acquire a link state of a communication link between every two of the network nodes, and acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize a connection state of each of the communication links;

[0007] Acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and obtain an output link optimized transmission rate sequence and optimized traffic distribution ratio of each network cluster;

[0008] The transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic distribution of the corresponding network cluster is adjusted according to each optimized traffic distribution ratio.

[0009] In some embodiments, obtaining the network cluster scheduling model includes:

[0010] Obtaining a load balancing optimization model for each of the network clusters;

[0011] Generate a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model;

[0012] The network cluster scheduling model of the network cluster system is generated based on the collaborative load balancing optimization model.

[0013] In some embodiments, obtaining the load balancing optimization model of each of the network clusters includes:

[0014] Acquire a transmission rate parameter and a maximum bandwidth of each of the communication links, and acquire a transmission rate parameter range constraint of the transmission rate parameter;

[0015] Based on the cumulative sum of the plurality of transmission rate parameters, a throughput model of each of the network clusters is obtained;

[0016] Acquire a plurality of ratios of the transmission rate parameters to the maximum bandwidth, and accumulate the ratios to obtain an energy utilization model of each of the network clusters;

[0017] Accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function;

[0018] The load balancing optimization model is generated based on the transmission rate parameter, the target load balancing optimization function and the transmission rate parameter range constraint.

[0019] In some embodiments, generating the collaborative load balancing optimization model of the network cluster based on the load balancing optimization model includes:

[0020] Generate a cluster state space based on the link connection matrix and a link bandwidth parameter, a link delay parameter, a queue packet loss parameter, and a queue depth generation parameter of each of the communication links;

[0021] generating a cluster action space based on the transmission rate parameter, and generating a cluster reward model based on the target load balancing optimization function;

[0022] Obtaining initial graph convolutional network parameters of a graph convolutional neural network, and generating an initial collaborative load balancing optimization model based on the cluster state space, the cluster action space, the cluster reward model, and the initial graph convolutional network parameters;

[0023] The initial collaborative load balancing optimization model is trained multiple times, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

[0024] In some embodiments, the training process of the initial collaborative load balancing optimization model includes:

[0025] Acquire a cluster training state and a training strategy of a current iteration, determine a cluster training action from the training strategy based on the cluster training state, and obtain a cluster training update state based on the cluster training action;

[0026] Based on the cluster reward model, the cluster training update state, the cluster training state and the cluster training action, a training loss value of the cluster training action is calculated;

[0027] Obtaining a fitting function of the graph convolutional neural network, and updating the initial graph convolutional network parameters based on the training loss value and the fitting function;

[0028] The training strategy is updated based on the updated graph convolutional neural network.

[0029] In some embodiments, generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model includes:

[0030] Based on the collaborative load balancing optimization model of each of the network clusters, an optimized cluster training action of each of the network clusters is obtained;

[0031] Based on all the optimized cluster training actions in each of the network clusters, obtaining a training traffic allocation ratio for each network cluster;

[0032] Obtaining an update status of each of the network clusters based on the training traffic allocation ratio, and updating the training traffic allocation ratio based on the update status;

[0033] The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0034] In some embodiments, the updated state includes an updated value of a target load balancing optimization function and an updated value of a packet loss rate, and updating the training traffic distribution ratio based on the updated state includes:

[0035] Selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a first target adjustment cluster, and updating the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate;

[0036] Based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value are obtained;

[0037] When the updated target load balancing optimization function update value is a negative number, and the updated packet loss rate update value is a positive number, selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a second target adjustment cluster;

[0038] The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

[0039] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application proposes a transmission optimization device for a network cluster system, wherein the network cluster system includes a plurality of network clusters, the network cluster includes a plurality of network nodes, and the device includes:

[0040] A state acquisition module, used to acquire the link state of the communication link between every two of the network nodes, and to acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize the connection state of each of the communication links;

[0041] An optimization parameter acquisition module is used to acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and obtain an output link optimized transmission rate sequence and an optimized traffic allocation ratio for each network cluster, wherein the link optimized transmission rate sequence and the optimized traffic allocation ratio are used to optimize the throughput and resource utilization within the network cluster system;

[0042] A parameter adjustment module, used to adjust the transmission rate of each communication link in the corresponding network cluster one by one according to each link optimization transmission rate sequence, and to adjust the flow distribution of the corresponding network cluster one by one according to each optimized flow distribution ratio;

[0043] The information transmission module is used to transmit information in at least two of the network nodes after the transmission rate of at least one of the communication links and the flow distribution of at least one of the network clusters are adjusted.

[0044] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, it implements the transmission optimization method of the network cluster system as described in the first aspect.

[0045] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a storage medium, which is a computer-readable storage medium, and the storage medium stores a computer program. When the computer program is executed by a processor, the transmission optimization method of the network cluster system described in the first aspect above is implemented.

[0046] The transmission optimization method and related equipment of a network cluster system proposed in an embodiment of the present application, the network cluster system includes multiple network clusters, and the network cluster includes multiple network nodes. The method includes: first, obtaining the link status of the communication link between every two network nodes, and obtaining the link connection matrix, the link connection matrix is ​​used to characterize the connection status of each communication link; then, obtaining the network cluster scheduling model, inputting all link statuses and link connection matrices into the network cluster scheduling model, and obtaining the output link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster; finally, adjusting the transmission rate of each communication link in the corresponding network cluster according to each link optimized transmission rate sequence, and adjusting the traffic allocation of the corresponding network cluster according to each optimized traffic allocation ratio. The embodiments of the present application utilize the link status of the changing communication links in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicatively connected, and combine it with a network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, so as to obtain link optimized transmission rate sequences and optimized traffic allocation ratios of multiple network clusters that are appropriately matched with the link status and the link connection matrix, so as to adjust the transmission rates of multiple network nodes in each network cluster and the traffic allocation of each network cluster, and thus effectively improve the utilization of transmission resources when transmitting information between network nodes in the network cluster system.

[0047] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a structural diagram of a network cluster system provided in one embodiment of the present application.

[0049] Figure 2 This is a flowchart of a transmission optimization method for a network cluster system provided in yet another embodiment of the present application.

[0050] Figure 3 yes Figure 2 Flow chart of step 202 in FIG.

[0051] Figure 4 yes Figure 3 Flow chart of step 301 in FIG.

[0052] Figure 5 It is a schematic diagram of a multi-network collaborative load balancing scheduling framework provided by another embodiment of the present application.

[0053] Figure 6 yes Figure 3 Flow chart of step 302 in FIG.

[0054] Figure 7 This is a flowchart of generating an initial collaborative load balancing optimization model provided by another embodiment of the present application.

[0055] Figure 8 This is a schematic diagram of the training process of an initial collaborative load balancing optimization model provided by another embodiment of the present application.

[0056] Fig. 9 This is a schematic diagram of a training algorithm for an initial collaborative load balancing optimization model provided in another embodiment of the present application.

[0057] Fig.10 Figure 3 Flow chart of step 303 in FIG.

[0058] Fig.11 This is a schematic diagram of a generation process of a network cluster scheduling model provided by another embodiment of the present application.

[0059] Fig.12 This is a schematic diagram of an algorithm for generating a network cluster scheduling model provided in yet another embodiment of the present application.

[0060] Fig.13 yes Fig.10 Flowchart of step 1003 in FIG.

[0061] Fig.14 It is a structural diagram of a transmission optimization device for a network cluster system provided in one embodiment of the present application.

[0062] Fig.15 It is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] It should be noted that although the functional modules are divided in the device schematic and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0065] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0066] First, some nouns involved in this application are analyzed:

[0067] Deep reinforcement learning is a branch of machine learning that combines the powerful representation learning capabilities of deep learning with the goal-oriented mechanism of reinforcement learning, allowing agents to learn how to make optimal decisions in an uncertain environment. Specifically, deep reinforcement learning uses deep neural networks as function approximators to estimate the value of actions or directly output action strategies, so that agents can learn the optimal path to achieve specific goals through trial and error in the process of interacting with the environment. This approach greatly improves the learning and generalization capabilities of the model, especially when dealing with high-dimensional input data such as images and natural language.

[0068] With the rapid growth of modern computing needs, large-scale network clusters have been built more and more perfectly. With the continuous improvement of hardware level, large-scale networks with multi-network integration and heterogeneous coexistence are also a mainstream development trend. However, since the number of network nodes in large-scale networks is usually large, it is a relatively complex problem to perform appropriate resource scheduling in large-scale networks.

[0069] In the related art, suitable communication link parameters are usually pre-configured between each two nodes in order to improve the reliability of data transmission according to the communication transmission data requirements between each two nodes in a large-scale network cluster. However, due to the high dynamic changes of the communication link between each two nodes in a large-scale network cluster, after using this resource configuration method, when information is transmitted in a large-scale network cluster, there is a situation where the transmission resource utilization rate is low.

[0070] In order to improve the utilization rate of transmission resources when large-scale network clusters transmit information, the embodiments of the present application utilize the link status of the changing communication links in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicationally connected, and combine it with the network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, so as to obtain the link optimization transmission rate sequence and optimized traffic distribution ratio of multiple network clusters that match the link status and link connection matrix appropriately, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, so as to effectively improve the utilization rate of transmission resources when information is transmitted between network nodes in the network cluster system.

[0071] In order to better describe the transmission optimization method of the network cluster system provided by the present application, the following first describes the network cluster system applied to the transmission optimization method of the network cluster system. Figure 1 , is a schematic diagram of the structure of a network cluster system provided in an embodiment of the present application. Figure 1As shown in , it includes multiple network clusters, each of which includes multiple network nodes. Among them, every two network nodes can directly communicate with each other, and every two network clusters can also communicate with each other.

[0072] In conjunction with the above-mentioned network cluster system, the transmission optimization method and related devices of the network cluster system provided in the embodiment of the present application will be further described below. The transmission optimization method of the network cluster system provided in the embodiment of the present application can be applied to any network node with computing function in the network cluster system, or a processing controller connected to the network cluster system.

[0073] The following will describe in detail the transmission optimization method of the network cluster system in the embodiment of the present application. Figure 2 , which is an optional flow chart of the transmission optimization method of the network cluster system provided in the embodiment of the present application, Figure 2 The method in the embodiment may include but is not limited to steps 201 to 203. Figure 2 The order of step 201 to step 203 is not specifically limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0074] Step 201: Obtain the link status of the communication link between every two network nodes, and obtain a link connection matrix.

[0075] The following is a detailed description of step 201.

[0076] In some embodiments, in order to improve the utilization rate of transmission resources when large-scale network clusters are transmitting information, it is necessary to study and analyze large-scale network clusters in advance. Figure 1 All network nodes participating in the cooperative transmission are regarded as a fusion network, and a fixed undirected network can be obtained. At this point, the clustering idea is introduced to divide the entire network, and M network clusters can be obtained to form a network cluster set: At this point, define the network cluster k m The link set between every two network nodes in is l (m,j) represents the network cluster k m The jth communication link. m represents the network cluster k m The number of network nodes in the network. (m,p) represents the network cluster k m There are p communication links. It can be understood that any network cluster k m The total number of links p is arbitrary.

[0077] In order to improve the utilization of transmission resources when large-scale network clusters are transmitting information, it is necessary to obtain the link status S of all communication links between every two network nodes in each network cluster before each resource management configuration of the network cluster system. m (t)∈R p×fs , where fs represents the link status including: link bandwidth in the communication link, link transmission delay, packet loss rate when queuing during transmission, and queue length when queuing during transmission. m represents the mth network cluster, and p represents the number of communication links in the network cluster.

[0078] Since in practical applications, usually not every two network nodes need to transmit information in the communication link, it is also necessary to obtain the link connection matrix ε of all network clusters in advance. m,ij ) nm×nm , where l m,ij For any two network nodes n in the mth network cluster (m,i) With n (m,j) Connectivity; nm is equivalent to n in the above m , that is, the same network cluster k m The number of network nodes in the network. (m,ij) =1 indicates that the two network nodes are connected; otherwise l (m,ij) =0, indicating that the link is disconnected.

[0079] Step 202: Obtain a network cluster scheduling model, input all link states and link connection matrices into the network cluster scheduling model, and obtain output link optimized transmission rate sequence and optimized traffic distribution ratio of each network cluster.

[0080] The following is a detailed description of step 202.

[0081] In some embodiments, based on the deep reinforcement learning framework and the network cluster system, a system is constructed in advance that can effectively output the most suitable resource configuration parameters in the network cluster system (i.e., the link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster) for different link states and link connection matrices, so as to further improve the transmission resource utilization when large-scale network clusters transmit information. The following will further describe how to construct the network cluster scheduling model.

[0082] Reference Figure 3 , obtaining a network cluster scheduling model, including the following steps 301 to 303.

[0083] Step 301: Obtain a load balancing optimization model for each network cluster.

[0084] Step 301 is described in detail below.

[0085] In some embodiments, in order to construct a suitable network cluster scheduling model, it is necessary to perform mathematical modeling in advance according to the network cluster system to construct a mathematical optimization model for improving the utilization of transmission resources, so as to facilitate the subsequent use of the mathematical optimization model for reinforcement learning model establishment. The following will further describe how to construct a mathematical optimization model corresponding to the network cluster system.

[0086] Reference Figure 4 , obtaining a load balancing optimization model for each network cluster, including the following steps 401 to 405.

[0087] Step 401: Acquire the transmission rate parameter and maximum bandwidth of each communication link, and acquire the transmission rate parameter range constraint of the transmission rate parameter.

[0088] Step 402: Based on the cumulative sum of multiple transmission rate parameters, a throughput model of each network cluster is obtained.

[0089] Step 403: Obtain ratios of multiple transmission rate parameters and maximum bandwidth, and accumulate the ratios to obtain an energy utilization model for each network cluster.

[0090] Step 404: Accumulate the throughput model and the energy utilization model to generate a target load balancing optimization function.

[0091] Step 405: Generate a load balancing optimization model based on the transmission rate parameter, the target load balancing optimization function, and the transmission rate parameter range constraint.

[0092] Steps 401 to 405 are described in detail below.

[0093] In some embodiments, in order to clearly understand the network node transmission control process in the network cluster system, the traffic scheduling process in the network cluster system is divided into continuous discrete time slots t∈{0,1,2,…}. Define the network cluster k m Communication link within (m,j) The arrival rate, transmission rate, and packet loss rate are α (m,j) (t), b (m,j) (t), d (m,j) (t). After the clustering is performed in the network cluster system, each network cluster k m There will be an internal corresponding transmission control strategy π m Execute transfer action a m (t) Control the transmission rate of each link, that is, Define network cluster k m When all communication links in the t time slot, the link state matrix corresponding to the link state is S m (t)∈Rp×fs Considering the heterogeneity of heterogeneous network node resources in the network cluster system, fs is defined as the maximum state attribute dimension of all links. At this time, the traffic transmission control problem in each network cluster is transformed into: according to the link state S m (t), the transmission control strategy π within the scheduling network cluster m Execute control action a m (t), maximizing the network throughput within the network cluster T m (t) and maximize resource utilization U m (t) Target.

[0094] That is, for each network cluster, first determine the transmission rate parameter (ie, the transmission rate b) of each communication link (eg, the jth communication link) in each network cluster (eg, the mth network cluster). (m,j) (t)) and the maximum bandwidth B m,j , and determine the transmission rate parameter b (m,j) (t) Transmission rate parameter range constraint b (m,j) (t)≤B m,j .

[0095] Next, based on multiple transmission rate parameters b (m,j) The cumulative sum of (t) is used to obtain the throughput model of each network cluster And, obtain the ratio of multiple transmission rate parameters and maximum bandwidth, and accumulate the ratios to obtain the energy utilization model U of each network cluster. m (t)=Σ (m,j) b (m,j) (t) / B m,j Then, the throughput model and energy utilization model are accumulated to generate the target load balancing optimization function

[0096] Finally, based on the transmission rate parameter b (m,j) (t), target load balancing optimization function ξ e1 As well as the transmission rate parameter range constraints, the load balancing optimization model for each network cluster is generated as shown in the following optimization problem (P1).

[0097]

[0098] α (m,j) (t)≤B m,j ,b (m,j) (t)≤B m,j , (4)

[0099] a m (t) = π m (S m (t)). (5)

[0100] Among them, ST is the scheduling time interval. is the network cluster k m In time slot t, all communication links l (m,j) The transmission rate b (m,j) (t). β is the balance weight between the objectives of maximizing throughput and maximizing resource utilization in each network cluster. m,j is the network cluster k m The maximum bandwidth of the jth link. Formula (1) is the overall goal (i.e., the target load balancing optimization function), which strives to achieve a trade-off between maximizing throughput and maximizing resource utilization within the ST time interval. Formula (2) is calculated by network cluster k m The throughput in time slot t is calculated by the transmission rate of the network cluster k. Formula (3) m The bandwidth resource utilization rate in time slot t is calculated based on the transmission rate of the network cluster k. Formula (4) m Communication Link (m,j) The transmission rate and arrival rate are constrained and cannot be greater than the maximum transmission bandwidth B of the communication link. m,j Formula (5) is the network cluster k m The execution strategy constraints in time slot t. This load balancing optimization model can be understood as: under the constraint of limited bandwidth resources, network cluster k m Finding the optimal transmission control strategy π m . Then π m According to the network status S at time slot t m (t) Conduct transmission control strategy guidance m (t), achieving the trade-off goal of maximizing network throughput and maximizing resource utilization.

[0101] Through the above steps 401 to 405, the relevant mathematical modeling parameters in the network cluster system are used to adjust the transmission rate of the communication link between the network nodes in each network cluster to achieve the purpose of improving the throughput and bandwidth resource utilization in the network cluster, and build a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model, which can effectively use the collaborative load balancing optimization model to improve the resource utilization in the network cluster system.

[0102] In some embodiments, in an actual network cluster system In the process, due to the randomness of the transmission traffic and the dynamic nature of the communication link state, the load capacity of the communication link will fluctuate. Therefore, a queuing model is further defined to temporarily store additional data that exceeds the instantaneous load of the network, such as in the t time slot network cluster k m The queue model in the jth link is Q(m,j) (t). And further based on the queuing model, transmission rate and packet loss rate of time slot t, the queuing model update model is generated as shown in the following formula (6).

[0103]

[0104] At this time, the network cluster k m The set of all link queue vectors in is Taking into account the overall transmission stability of the network cluster system, assuming that the network cluster k m In control strategy and the maximum network capacity Λ, any communication link l (m,j) The long-term queue depth Q (m,j) (t) are subject to the long-term stability constraint as shown in the following formula (7).

[0105]

[0106] Based on the above long-term stability constraint analysis and the load balancing optimization model of each network cluster, the load balancing optimization model can be further transformed into a collaborative load balancing model of the network cluster system. At this time, assuming that the access network node of the network cluster m needs to send large-scale data traffic to the network node of another network cluster g, according to the connectivity E of each network cluster, it is necessary to select multiple adjacent network clusters to form a collaborative transmission group. Perform load balancing transmission control. At this point, based on the load balancing optimization model P1 of a single network cluster, the cluster collaborative load balancing model P2 can be further obtained as shown below.

[0107]

[0108] in, The goal of the cluster collaboration load balancing model P2 is to transform the collaborative transmission within each network cluster into collaborative transmission between multiple network clusters in the network cluster system. Formula (8) is the goal of this model, which aims to achieve the joint goal of maximizing throughput and maximizing bandwidth utilization during the cluster collaboration process. Formula (9) calculates the total throughput of the collaboration between network clusters in time slot t. Formula (10) calculates the total bandwidth utilization of the collaboration between network clusters in time slot t. Formula (11) represents the collaboration strategy between network clusters. Under control, according to the control strategy of each network cluster m Control the traffic distribution ratio of the corresponding network cluster m (t). Formula (12) is the stability constraint of the cooperative transmission queue.

[0109] Step 302: Generate a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model.

[0110] Step 302 is described in detail below.

[0111] In some embodiments, after obtaining the load balancing optimization model of each network cluster, in order to achieve accurate multi-network cluster collaborative load balancing scheduling, taking into account the large network scale and the difficulty of perception and control in the process of multi-network cluster collaborative transmission in the network cluster system, a multi-network collaborative load balancing scheduling framework is studied and designed. Figure 5 , is a schematic diagram of a multi-network collaborative load balancing scheduling framework provided by an embodiment of the present application. Figure 5 As shown in , the idea of ​​distributed architecture is introduced to manage the entire network in the network cluster system in clusters. Each cluster has an independent perception module, a policy update iteration module, and a policy execution control module, which can speed up the perception and control process. At this time, it is difficult for the network distributed cluster management to achieve global optimal scheduling. Therefore, in this embodiment, a distributed asynchronous learning method is also introduced to perform distributed asynchronous learning updates on the scheduling strategy of each cluster to enhance the global average optimal scheduling capability.

[0112] exist Figure 5 The multi-network collaborative load balancing scheduling framework shown in the figure mainly includes network perception function, network clustering management function, policy update function, and policy asynchronous learning function. The network perception function mainly uses the INT framework to perceive the network node status and traffic; the network clustering management function is mainly responsible for clustering networks with the same transmission attributes; the policy update function mainly uses the perception data and the Actor-Critic framework of reinforcement learning to iteratively generate policies; the policy asynchronous learning function mainly uses the distributed asynchronous learning method to share policies for algorithms with the same model structure and the same attribute input. The specific framework is based on the "three-layer" design of the intelligent service layer, resource adaptation layer, and network component layer, and designs five components to realize the network perception, policy generation, and policy delivery process. The components are mainly distributed in the "network component layer" and "resource adaptation layer". The specific functions of the components are described as follows.

[0113] Perception Observer: Mainly responsible for perceiving and detecting all network component resources and traffic. This chapter mainly combines the INT framework to perform detection tasks.

[0114] Cluster manager: performs clustering and grouping control of nodes according to clustering control strategies.

[0115] Cross-layer parser: Mainly responsible for parsing and extracting perception data and encapsulating and sending dynamic policy data packets. Specifically, when the cross-layer parser receives the INT perception data packet from the perception observer, it will parse the INT data packet to extract the perception data. On the other hand, when the cross-layer parser receives the policy action from the policy generator or the policy parameters of the asynchronous learner, it will generate a policy package to encapsulate and send the corresponding action and parameters.

[0116] Data analyzer: mainly responsible for further formatting of perception data and providing the required formatted data input for the algorithm.

[0117] Policy generator: Generates and updates policies based on embedded algorithms and formatted perception data.

[0118] Asynchronous learner: It mainly performs periodic learning and updating based on the strategy of each cluster, and sends the strategy parameters to each cluster head for traffic load balancing scheduling.

[0119] Adaptive scheduler: Controls the traffic transmission rate of all links according to the scheduling strategy.

[0120] Based on Figure 5 The multi-network collaborative load balancing scheduling framework shown below further describes how to generate a collaborative load balancing optimization model for a network cluster based on the load balancing optimization model.

[0121] Reference Figure 6 , generating a collaborative load balancing optimization model of a network cluster based on the load balancing optimization model, including the following steps 601 to 604.

[0122] Step 601: Generate a cluster state space based on the link connection matrix and link bandwidth parameters, link delay parameters, queue packet loss parameters, and queue depth generation parameters of each communication link.

[0123] Step 602: Generate a cluster action space based on the transmission rate parameter, and generate a cluster reward model based on the target load balancing optimization function.

[0124] Step 603: Obtain initial graph convolutional network parameters of the graph convolutional neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model and the initial graph convolutional network parameters.

[0125] Step 604: Perform multiple rounds of training on the initial collaborative load balancing optimization model, and obtain a collaborative load balancing optimization model of the network cluster based on the trained initial collaborative load balancing optimization model.

[0126] Steps 601 to 604 are described in detail below.

[0127] In some embodiments, taking into account the heterogeneity of network node resources within the network cluster system and the different load capacities of each communication link, a graph convolutional neural network (GCN) is introduced. The "barrel" short board idea is used to cluster adjacent nodes / links to realize the joint load capacity analysis of multiple nodes, and then generate strategies for transmission control.

[0128] Then, combined with the framework idea of ​​reinforcement learning, the link state S of the communication link between each two network nodes in each time slot network cluster system is calculated. m (t)∈R p×fs , where fs includes link bandwidth parameters, link delay parameters, queue packet loss parameters, and queue depth generation parameters, and is further combined with the link connection matrix ε=(l m,ij ) nm×nm Generate a cluster state space in each time slot network cluster system.

[0129] Then, the transmission rate parameter of the communication link between every two network nodes in each time slot network cluster system is used to generate the cluster action space

[0130] In the reinforcement learning framework, the reward value is the feedback after the action is executed. The policy parameters are adjusted according to the quality of the feedback, and then the control action is adjusted to meet the current S m (t) is the optimal control. Therefore, according to the target load balancing optimization function (1) in the load balancing optimization model P1, the cluster reward model is generated as shown in the following formula (13).

[0131]

[0132] In addition, according to the above definition, the input layer of the graph convolutional neural network is set to {S m (t), A}. The hidden layer iteration process of the graph convolutional neural network is shown in the following formula (14).

[0133]

[0134] Among them, the input of the l+1 layer network is H l ∈R P×fs , in the initial output layer H 0 =S m (t) P×fs fs is the dimension of the network state feature vector. P is the number of links. To add a self-connected connected adjacency matrix, I P is the unit matrix. A is the current network cluster k mThe adjacency matrix E of . is the degree matrix, W l is the graph neural network parameter. σ is the activation function. At this time, the polynomial fitting function of the graph convolutional neural network is defined as shown in the following formula (15).

[0135] Z=f(X,A) (15)

[0136] X is the input of each layer. In the initial output layer, X = H 0 =S m (t) P×fs At this point, based on the above cluster reward model (13), the loss function of the graph convolutional neural network can be further generated as shown in the following formula (16).

[0137] Loss=(ξ r (t)-ξ r (t-1))lnZ. (16)

[0138] Based on the above description, combined with the reinforcement learning framework, we first determine the initial graph convolutional network parameters of the graph convolutional neural network (which includes the relevant iterative parameters in the hidden layer, i.e., the relevant parameters in formula (14)); then generate the initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model and the initial graph convolutional network parameters.

[0139] Next, based on the reinforcement learning algorithm framework, the initial collaborative load balancing optimization model is trained for multiple rounds, and the collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model, so as to effectively improve the transmission resource utilization of the network cluster system in practical applications. The following will further describe how to train the initial collaborative load balancing optimization model.

[0140] Reference Figure 7 The training process of the initial collaborative load balancing optimization model includes the following steps 701 to 704.

[0141] Step 701: Obtain the cluster training state and training strategy of the current iteration, determine the cluster training action from the training strategy based on the cluster training state, and obtain the cluster training update state based on the cluster training action.

[0142] Step 702: Based on the cluster reward model, the cluster training update state, the cluster training state and the cluster training action, the training loss value of the cluster training action is calculated.

[0143] Step 703: Obtain a fitting function of the graph convolutional neural network, and update the initial graph convolutional network parameters based on the training loss value and the fitting function.

[0144] Step 704: Update the training strategy based on the updated graph convolutional neural network.

[0145] Steps 701 to 704 are described in detail below.

[0146] In some embodiments, based on the above analysis, the graph neural network (GCN) is combined with the deep reinforcement learning method, and the internal parameters of the graph neural network are trained through multiple parameter iterations to obtain the transmission control strategy π of each network cluster. m . Reference Figure 8 , is a schematic diagram of the training process of an initial collaborative load balancing optimization model provided in an embodiment of the present application. The training process framework is mainly divided into two parts: online scheduling process and offline training process. The online scheduling process collects the link status in the network cluster system through perception, and then combines the link status and the link connection matrix into a graph network status as the input of the scheduler. Transmission control strategy π m According to the input information, the corresponding communication link transmission rate control action is output to control the flow transmission in the network cluster system. The offline training process mainly uses the latest experience data of the network cluster system to carry out the transmission control strategy π m Training. Formatted experience data is formed based on the perception of link status, and then the experience data is stored centrally. The two neural networks in the trainer perform action predictions respectively, and update the neural network strategy parameters based on the loss function. Finally, the trainer updates the updated neural network strategy parameters to the scheduler for a new round of scheduling control. The details are described as follows.

[0147] Reference Fig. 9 , is a schematic diagram of a training algorithm for an initial collaborative load balancing optimization model provided in an embodiment of the present application. Fig. 9 In the algorithm shown, the link connection matrix E is first input. The network and framework components are initialized, and network perception and transmission control based on INT are performed (i.e., step 1 in Algorithm 1). Then, the graph convolutional neural network scheduling strategy parameters are randomly initialized. (i.e., step 2 in Algorithm 1). Next, perform network perception and control, collect state information, and perform strategy training and updating (i.e., steps 3 to 11 in Algorithm 1). Specifically, obtain the network cluster k of the current iteration. m The cluster training state S m (t) (i.e., step 4 in Algorithm 1) and training strategy π m , then based on the cluster training state S m (t) Determine cluster training action a from the training strategy m (t) (i.e., step 5 in Algorithm 1), and based on the cluster training action, the cluster training update state S is obtained m(t+1), and the training loss value of the cluster training action is calculated by the above formula (16), and the test array {S m (t),a m (t),R loss ,S m (t+1)} (i.e., steps 6 to 9 in Algorithm 1). Next, m data are randomly selected from the experience pool, and the graph convolutional neural network gradient solution and strategy update are performed according to the above fitting function (15) and the training loss value (line 10 in Algorithm 1). When executing the ST round strategy update or the loss value R loss <ω, output strategy

[0148] The overall overhead of Algorithm 1 includes the storage of experience values ​​and the training process of offline strategies. The overhead of this process increases as the time interval between strategy updates decreases. Therefore, in the experimental stage, the offline training stage is set to run on a separate server. The generation and transmission of other perception data packets and strategy data packets are mainly related to the frequency of strategy updates, and the overhead is relatively small. In addition, the algorithm is multi-channel cooperative transmission, which will cause the problem of data disorder at the receiving end, resulting in additional data storage at the receiving end and data packet disorder processing overhead.

[0149] Through the above steps 601 to 604, and steps 701 to 704, based on the reinforcement learning framework, the link state and link connection matrix in the load balancing optimization model generated after mathematical modeling of the network cluster system, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the graph convolutional neural network are used to build an initial collaborative load balancing optimization model, and the reinforcement learning framework is used to train the initial collaborative load balancing optimization model to obtain a suitable collaborative load balancing optimization model, so as to utilize the "barrel" short board idea to realize the joint load capacity analysis of multiple nodes through the clustering of adjacent nodes / links, and then generate a strategy for transmission control, so as to effectively improve the transmission resource utilization rate when information is transmitted within each network cluster in the network cluster system.

[0150] Step 303: Generate a network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model.

[0151] Step 303 is described in detail below.

[0152] In some embodiments, after obtaining the collaborative load balancing optimization model of a single network cluster, it is necessary to further consider the collaborative load balancing problem of multiple network clusters in the global network cluster system. When a network cluster m accesses any network node and needs to send large-scale data traffic to any network node of another network cluster g, multiple adjacent network clusters are selected to form a collaborative transmission group according to the connectivity E of each network cluster. And carry out corresponding flow distribution ratio control Realize cooperative transmission group Based on this, it is necessary to further generate a network cluster scheduling model based on the collaborative load balancing optimization model of multiple network clusters in the network cluster system to improve the utilization of transmission resources when large-scale network clusters are transmitting information. The following will further describe how to generate a network cluster scheduling model.

[0153] Reference Fig.10 , generating a network cluster scheduling model of a network cluster system based on a collaborative load balancing optimization model, including the following steps 1001 to 1004.

[0154] Step 1001: Based on the collaborative load balancing optimization model of each network cluster, an optimized cluster training action of each network cluster is obtained.

[0155] Step 1002: Based on all optimized cluster training actions in each network cluster, obtain the training traffic allocation ratio of each network cluster.

[0156] Step 1003: Obtain an update status of each network cluster based on the training traffic allocation ratio, and update the training traffic allocation ratio based on the update status.

[0157] Step 1004: Obtain a network cluster scheduling model based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0158] Steps 1001 to 1004 are described in detail below.

[0159] In some embodiments, when facing multiple network cluster systems in a network cluster system, the coordinated transmission group To solve the load balancing problem, it is necessary to optimize the transmission rate strategy of a single network cluster and the traffic distribution of multiple network clusters at the same time. Fig.11 , is a schematic diagram of a generation process of a network cluster scheduling model provided in an embodiment of the present application.

[0160] like Fig.11As shown in , first of all, due to the unbalanced deployment of network components, the number of network nodes and connectivity of each network cluster in the original network cluster system are mostly inconsistent, that is, it is necessary to supplement the most complete graph structure for all original groups and format them into the same network structure, that is, the number of graph nodes and connectivity remain consistent. At this time, execute Algorithm 2 to perform load balancing training for each single cluster group. In order to achieve collaborative load balancing among multiple cluster groups, a distributed asynchronous learning method is introduced to share and learn parameters for the same formatted graph structure. Finally, according to the transmission requirements of the sending and receiving ends of the traffic, the graph structure is formatted into a multi-branch tree structure, and according to the strategies of each network cluster Predict the transmission rate weight of each link, and introduce multi-branch tree and simulated annealing ideas to dynamically adjust the traffic distribution ratio of each group Realize dynamic adaptation and scheduling of traffic and network resources.

[0161] Reference Fig.12 , is a schematic diagram of a generation algorithm of a network cluster scheduling model provided by an embodiment of the present application. Based on the above model and problem transformation, a multi-cluster collaborative load balancing scheduling algorithm based on distributed asynchronous learning is proposed, such as Fig.12 The algorithm mainly includes two parts: the calculation of the proportion of collaborative traffic distribution between network clusters (steps 3 to 12 in Algorithm 2) and the asynchronous learning of the group output strategy (steps 13 to 17 in Algorithm 2). First, initialize the components of the framework and perform network perception and control based on the INT framework (step 1 in Algorithm 2). Then, call Algorithm 1, which is the collaborative load balancing optimization model based on each network cluster, to obtain the optimized cluster training action a of each network cluster. m (t) and execute (Step 3 in Algorithm 2). Then, according to the traffic sending and receiving end addresses, each network cluster is multi-tree path planned, and according to all the optimized cluster training actions in each network cluster, that is, the cumulative transmission rate a m (t) Calculate the root node weight of the network cluster, that is, the training traffic allocation ratio w m (Line 3 in Algorithm 2). Then the root node weights w of all network clusters are m Normalization is performed and traffic allocation scheduling is performed (lines 5 to 6 in Algorithm 2) to obtain the updated state of each network cluster, and then the training traffic allocation ratio of each network cluster is updated based on the updated state (lines 7 to 17 in Algorithm 2), as described below.

[0162] Reference Fig.13 , updating the training traffic allocation ratio based on the updated status, including the following steps 1301 to 1304.

[0163] Step 1301: Select a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a first target adjustment cluster, and update the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate.

[0164] Step 1302: Based on the updated training traffic distribution ratio, obtain an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value.

[0165] Step 1303: When the updated target load balancing optimization function update value is a negative number and the updated packet loss rate update value is a positive number, select a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as the second target adjustment cluster.

[0166] Step 1304: Select the network cluster with the largest updated target load balancing optimization function value as the benchmark cluster, and update the training traffic allocation ratio of the second target adjustment cluster based on the training traffic allocation ratio of the benchmark cluster.

[0167] Steps 1301 to 1304 are described in detail below.

[0168] In some embodiments, reference Fig.12 As shown in Algorithm 2 in , after executing the traffic distribution scheduling to obtain the updated state of each network cluster, the updated state of each network cluster includes the target load balancing optimization function update value ξ e1 (t+1)-ξ e1 (t) and packet loss rate update value D m (t+1)-D m (t), detect the network status of each group and perform performance analysis before and after traffic execution, define the overall throughput and bandwidth resource utilization of the network cluster as joint goals, and when the performance of time slot t is less than the performance of time slot t+1 (ξ e1 (t)<ξ e1 (t+1), that is, the updated value of the target load balancing optimization function ξ e1 (t+1)-ξ e1 (t) is negative), and the total packet loss in the t+1 time slot increases (D m (t) <D m (t+1), that is, the packet loss rate update value D m (t+1)-D m (t) is a negative number), the network cluster k is determined m In the case of traffic overload, the network cluster k m As the first target adjustment cluster, the first target adjustment cluster k is then adjusted by a ratio of one minus the traffic degradation rate ζ. mThe training traffic allocation ratio is adjusted (lines 7 to 11 in Algorithm 2). After the ratio adjustment is completed, the traffic allocation ratio is adjusted again (step 12 in Algorithm 2). At this time, all network clusters are traversed again, that is, based on the updated training traffic allocation ratio, the updated target load balancing optimization function update value, the updated packet loss rate update value, and the updated target load balancing optimization function value are obtained again. When the overall performance is weakened and the packet loss increases (ξ e1 (t)<ξ e1 (t+1) and D m (t) <D m (t+1), that is, the updated value of the target load balancing optimization function ξ e1 (t+1)-ξ e1 (t) is negative and the packet loss rate is updated to D m (t+1)-D m (t) is a negative number), determine the current group k m The strategy has low performance and clusters the network into k m As the second target adjustment cluster, select the target load balancing optimization function value ξ from multiple network clusters at this time e1 (t) The best network cluster k j As the benchmark cluster, then based on the training traffic allocation ratio of the benchmark cluster, the second objective is updated with probability ε to adjust the training traffic allocation ratio θ of the cluster m =εθ m +(1-ε)θ j . Finally, the adjusted flow ratio is output The overhead of Algorithm 2 is mainly the parameter sharing additive update of adjacent strategies, which has a small overhead.

[0169] Through the above steps 1001 to 1004, and steps 1301 to 1304, the collaborative load balancing optimization model of each network cluster is integrated, and the obtained optimized cluster training action is used to plan the training traffic allocation ratio of each network cluster, and then the training traffic allocation ratio of the network cluster is gradually updated based on different update states to obtain a suitable network cluster scheduling model for the network cluster system. In actual applications, in complex large-scale networks, based on the traffic states and link connection matrices of different network clusters, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio can be generated for each network cluster, so as to improve the transmission resource utilization when large-scale network clusters transmit information.

[0170] Step 203: adjusting the transmission rate of each communication link in the corresponding network cluster according to the optimized transmission rate sequence of each link, and adjusting the flow distribution of the corresponding network cluster according to each optimized flow distribution ratio.

[0171] The following is a detailed description of step 203.

[0172] In some embodiments, after obtaining the link status of all communication links of the current network cluster system and the link connection matrix of the optimal link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster based on the network cluster scheduling model, the transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic allocation of the corresponding network cluster is adjusted according to each optimized traffic allocation ratio. Then, after the parameter adjustment, data transmission of each network node is performed in the network cluster system to improve the utilization rate of transmission resources.

[0173] The transmission optimization method and related equipment of the network cluster system proposed in the embodiment of the present application, the network cluster system includes multiple network clusters, and the network cluster includes multiple network nodes. The method includes: first, obtaining the link status of the communication link between every two network nodes, and obtaining the link connection matrix, the link connection matrix is ​​used to characterize the connection status of each communication link; then, obtaining the transmission rate parameter and maximum bandwidth of each communication link, and obtaining the transmission rate parameter range constraint of the transmission rate parameter, based on the cumulative sum of multiple transmission rate parameters, obtaining the throughput model of each network cluster, obtaining the ratio of multiple transmission rate parameters to the maximum bandwidth, and accumulating the ratios to obtain the energy utilization model of each network cluster, accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function, based on the transmission rate parameters, the target load balancing optimization function and the transmission rate parameter range constraint, generating a load balancing optimization model, based on the link connection matrix and the link bandwidth parameters, link delay parameters, queuing packet loss parameters and queue depth generation parameters of each communication link to generate a cluster. state space, generate a cluster action space based on the transmission rate parameter, and generate a cluster reward model based on the target load balancing optimization function, obtain the initial graph convolution network parameters of the graph convolution neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model and initial graph convolution network parameters, perform multiple rounds of training on the initial collaborative load balancing optimization model, and obtain the collaborative load balancing optimization model of the network cluster based on the trained initial collaborative load balancing optimization model. The training process of the initial collaborative load balancing optimization model includes: obtaining the cluster training state and training strategy of the current iteration, determining the cluster training action from the training strategy based on the cluster training state, and obtaining the cluster training update state based on the cluster training action, calculating the training loss value of the cluster training action based on the cluster reward model, cluster training update state, cluster training state and cluster training action, obtaining the fitting function of the graph convolution neural network, and updating the initial graph convolution network parameters based on the training loss value and the fitting function, and updating the training strategy based on the updated graph convolution neural network;Furthermore, based on the collaborative load balancing optimization model of each network cluster, the optimized cluster training action of each network cluster is obtained; based on all the optimized cluster training actions in each network cluster, the training traffic distribution ratio of each network cluster is obtained; the network cluster whose target load balancing optimization function update value is negative and whose packet loss rate update value is positive is selected as the first target adjustment cluster; the training traffic distribution ratio of the first target adjustment cluster is updated based on a minus traffic degradation rate; based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value and an updated target load balancing optimization function value are obtained; when the updated target load balancing optimization function update value is negative and the updated packet loss rate update value is positive, the target load balancing optimization function update value is selected. The network cluster with a negative value and a positive packet loss rate update value is selected as the second target adjustment cluster, and the network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster. The training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster. The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic allocation ratio. All link states and link connection matrices are input into the network cluster scheduling model to obtain the output link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster; finally, the transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic allocation of the corresponding network cluster is adjusted according to each optimized traffic allocation ratio. ;

[0174] The embodiment of the present application utilizes the link state of the changing communication link in the network cluster system and the link connection matrix used to characterize whether the network nodes are communicatively connected, and combines the network cluster scheduling model constructed in advance based on the communication process performed by the network nodes in the network cluster, so as to obtain the link optimization transmission rate sequence and optimized traffic distribution ratio of multiple network clusters that match the link state and the link connection matrix appropriately, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, so as to effectively improve the transmission resource utilization rate when information is transmitted between network nodes in the network cluster system; and, utilizes the relevant mathematical modeling parameters in the network cluster system, so as to adjust the transmission rate of the communication link between network nodes in each network cluster, so as to achieve the purpose of improving the throughput and bandwidth resource utilization rate in the network cluster, and construct a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model, and the collaborative load balancing optimization model can be effectively used to improve the resource utilization rate in the network cluster system; in addition, based on the reinforcement learning framework, the load balancing optimization model generated by mathematical modeling of the network cluster system is utilized. The link state and link connection matrix in the network cluster system, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the graph convolutional neural network are used to build an initial collaborative load balancing optimization model, and the reinforcement learning framework is used to train the initial collaborative load balancing optimization model to obtain a suitable collaborative load balancing optimization model, so as to use the "barrel" short board idea to realize the joint load capacity analysis of multiple nodes through the clustering of adjacent nodes / links, and then generate a strategy for transmission control to effectively improve the transmission resource utilization rate when each network cluster in the network cluster system transmits information; and, by integrating the collaborative load balancing optimization model of each network cluster, and using the obtained optimized cluster training action to plan the training traffic allocation ratio of each network cluster, and then gradually updating the training traffic allocation ratio of the network cluster based on different update states, a suitable network cluster scheduling model for the network cluster system can be obtained, so that in practical applications, based on the traffic state and link connection matrix of different network clusters in complex large-scale networks, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster can be generated, so as to improve the transmission resource utilization rate when large-scale network clusters transmit information.

[0175] The present application also provides a transmission optimization device for a network cluster system, which can implement the transmission optimization method for the network cluster system. Fig.14 , the device 1400 comprises:

[0176] The state acquisition module 1410 is used to acquire the link state of the communication link between every two network nodes, and to acquire a link connection matrix, where the link connection matrix is ​​used to characterize the connection state of each communication link;

[0177] The optimization parameter acquisition module 1420 is used to obtain a network cluster scheduling model, input all link states and link connection matrices into the network cluster scheduling model, and obtain the link optimization transmission rate sequence and optimized traffic allocation ratio of each network cluster as output. The link optimization transmission rate sequence and optimized traffic allocation ratio are used to optimize the throughput and resource utilization within the network cluster system.

[0178] The parameter adjustment module 1430 is used to adjust the transmission rate of each communication link in the corresponding network cluster one by one according to the transmission rate sequence of each link optimization, and adjust the flow distribution of the corresponding network cluster one by one according to each optimized flow distribution ratio;

[0179] The information transmission module 1440 is used to transmit information between at least two network nodes after the transmission rate of at least one communication link and the traffic distribution of at least one network cluster are adjusted.

[0180] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0181] Obtain the load balancing optimization model for each network cluster;

[0182] Generate a collaborative load balancing optimization model for network clusters based on the load balancing optimization model;

[0183] A network cluster scheduling model for a network cluster system is generated based on a collaborative load balancing optimization model.

[0184] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0185] Obtaining a transmission rate parameter and a maximum bandwidth of each communication link, and obtaining a transmission rate parameter range constraint of the transmission rate parameter;

[0186] Based on the cumulative sum of multiple transmission rate parameters, a throughput model of each network cluster is obtained;

[0187] Obtain ratios of multiple transmission rate parameters and maximum bandwidth, and accumulate the ratios to obtain an energy utilization model for each network cluster;

[0188] The throughput model and the energy utilization model are accumulated to generate the target load balancing optimization function;

[0189] A load balancing optimization model is generated based on the transmission rate parameters, the target load balancing optimization function and the transmission rate parameter range constraints.

[0190] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0191] Generate a cluster state space based on the link connection matrix and link bandwidth parameters, link delay parameters, queue packet loss parameters, and queue depth generation parameters of each communication link;

[0192] Generate cluster action space based on transmission rate parameters, and generate cluster reward model based on target load balancing optimization function;

[0193] Obtain the initial graph convolutional network parameters of the graph convolutional neural network, and generate an initial collaborative load balancing optimization model based on the cluster state space, cluster action space, cluster reward model, and the initial graph convolutional network parameters;

[0194] The initial collaborative load balancing optimization model is trained for multiple rounds, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

[0195] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0196] Obtain the cluster training state and training strategy of the current iteration, determine the cluster training action from the training strategy based on the cluster training state, and obtain the cluster training update state based on the cluster training action;

[0197] Based on the cluster reward model, the cluster training update state, the cluster training state, and the cluster training action, the training loss value of the cluster training action is calculated;

[0198] Get the fitting function of the graph convolutional neural network, and update the initial graph convolutional network parameters based on the training loss value and the fitting function;

[0199] Update the training strategy based on the updated graph convolutional neural network.

[0200] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0201] Based on the collaborative load balancing optimization model of each network cluster, the optimized cluster training action of each network cluster is obtained;

[0202] Based on all the optimized cluster training actions in each network cluster, the training traffic allocation ratio of each network cluster is obtained;

[0203] Obtaining an updated state of each network cluster based on the training traffic allocation ratio, and updating the training traffic allocation ratio based on the updated state;

[0204] The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

[0205] In some embodiments, the optimization parameter acquisition module 1420 is further used to:

[0206] Select a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as the first target adjustment cluster, and update the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate;

[0207] Based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value are obtained;

[0208] When the updated value of the target load balancing optimization function is a negative number and the updated value of the packet loss rate is a positive number, a network cluster whose updated value of the target load balancing optimization function is a negative number and whose updated value of the packet loss rate is a positive number is selected as the second target adjustment cluster;

[0209] The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

[0210] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, the specific implementation method of the transmission optimization device of the network cluster system is basically the same as the specific implementation method of the transmission optimization method of the above network cluster system, and will not be repeated here.

[0211] In an embodiment of the present application, a transmission optimization device of a network cluster system utilizes the link state of a changing communication link in the network cluster system and a link connection matrix used to characterize whether network nodes are connected for communication, and constructs a network cluster scheduling model based on the communication process performed by network nodes in the network cluster in advance, thereby obtaining a link optimization transmission rate sequence and an optimized traffic distribution ratio of multiple network clusters that match the link state and the link connection matrix appropriately, so as to adjust the transmission rate of multiple network nodes in each network cluster and the traffic distribution of each network cluster, thereby effectively improving the transmission resource utilization rate when information is transmitted between network nodes in the network cluster system; and, utilizing relevant mathematical modeling parameters in the network cluster system, so as to adjust the transmission rate of the communication link between network nodes in each network cluster, so as to achieve the purpose of improving the throughput and bandwidth resource utilization rate in the network cluster, and construct a load balancing optimization model for each network cluster, so as to facilitate the subsequent use of the load balancing optimization model in combination with the reinforcement learning framework to generate a collaborative load balancing optimization model, and effectively utilize the collaborative load balancing optimization model to improve the resource utilization rate in the network cluster system; in addition, based on the reinforcement learning framework, utilizing the load balancing optimization model generated by mathematical modeling of the network cluster system The link state and link connection matrix in the load balancing optimization model, the transmission rate of each communication link, the reward model generated by the target load balancing optimization function, and the initial collaborative load balancing optimization model are constructed in combination with the graph convolutional neural network, and the initial collaborative load balancing optimization model is trained using the reinforcement learning framework to obtain a suitable collaborative load balancing optimization model, so as to use the "barrel" short board idea to realize the joint load capacity analysis of multiple nodes through the clustering of adjacent nodes / links, and then generate a strategy for transmission control to effectively improve the transmission resource utilization rate when information is transmitted within each network cluster in the network cluster system; and, by integrating the collaborative load balancing optimization model of each network cluster, and using the obtained optimized cluster training action to plan the training traffic allocation ratio of each network cluster, and then gradually updating the training traffic allocation ratio of the network cluster based on different update states, to obtain a suitable network cluster scheduling model for the network cluster system, so that in practical applications, based on the traffic state and link connection matrix of different network clusters in complex large-scale networks, a suitable link optimized transmission rate sequence and optimized traffic allocation ratio of each network cluster can be generated to improve the transmission resource utilization rate when large-scale network clusters transmit information.

[0212] The present application also provides an electronic device, including:

[0213] at least one memory;

[0214] at least one processor;

[0215] at least one program;

[0216] The program is stored in the memory, and the processor executes the at least one program to implement the transmission optimization method of the network cluster system implemented in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0217] See also Fig.15 , Fig.15 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0218] The processor 1501 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0219] The memory 1502 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 1502 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1502, and the processor 1501 calls and executes the transmission optimization method of the network cluster system of the embodiment of the present application;

[0220] Input / output interface 1503, used to implement information input and output;

[0221] Communication interface 1504, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WIFI, Bluetooth, etc.);

[0222] A bus 1505 that transmits information between the various components of the device (e.g., the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504);

[0223] The processor 1501 , the memory 1502 , the input / output interface 1503 and the communication interface 1504 are connected to each other in communication within the device via the bus 1505 .

[0224] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the transmission optimization method of the above-mentioned network cluster system is implemented.

[0225] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0226] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0227] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0228] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0229] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0230] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. The mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0236] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A transmission optimization method for a network cluster system, characterized in that: The network cluster system includes a plurality of network clusters, the network cluster includes a plurality of network nodes, and the method includes: Acquire a link state of a communication link between every two of the network nodes, and acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize a connection state of each of the communication links; Acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and obtain an output link optimized transmission rate sequence and optimized traffic distribution ratio of each network cluster; The transmission rate of each communication link in the corresponding network cluster is adjusted according to each link optimized transmission rate sequence, and the traffic distribution of the corresponding network cluster is adjusted according to each optimized traffic distribution ratio.

2. The transmission optimization method of the network cluster system according to claim 1, characterized in that: The obtaining of the network cluster scheduling model includes: Obtaining a load balancing optimization model for each of the network clusters; Generate a collaborative load balancing optimization model of the network cluster based on the load balancing optimization model; The network cluster scheduling model of the network cluster system is generated based on the collaborative load balancing optimization model.

3. The transmission optimization method of the network cluster system according to claim 2, characterized in that: The obtaining of the load balancing optimization model of each of the network clusters includes: Obtaining a transmission rate parameter and a maximum bandwidth of each of the communication links, and obtaining a transmission rate parameter range constraint of the transmission rate parameter; Based on the cumulative sum of the plurality of transmission rate parameters, a throughput model of each of the network clusters is obtained; Acquire a plurality of ratios of the transmission rate parameters to the maximum bandwidth, and accumulate the ratios to obtain an energy utilization model of each of the network clusters; Accumulating the throughput model and the energy utilization model to generate a target load balancing optimization function; The load balancing optimization model is generated based on the transmission rate parameter, the target load balancing optimization function and the transmission rate parameter range constraint.

4. The transmission optimization method of the network cluster system according to claim 3, characterized in that: The generating the collaborative load balancing optimization model of the network cluster based on the load balancing optimization model includes: Generate a cluster state space based on the link connection matrix and a link bandwidth parameter, a link delay parameter, a queue packet loss parameter, and a queue depth generation parameter of each of the communication links; generating a cluster action space based on the transmission rate parameter, and generating a cluster reward model based on the target load balancing optimization function; Obtaining initial graph convolutional network parameters of a graph convolutional neural network, and generating an initial collaborative load balancing optimization model based on the cluster state space, the cluster action space, the cluster reward model, and the initial graph convolutional network parameters; The initial collaborative load balancing optimization model is trained multiple times, and a collaborative load balancing optimization model of the network cluster is obtained based on the trained initial collaborative load balancing optimization model.

5. The transmission optimization method of the network cluster system according to claim 4, characterized in that: The training process of the initial collaborative load balancing optimization model includes: Acquire a cluster training state and a training strategy of a current iteration, determine a cluster training action from the training strategy based on the cluster training state, and obtain a cluster training update state based on the cluster training action; Based on the cluster reward model, the cluster training update state, the cluster training state and the cluster training action, a training loss value of the cluster training action is calculated; Obtaining a fitting function of the graph convolutional neural network, and updating the initial graph convolutional network parameters based on the training loss value and the fitting function; The training strategy is updated based on the updated graph convolutional neural network.

6. The transmission optimization method of the network cluster system according to claim 2, characterized in that: The generating the network cluster scheduling model of the network cluster system based on the collaborative load balancing optimization model includes: Based on the collaborative load balancing optimization model of each of the network clusters, an optimized cluster training action of each of the network clusters is obtained; Based on all the optimized cluster training actions in each of the network clusters, obtaining a training traffic allocation ratio for each network cluster; Obtaining an update status of each of the network clusters based on the training traffic allocation ratio, and updating the training traffic allocation ratio based on the update status; The network cluster scheduling model is obtained based on the collaborative load balancing optimization model of the network cluster and the updated training traffic distribution ratio.

7. The transmission optimization method of the network cluster system according to claim 6, characterized in that: The updated state includes an updated value of a target load balancing optimization function and an updated value of a packet loss rate, and updating the training traffic distribution ratio based on the updated state includes: Selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a first target adjustment cluster, and updating the training traffic allocation ratio of the first target adjustment cluster based on a minus traffic degradation rate; Based on the updated training traffic distribution ratio, an updated target load balancing optimization function update value, an updated packet loss rate update value, and an updated target load balancing optimization function value are obtained; When the updated target load balancing optimization function update value is a negative number, and the updated packet loss rate update value is a positive number, selecting a network cluster whose target load balancing optimization function update value is a negative number and whose packet loss rate update value is a positive number as a second target adjustment cluster; The network cluster with the largest updated target load balancing optimization function value is selected as the benchmark cluster, and the training traffic allocation ratio of the second target adjustment cluster is updated based on the training traffic allocation ratio of the benchmark cluster.

8. A transmission optimization device for a network cluster system, characterized in that: The network cluster system includes a plurality of network clusters, the network cluster includes a plurality of network nodes, and the device includes: A state acquisition module, used to acquire the link state of the communication link between every two of the network nodes, and to acquire a link connection matrix, wherein the link connection matrix is ​​used to characterize the connection state of each of the communication links; An optimization parameter acquisition module is used to acquire a network cluster scheduling model, input all the link states and the link connection matrix into the network cluster scheduling model, and obtain an output link optimized transmission rate sequence and an optimized traffic allocation ratio for each network cluster, wherein the link optimized transmission rate sequence and the optimized traffic allocation ratio are used to optimize the throughput and resource utilization within the network cluster system; A parameter adjustment module, used to adjust the transmission rate of each communication link in the corresponding network cluster one by one according to each link optimized transmission rate sequence, and to adjust the flow distribution of the corresponding network cluster one by one according to each optimized flow distribution ratio; The information transmission module is used to transmit information in at least two of the network nodes after the transmission rate of at least one of the communication links and the flow distribution of at least one of the network clusters are adjusted.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the transmission optimization method of the network cluster system according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the transmission optimization method of the network cluster system according to any one of claims 1 to 7 is implemented.

Images