Collection communication method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a set communication method and device, computer equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring network state information of each intelligent calculation center in an intelligent calculation network; performing decision processing on the network state information through a pre-trained reinforcement learning model to obtain a set communication strategy for each intelligent calculation center; controlling data interaction among the intelligent calculation centers according to the set communication strategy; wherein the pre-trained reinforcement learning model is obtained by taking network state information as a state space and defining training performance of model training of the intelligent computing network as reward function training. By adopting the method, the set communication efficiency can be improved, the network congestion can be reduced, and the training rate of cross-computing-center model training can be further improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a collective communication method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of information technology, intelligent computing networks have become an important development direction in the field of modern computing. Intelligent computing networks integrate advanced technologies such as high-performance computing, big data analysis, and artificial intelligence to achieve rapid processing, intelligent analysis, and efficient decision-making of massive data.

[0003] In the process of training models in a network environment across intelligent computing centers, a large amount of communication is required between GPUs (graphics processing units), such as parameters, gradients, intermediate activation values, etc. These data transmission requirements have high requirements for network bandwidth and latency. When the network bandwidth is insufficient to meet the data transmission requirements, or the network latency is too high, network congestion may occur, thus affecting the model training rate. Summary of the invention

[0004] Based on this, it is necessary to provide a collective communication method, apparatus, computer equipment, computer-readable storage medium and computer program product to address the technical problem that network congestion is prone to occur in the network environment across the above-mentioned intelligent computing centers, which affects the model training rate.

[0005] In a first aspect, the present application provides a collective communication method. The method comprises:

[0006] Obtain the network status information of each intelligent computing center in the intelligent computing network;

[0007] Performing decision processing on the network status information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center;

[0008] According to the collective communication strategy, control the data interaction between the intelligent computing centers;

[0009] Among them, the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training.

[0010] In one embodiment, the training process of the pre-trained reinforcement learning model includes:

[0011] Define the state space, action space, and reward function of the reinforcement learning model, and initialize the model parameters;

[0012] In the current state, an action corresponding to the action space is selected according to the strategy of the reinforcement learning model, and the action is executed to obtain the next state and feedback information of the intelligent computing network;

[0013] Calculating a reward value according to the feedback information and the reward function;

[0014] The model parameters of the reinforcement learning model are updated according to the reward value until a training end condition is reached to obtain the pre-trained reinforcement learning model.

[0015] In one of the embodiments, the feedback information includes the throughput of the intelligent computing network for model training, the amount of data affected by network delay, and the amount of data affected by resource waste;

[0016] The calculating the reward value according to the feedback information and the reward function includes:

[0017] Calculate the total amount of data affected by the network delay and the amount of data affected by the resource waste based on the reward function;

[0018] The difference between the throughput and the total data volume is calculated to obtain the reward value.

[0019] In one embodiment, each of the intelligent computing centers includes multiple computing nodes; the strategy of the reinforcement learning model includes:

[0020] For any two intelligent computing centers, obtaining the difference between the numbers of computing nodes in the two intelligent computing centers;

[0021] If the quantity difference is smaller than a first threshold, the two intelligent computing centers are controlled to first perform internal collective communication respectively, and then perform collective communication between the two intelligent computing centers.

[0022] In one embodiment, the strategy of the reinforcement learning model further includes:

[0023] If the number difference is greater than a second threshold, controlling the intelligent computing center with fewer computing nodes in the two intelligent computing centers to first perform internal collective communication, and synchronize the execution result to the other intelligent computing center;

[0024] Control the other intelligent computing center to perform internal collective communication.

[0025] In one embodiment, the strategy of the reinforcement learning model further includes:

[0026] When setting the data parallel mode for the intelligent computing network, different intelligent computing centers are set to perform processing tasks for different data sets.

[0027] In a second aspect, the present application also provides a collective communication device. The device includes:

[0028] An information acquisition module is used to obtain network status information of each intelligent computing center in the intelligent computing network;

[0029] A strategy determination module, used to perform decision processing on the network state information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center; wherein the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training;

[0030] The collective communication module is used to control the data interaction between the various intelligent computing centers according to the collective communication strategy.

[0031] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0032] Obtain the network status information of each intelligent computing center in the intelligent computing network;

[0033] Performing decision processing on the network status information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center;

[0034] According to the collective communication strategy, control the data interaction between the intelligent computing centers;

[0035] Among them, the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Obtain the network status information of each intelligent computing center in the intelligent computing network;

[0038] Performing decision processing on the network status information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center;

[0039] According to the collective communication strategy, control the data interaction between the intelligent computing centers;

[0040] Among them, the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training.

[0041] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0042] Obtain the network status information of each intelligent computing center in the intelligent computing network;

[0043] Performing decision processing on the network status information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center;

[0044] According to the collective communication strategy, control the data interaction between the intelligent computing centers;

[0045] Among them, the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training.

[0046] The above-mentioned collective communication method, device, computer equipment, storage medium and computer program product, through pre-training, uses network state information as the state space, defines the training performance of the intelligent computing network for model training as the reward function, and uses the reinforcement learning model to generate a collective communication strategy for each intelligent computing center according to the network state information of each intelligent computing center in the intelligent computing network, so as to control the data interaction between each intelligent computing center. This method applies the reinforcement learning algorithm to the determination of the collective communication strategy, and generates the collective communication strategy based on the network state information of each intelligent computing center, so that the collective communication strategy can dynamically adapt to the network state of the intelligent computing network, thereby ensuring that each intelligent computing center can interact with data in the optimal communication mode, improve the efficiency of collective communication, reduce network congestion, and thus improve the training rate of model training across computing centers. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] FIG1 is an application environment diagram of a collective communication method in one embodiment;

[0048] FIG2 is a schematic diagram of a flow chart of a collective communication method in one embodiment;

[0049] FIG3 is a schematic diagram of a process flow of a reinforcement learning model training step in one embodiment;

[0050] FIG4 is a schematic diagram of a process flow of a reinforcement learning model training process in another embodiment;

[0051] FIG5 is a schematic diagram of the architecture of a collective communication algorithm based on reinforcement learning optimization in an intelligent computing network environment in one embodiment;

[0052] FIG6 is a block diagram of a collective communication device in one embodiment;

[0053] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0056] It can be understood that there is a large-scale distributed machine learning task involving multiple computing nodes distributed in different intelligent computing centers. These nodes need to frequently exchange and synchronize data for model training. However, due to the complexity and dynamics of the network environment, traditional collective communication algorithms may not achieve optimal performance when facing different network conditions, resulting in extended training time and waste of resources. Since the traffic of model training has a certain regularity, the collective communication algorithm can be optimized based on reinforcement learning, and thus the present application proposes a method for optimizing collective communication strategies based on reinforcement learning.

[0057] The collective communication method provided in the embodiment of the present application can be applied to the application environment as shown in Figure 1. Among them, the terminal 102 communicates with the intelligent computing network 104 through the network, and the intelligent computing network 104 includes multiple intelligent computing centers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers, Internet of Things devices, etc. The terminal 102 obtains the network status information of each intelligent computing center in the intelligent computing network 104; the network status information is processed by decision-making through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center; according to the collective communication strategy, the data interaction between each intelligent computing center is controlled. In this way, the collective communication algorithm of each intelligent computing center in the intelligent computing network is optimized, the time occupied by collective communication is reduced, and the model training rate across intelligent computing centers is improved.

[0058] In one embodiment, as shown in FIG. 2 , a collective communication method is provided, which is described by taking the method applied to the terminal 102 in FIG. 1 as an example, and includes the following steps:

[0059] Step S210: Obtain network status information of each intelligent computing center in the intelligent computing network.

[0060] Among them, the network status information is used to characterize the performance of the network, which may include network bandwidth usage, latency, packet loss rate, GPU communication time, and the amount of parameters transmitted.

[0061] In a specific implementation, the network status information of each intelligent computing center in the intelligent computing network can be monitored through a network monitoring tool and sent to the terminal, so that the terminal can obtain the network status information of each intelligent computing center.

[0062] Step S220, making decisions on the network status information through the pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center.

[0063] The pre-trained reinforcement learning model is trained by using the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function. Reinforcement Learning is a machine learning technology that learns the optimal behavior strategy through interaction with the environment.

[0064] It can be understood that the intelligent computing center undertakes massive data processing and complex computing tasks, and the collective communication algorithm plays a key role in it. Therefore, it is necessary to optimize the collective communication algorithm because it directly affects the data transmission efficiency and resource utilization efficiency of the intelligent computing center. Therefore, in order to reduce data transmission delays, improve throughput, and ensure smoother data synchronization and collaboration between different computing nodes. This application proposes a method for optimizing the collective communication algorithm using reinforcement learning, taking the network state as the state space, the data transmission mode (such as data transmission batch size, communication protocol, data compression level, etc.) as the behavior, that is, the action space, and the training performance of the intelligent computing network for model training (affected by network performance indicators (such as throughput, delay, etc.)) as the reward function, and training the reinforcement learning model. The collective communication strategy of each intelligent computing center is determined by the pre-trained reinforcement learning model.

[0065] In a specific implementation, the network status information of each intelligent computing center in the intelligent computing network can be input into a pre-trained reinforcement learning model, which is obtained through interactive training with the network environment. Therefore, an optimized collective communication strategy for each intelligent computing center can be obtained. The collective communication strategy can include communication control parameters such as the data transmission order and data transmission rate between each intelligent computing center.

[0066] Step S230, controlling data interaction between various intelligent computing centers according to the collective communication strategy.

[0067] In a specific implementation, after the collective communication strategy is generated, the data interaction between various intelligent computing centers can be controlled according to the communication control parameters in the collective communication strategy.

[0068] In this method, the monitoring frequency can be set regularly or in other ways. When the network status of each intelligent computing center is monitored to change, the collective communication strategy of each intelligent computing center will be adaptively updated to control each intelligent computing center to communicate in a timely manner according to the collective communication strategy that conforms to its network status, thereby reducing the time occupied by collective communication, improving the efficiency of collective communication, and thus increasing the training rate of model training across computing centers.

[0069] In the above collective communication method, a reinforcement learning model is pre-trained with network state information as the state space and the training performance of the intelligent computing network for model training as the reward function. The reinforcement learning model is used to generate a collective communication strategy for each intelligent computing center based on the network state information of each intelligent computing center in the intelligent computing network to control the data interaction between each intelligent computing center. This method applies the reinforcement learning algorithm to the determination of the collective communication strategy, and generates the collective communication strategy based on the network state information of each intelligent computing center, so that the collective communication strategy can dynamically adapt to the network state of the intelligent computing network, thereby ensuring that each intelligent computing center can interact with data in the optimal communication mode, improving the efficiency of collective communication, reducing network congestion, and thereby improving the training rate of model training across computing centers.

[0070] In an exemplary embodiment, as shown in FIG3 , the training steps of the pre-trained reinforcement learning model include:

[0071] Step S310, defining the state space, action space and reward function of the reinforcement learning model, and initializing the model parameters;

[0072] Step S320: In the current state, select an action corresponding to the action space according to the strategy of the reinforcement learning model, and execute the action to obtain the next state and feedback information of the intelligent computing network;

[0073] Step S330, calculating a reward value according to the feedback information and the reward function;

[0074] Step S340, updating the model parameters of the reinforcement learning model according to the reward value until the training end condition is reached, thereby obtaining a pre-trained reinforcement learning model.

[0075] In the specific implementation, before training the reinforcement learning model, the environment of the reinforcement learning model is first defined, including the state space, action space, reward function and time interval.

[0076] The state space is defined as network state information, which may include factors such as current network bandwidth utilization, computational load of each node, data transmission delay, and the scale of the ongoing collective communication task. This information can be used as the dimension of the state space to form a vector representation of the state. For example, the state can be represented by a vector: S = (bandwidth utilization, computational load 1, computational load 2...data transmission delay, task scale).

[0077] The action space is defined as the data transmission mode, and the action can be understood as the nemesis action performed in each state, which can include adjusting the data transmission batch size, selecting different communication protocols (such as TCP (Transmission Control Protocol) or UDP (User Datagram Protocol) different variants), adjusting the data compression level, etc. The action can also be expressed as a vector, for example: A = (batch size adjustment value, communication protocol selection, compression level adjustment value).

[0078] Among them, the reward function is used to evaluate the effect of each action. The reward function can be designed to be related to the training performance (such as training efficiency, stability, etc.) of the intelligent computing network for model training.

[0079] The time interval is used to determine the frequency of status updates. For example, the status can be updated every 5 seconds, with a time interval of T = 5s. This parameter needs to be adjusted according to the dynamic change speed of the network environment and the bearing capacity of computing resources. Too fast may lead to waste of computing resources, and too slow may fail to capture environmental changes in time.

[0080] After completing the definition of the environment of the reinforcement learning model, the model parameters can be initialized, including the weights of the neural network (if the algorithm uses a neural network), the learning rate, the discount factor, etc. The learning rate can be initially set to a relatively large value, such as 0.001, and then adjusted according to the convergence during the training process. The discount factor is generally between [0.9, 1], which determines the degree of influence of future rewards on current decisions, and is determined according to the time span of the task and the timeliness of the reward.

[0081] Further, model training can be performed. Next, select an action according to the reinforcement learning model’s strategy , and execute the action. Observe the feedback from the environment and get the next state The reward value is calculated based on the feedback information and reward function. .

[0082] Will( , , , ) are stored in the experience replay buffer. The size of the experience replay buffer can be set according to the computing resources and the diversity of training data. Generally, it can be set to several thousand to tens of thousands of experience samples. A batch of samples are randomly selected from the experience replay buffer to train the neural network of the reinforcement learning model. The parameters of the neural network are updated by minimizing the loss function, making the Q value estimation more accurate.

[0083] Repeat the above steps and continuously update the strategy of the reinforcement learning model until the training end condition is reached, such as the reward value converges or the training time reaches the upper limit. The training is ended and the pre-trained reinforcement learning model is obtained. The entire iterative training process is shown in Figure 4.

[0084] In this embodiment, through the precise state and action definition of the reinforcement learning model, the intelligent reward mechanism and the continuous strategy optimization, efficient data transmission and resource management can be achieved in a complex intelligent computing network environment, greatly improving network performance.

[0085] In an exemplary embodiment, the feedback information includes the throughput of the intelligent computing network for model training, the amount of data affected by network delay, and the amount of data affected by resource waste;

[0086] The above step S330 calculates the reward value according to the feedback information and the reward function, including: calculating the total data volume affected by network delay and the data volume affected by resource waste based on the reward function; calculating the difference between the throughput and the total data volume to obtain the reward value.

[0087] In the specific implementation, the reward function can be defined as a function of the throughput of model training (the amount of data processed per unit time) and the network delay and resource waste cost. It can be specifically expressed as:

[0088]

[0089] Among them, R represents the reward value, α and β are weight coefficients, which can be adjusted according to actual conditions.

[0090] When calculating the reward value based on this reward function, first add the amount of data affected by network delay and the amount of data affected by resource waste to get the total amount of data affected by both. Then subtract the throughput of model training from the total amount of data to get the reward value.

[0091] In this embodiment, the reward value is calculated by considering the throughput of model training, the amount of data affected by network delays, and the amount of data affected by resource waste. The obtained reward value not only reflects the basic performance of the network, but also takes factors such as delays and resource waste into consideration, so that the decision-making strategy of the reinforcement learning model based on this training is more accurate and comprehensive, thereby improving the effect of the subsequent collective communication strategy generated based on the reinforcement learning model.

[0092] It is understandable that because cross-region interconnection takes a long time, long-distance communication should be avoided as much as possible when designing the strategy of the reinforcement learning model. Because the collective communication time is related to the number of communicating nodes, it is necessary to consider both the intra-domain delay affected by the number of computing nodes in each intelligent computing center and the delay of long-distance transmission. For ease of explanation, the collective communication of intelligent computing centers in two different regions during the model training process is used as an example to illustrate the collective communication algorithm rules.

[0093] In an exemplary embodiment, each intelligent computing center includes multiple computing nodes; the strategy of the reinforcement learning model includes: for any two intelligent computing centers, obtaining the difference in the number of computing nodes in the two intelligent computing centers; if the difference in the number is less than a first threshold, controlling the two intelligent computing centers to first perform internal collective communication, and then perform collective communication between the two intelligent computing centers.

[0094] The computing node may be a GPU node.

[0095] In the specific implementation, the number of computing nodes in each intelligent computing center can be obtained in advance. For any two intelligent computing centers, if the difference in the number of computing nodes between the two intelligent computing centers is less than the first threshold, that is, the number of computing nodes between the two intelligent computing centers is similar, the two intelligent computing centers are controlled to first perform internal collective communication, and then perform long-distance collective communication between the two intelligent computing centers.

[0096] It can be understood that when the number of computing nodes in two intelligent computing centers is similar, the time difference between the two intelligent computing centers for collective communication is small. Therefore, the two intelligent computing centers are first allowed to conduct internal collective communication so that the data can be merged and integrated, and then long-distance collective communication is conducted between the two intelligent computing centers. This can reduce the amount of data that needs to be transmitted over long distances, reduce the burden of cross-center communication, reduce overall delays, and improve overall communication efficiency.

[0097] In another situation, if the difference in the number of computing nodes in two intelligent computing centers is greater than the second threshold, the intelligent computing center with fewer computing nodes in the two intelligent computing centers is controlled to first perform internal collective communication and synchronize the execution results to the other intelligent computing center; and the other intelligent computing center is controlled to perform internal collective communication.

[0098] In the specific implementation, for any two intelligent computing centers, if the difference in the number of computing nodes of the two intelligent computing centers is greater than the first threshold, that is, the number of computing nodes of the two intelligent computing centers is greatly different, then the intelligent computing center with fewer computing nodes should first perform internal collective communication, and then synchronize the execution results to the other intelligent computing center with more computing nodes through long-distance collective communication. After that, the other intelligent computing center with more computing nodes will perform collective communication again.

[0099] It can be understood that when the number of computing nodes in the two intelligent computing centers is unbalanced, the node with fewer computing nodes can complete the initial processing faster. If the two intelligent computing centers perform collective communication separately, they need to wait for the intelligent computing center with more computing nodes to complete the processing before executing long-distance collective communication. Then the intelligent computing center with fewer computing nodes will need to wait for a longer time, thereby increasing the delay. Therefore, in this embodiment, when the number of nodes in the two intelligent computing centers is far apart, the intelligent computing center with fewer computing nodes is first made to perform internal collective communication processing, and then the execution results are synchronized to the center with more nodes in a timely manner through long-distance collective communication. In this way, the intelligent computing center with more computing nodes only needs to wait for less time, thereby avoiding long-term occupation of network resources.

[0100] In the above embodiments, by optimizing the communication strategy according to the difference in the number of computing nodes, the overall performance and efficiency of the intelligent computing network can be effectively improved. When the number of nodes is similar, internal communication is prioritized to reduce latency and improve throughput; when the number of nodes is different, a smaller number of nodes are processed first, and resources and network loads are used reasonably, ultimately achieving efficient long-distance data synchronization and computing task execution. In this way, the system can not only respond quickly, but also maintain efficient resource utilization.

[0101] In an exemplary embodiment, the strategy of the reinforcement learning model also includes: when setting a data parallel mode for the intelligent computing network, setting different intelligent computing centers to perform processing tasks for different data sets.

[0102] Data Parallelism (DP) means that each computing node runs the same model code, and the data set is split into multiple parts and assigned to different computing nodes for training.

[0103] It can be understood that in the data parallel mode, large data sets are usually divided into multiple small blocks, and these small blocks are assigned to different computing nodes for processing. Each node performs calculations independently, and the final results need to be aggregated through collective communication. When different computing nodes are physically far apart, long-distance communication will lead to high latency and low bandwidth utilization. If collective communication involves long-distance message transmission across data centers or regions, it may lead to performance bottlenecks. Therefore, in order to reduce communication latency and improve efficiency, when designing data parallel solutions, priority should be given to how to reduce or optimize these long-distance communications.

[0104] Specifically, this embodiment adopts a method of matching the data partitioning method with the physical layout of computing resources, for example, distributing data nodes in each intelligent computing center instead of dividing them across multiple intelligent computing centers. In other words, each computing node in each intelligent computing center processes the same data set, and collective communication convergence is performed within the intelligent computing center, so as to avoid computing nodes in different intelligent computing centers from performing training tasks for the same data set.

[0105] In this embodiment, when setting the data parallel mode for the intelligent computing network, different intelligent computing centers are set to perform processing tasks for different data sets, which can avoid long-distance message transmission across data centers. By reducing long-distance communications to reduce delays in data transmission, computing nodes can complete tasks more efficiently, thereby speeding up the overall computing rate.

[0106] In one embodiment, in order to facilitate those skilled in the art to understand the embodiments of the present application, the following will be described in conjunction with specific examples of the accompanying drawings. Referring to FIG5, a schematic diagram of an algorithm architecture for optimizing collective communication based on reinforcement learning in an intelligent computing network environment is shown, including: a state monitoring module, a data acquisition module, a reinforcement learning module, a strategy generation module, a collective communication module, and a data transmission module, wherein:

[0107] Status monitoring module: used to monitor the network status of each intelligent computing center, including key indicators such as network bandwidth usage, latency, packet loss rate, GPU communication time, and the amount of parameters transmitted during model training.

[0108] Data acquisition module: used to obtain network status information from the status monitoring module and pass it to the reinforcement learning module.

[0109] Reinforcement learning module: Based on the network status information transmitted by the data acquisition module, the reinforcement learning algorithm is used for learning and decision-making, the state space, action space and reward function are defined, the strategy is continuously optimized through interaction with the network environment, and the decision results are output.

[0110] Strategy generation module: Generates the optimization strategy of collective communication based on the decision results output by the reinforcement learning module.

[0111] Collective communication module: executes the collective communication strategy generated by the strategy generation module, responsible for data transmission and synchronization between different intelligent computing centers. Adjusts communication parameters such as data transmission rate and data transmission order according to the optimization strategy provided by the strategy generation module.

[0112] Data transmission module: intelligently controls data transmission according to the data transmission rate strategy provided by the strategy generation module. When the network status changes, the module adjusts the transmission behavior according to the new strategy.

[0113] Intelligent computing center network environment: Multiple intelligent computing centers together form a cross-intelligent computing center network environment to execute computing tasks and interact with data.

[0114] It can be understood that efficient collective communication algorithms are crucial for tasks such as large-scale distributed computing and deep learning model training. They can accelerate the computing process, improve the overall performance and service quality of the intelligent computing center, meet the growing computing needs, maintain advantages in fierce technological competition, and provide stronger support for the digital transformation and innovation of various industries. Therefore, the application method of the reinforcement learning algorithm provided in this embodiment in optimizing the collective communication algorithm can promote the development of intelligent computing network technology, improve network performance and stability, and provide more reliable network support for fields such as big data processing and intelligent decision-making.

[0115] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0116] Based on the same inventive concept, the embodiment of the present application also provides a collective communication device for implementing the collective communication method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more collective communication device embodiments provided below can refer to the limitations on the collective communication method above, and will not be repeated here.

[0117] In one embodiment, as shown in FIG6 , a collective communication device is provided, including:

[0118] The information acquisition module 610 is used to obtain the network status information of each intelligent computing center in the intelligent computing network;

[0119] A strategy determination module 620 is used to make decisions on the network state information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center; wherein the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function;

[0120] The collective communication module 630 is used to control the data interaction between various intelligent computing centers according to the collective communication strategy.

[0121] In one embodiment, the device also includes a model training module, which is used to define the state space, action space and reward function of the reinforcement learning model, and initialize the model parameters; in the current state, an action corresponding to the action space is selected according to the strategy of the reinforcement learning model, and the action is executed to obtain the next state and feedback information of the intelligent computing network; based on the feedback information and the reward function, the reward value is calculated; according to the reward value, the model parameters of the reinforcement learning model are updated until the training end conditions are met, and the pre-trained reinforcement learning model is obtained.

[0122] In one embodiment, the feedback information includes the throughput of the intelligent computing network for model training, the amount of data affected by network delays, and the amount of data affected by resource waste; the model training module is also used to calculate the total amount of data affected by network delays and the amount of data affected by resource waste based on the reward function; the difference between the throughput and the total amount of data is calculated to obtain the reward value.

[0123] In one of the embodiments, each intelligent computing center includes multiple computing nodes; the strategy of the reinforcement learning model includes: for any two intelligent computing centers, obtaining the difference in the number of computing nodes in the two intelligent computing centers; if the difference in the number is less than a first threshold, controlling the two intelligent computing centers to first perform internal collective communication, and then perform collective communication between the two intelligent computing centers.

[0124] In one of the embodiments, the strategy of the reinforcement learning model also includes: if the quantity difference is greater than a second threshold, controlling the intelligent computing center with a smaller number of computing nodes in the two intelligent computing centers to first perform internal collective communication, and synchronizing the execution results to the other intelligent computing center; controlling the other intelligent computing center to perform internal collective communication.

[0125] In one of the embodiments, the strategy of the reinforcement learning model also includes: when setting a data parallel mode for the intelligent computing network, setting different intelligent computing centers to perform processing tasks for different data sets.

[0126] Each module in the above collective communication device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0127] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a collective communication method is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, a trackball or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0128] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0129] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0130] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0131] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0132] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0133] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0134] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0135] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A collective communication method, characterized in that: The method comprises: Obtain network status information of each intelligent computing center in the intelligent computing network; The network status information is processed by decision making through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center; According to the collective communication strategy, control the data interaction between the various intelligent computing centers; Among them, the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training.

2. The method according to claim 1, characterized in that The training process of the pre-trained reinforcement learning model includes: Define the state space, action space, and reward function of the reinforcement learning model, and initialize the model parameters; In the current state, an action corresponding to the action space is selected according to the strategy of the reinforcement learning model, and the action is executed to obtain the next state and feedback information of the intelligent computing network; Calculating a reward value according to the feedback information and the reward function; The model parameters of the reinforcement learning model are updated according to the reward value until a training end condition is reached to obtain the pre-trained reinforcement learning model.

3. The method according to claim 2, characterized in that The feedback information includes the throughput of the intelligent computing network for model training, the amount of data affected by network delay, and the amount of data affected by resource waste; The step of calculating the reward value according to the feedback information and the reward function includes: Calculate the total amount of data affected by the network delay and the amount of data affected by the resource waste based on the reward function; The difference between the throughput and the total data volume is calculated to obtain the reward value.

4. The method according to any one of claims 1 to 3, characterized in that: Each of the intelligent computing centers includes multiple computing nodes; the strategy of the reinforcement learning model includes: For any two intelligent computing centers, obtaining the difference between the numbers of computing nodes in the two intelligent computing centers; If the quantity difference is smaller than a first threshold, the two intelligent computing centers are controlled to first perform internal collective communication respectively, and then perform collective communication between the two intelligent computing centers.

5. The method according to claim 4, characterized in that The strategy of the reinforcement learning model also includes: If the number difference is greater than a second threshold, controlling the intelligent computing center with fewer computing nodes in the two intelligent computing centers to first perform internal collective communication, and synchronize the execution result to the other intelligent computing center; Control the other intelligent computing center to perform internal collective communication.

6. The method according to any one of claims 1 to 3, characterized in that: The strategy of the reinforcement learning model also includes: When setting the data parallel mode for the intelligent computing network, different intelligent computing centers are set to perform processing tasks for different data sets.

7. A collective communication device, characterized in that: The device comprises: An information acquisition module is used to obtain network status information of each intelligent computing center in the intelligent computing network; A strategy determination module, used to perform decision processing on the network state information through a pre-trained reinforcement learning model to obtain a collective communication strategy for each intelligent computing center; wherein the pre-trained reinforcement learning model is obtained by taking the network state information as the state space and defining the training performance of the intelligent computing network for model training as the reward function training; The collective communication module is used to control the data interaction between the various intelligent computing centers according to the collective communication strategy.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the collective communication method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the collective communication method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the collective communication method according to any one of claims 1 to 6 are implemented.

Images