Data communication optimization methods, apparatus and systems

Abstract

This specification provides a data communication optimization method, apparatus, and system. The method includes: determining a data processing request; determining the data processing communication volume of the data to be processed and the corresponding communication group selection threshold based on a target set communication algorithm; and determining a target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold. This method selects a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. This allows for the selection of a suitable target communication group for data processing based on different data processing communication volumes and corresponding communication group selection thresholds. When applied to distributed training scenarios of deep learning models, this data communication optimization method can significantly improve communication performance and model training efficiency during model training.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of computer technology, and in particular to a data communication optimization method. Background Technology

[0002] Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. Its ultimate goal is to enable machines to possess analytical and learning capabilities like humans, allowing them to recognize data such as text, images, and sound. In distributed training, the workload for training the model is split and shared among multiple microprocessors called worker nodes. These worker nodes work in parallel to accelerate model training.

[0003] In the field of artificial intelligence, the performance of distributed training of deep learning models is mainly constrained by different network environments and communication algorithms. These different network environments and algorithms primarily refer to different network media or communication protocols. For example, there's Remote Direct Memory Access (RDMA) communication in InfiniBand networks, and socket communication in TCP / IP networks. Specifically, communication performance indicators mainly focus on the effectiveness and reliability of transmitted information. Effectiveness refers to the amount of channel resources consumed in transmitting a certain amount of information. Effectiveness indicators include channel bandwidth and latency. Bandwidth is the primary factor in scenarios with large data volumes, while latency is more sensitive in scenarios with small data volumes. Communication performance varies depending on the amount of data transmitted.

[0004] In a specific distributed training process, there are multiple communications, and the amount of data in each communication is often different. If the same communication processing method is used for all communication data of different sizes, it is impossible to guarantee optimal global communication. This results in poor communication performance and low training efficiency of the model in a specific distributed training process. Summary of the Invention

[0005] In view of this, embodiments of this specification provide a data communication optimization method. One or more embodiments of this specification also relate to a data communication optimization apparatus, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the embodiments of this specification, a data communication optimization method is provided, comprising:

[0007] A data processing request is determined, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed;

[0008] Based on the target set communication algorithm, the data processing communication volume of the data to be processed and the corresponding communication group selection threshold are determined;

[0009] Based on the data processing communication volume and the communication group selection threshold, a target communication group is determined from at least two communication groups, wherein the at least two communication groups correspond to the set communication algorithm;

[0010] The data processing request is executed according to the target communication group.

[0011] According to a second aspect of the embodiments of this specification, a data communication optimization apparatus is provided, comprising:

[0012] The request determination module is configured to determine a data processing request, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed;

[0013] The data determination module is configured to determine the data processing communication volume of the data to be processed and the corresponding communication group selection threshold based on the target set communication algorithm.

[0014] A communication group determination module is configured to determine a target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold, wherein the at least two communication groups correspond to the aggregate communication algorithm;

[0015] The request execution module is configured to execute the data processing request based on the target communication group.

[0016] According to a third aspect of the embodiments of this specification, a data communication optimization system is provided, including an adaptation node, an algorithm node, a prediction node, and a scheduling node, wherein:

[0017] The adapter node is configured to determine a data processing request sent from a network node, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed;

[0018] The algorithm node is configured to determine the target set communication algorithm and, based on the target set communication algorithm, determine the data processing communication volume of the data to be processed.

[0019] The scheduling node is configured to determine a target communication group from at least two communication groups based on the data processing traffic and the communication group selection threshold, wherein the at least two communication groups correspond to the target set communication algorithm, and the at least two communication groups and the communication group selection threshold are determined by the prediction node.

[0020] The scheduling node is also configured to execute the data processing request based on the target communication group.

[0021] According to a fourth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0022] Memory and processor;

[0023] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data communication optimization method.

[0024] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data communication optimization method described above.

[0025] According to a sixth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data communication optimization method.

[0026] An embodiment of this specification provides a data communication optimization method, comprising: determining a data processing request, wherein the data processing request carries data to be processed and a set communication algorithm corresponding to the data to be processed; determining, according to the set communication algorithm, the data processing communication volume of the data to be processed and a corresponding communication group selection threshold; determining a target communication group from at least two communication groups according to the data processing communication volume and the communication group selection threshold, wherein the at least two communication groups correspond to the set communication algorithm; and executing the data processing request according to the target communication group.

[0027] Specifically, this method selects a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. It can select a suitable target communication group for data processing according to different data processing communication volumes and corresponding communication group selection thresholds. This data communication optimization method can be applied to the distributed training scenario of deep learning models, which can greatly improve the communication performance and model training efficiency during the model training process. Attached Figure Description

[0028] Figure 1 This is a schematic diagram illustrating a specific scenario of a data communication optimization method provided in one embodiment of this specification;

[0029] Figure 2 This is a flowchart illustrating a data communication optimization method provided in one embodiment of this specification;

[0030] Figure 3 This is a flowchart illustrating the specific processing steps of a data communication optimization method provided in one embodiment of this specification.

[0031] Figure 4 This is an architecture diagram of a data communication optimization system provided in one embodiment of this specification;

[0032] Figure 5 This is a schematic diagram of the structure of a data communication optimization device provided in one embodiment of this specification;

[0033] Figure 6 This is a schematic diagram of the structure of a data communication optimization system provided in one embodiment of this specification;

[0034] Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0035] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0036] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0037] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0038] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0039] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0040] CPU (Central Processing Unit): As the core of a computer system for computation and control, it is the final execution unit for information processing and program execution. Currently, CPU clusters are used in fields such as recommendation, advertising, and search for artificial intelligence training tasks, which can ensure high cost-effectiveness.

[0041] GPU (Graphics Processing Unit): Also known as display core, visual processor, or display chip, it is a microprocessor specifically designed to perform image and graphics-related calculations on personal computers, workstations, game consoles, and some mobile devices (such as tablets and smartphones). It is currently widely used to accelerate artificial intelligence.

[0042] IaaS (Infrastructure As a Service): Cloud services that provide infrastructure such as computing, storage, and networking.

[0043] LL128 (Low Latency): A low-latency protocol where every 128 bytes transmitted consists of 8 bytes for the sign bit and 120 bytes for the data bits.

[0044] Superior solution: In all design spaces of the feasible region, the objective function corresponding to solution A is better than any other feasible solution.

[0045] Pareto (Pareto Optimality) is a better solution because in reality, multi-objective functions are essentially chaotic and it is impossible for a globally better solution to exist. For solution A, no other solution in the variable space can be found that is better than solution A (meaning that both objective functions are better than the function values ​​corresponding to A).

[0046] API (Application Programming Interface): A set of predefined functions designed to provide applications and developers with the ability to access a set of routines based on certain software or hardware, without needing to access the source code or understand the details of the internal workings.

[0047] Comm (Com manager, communication management module): This refers to the instantiation of a specific communication library class object. This involves the low latency optimization protocol LL128 and the bandwidth optimization protocol simple, etc. Among them, simple (SIP for Instant Messaging and Presence Leveraging Extensions) is an instant messaging communication protocol family based on the SIP (session initialization protocol).

[0048] Bw (Band width): refers to the amount of data that can be transmitted in a fixed amount of time, that is, the data transmission capacity in the transmission channel.

[0049] Lat (Latency): The time taken for a client to complete an I / O operation. I / O is an abbreviation for Input / Output, which is essentially the process by which the computer schedules the writing and output of data in various storage systems (including memory and external storage).

[0050] Comm_bw: Bandwidth-optimized communication group, used to build communication groups for bandwidth optimization in the simple protocol.

[0051] Comm_lat: Delay-optimized communication group, used to build a communication group for delay optimization in the LL128 protocol.

[0052] Communication performance metrics primarily focus on the effectiveness and reliability of transmitted information. Effectiveness refers to the amount of channel resources consumed in transmitting a given amount of information. Channel resources include bandwidth and latency. Bandwidth is the primary factor in scenarios with large data volumes, while latency is more sensitive in scenarios with small data volumes. Communication performance varies depending on the data volume. To address this, existing communication libraries in the industry have added two different algorithms: low-latency optimization and bandwidth optimization. However, these libraries can only use one specific algorithm at a time. In a specific distributed training process, the amount of communication per instance often varies. Using the same algorithm for each instance of distributed training cannot guarantee optimal performance. Therefore, achieving a globally consistent, optimal latency and bandwidth distributed training algorithm across different device models and network topologies has become a challenge.

[0053] In practical implementation, the above-mentioned problems in the embodiments of this specification can be abstracted into a multi-objective function problem:

[0054] That is, Min(Perf(S1_AlgoType1, Comm1)+Perf(S2_AlgoType2, Comm2)+Perf(Sn_AlgoTypen, Commn),n in{1:N};

[0055] Where N represents the number of communications involved in one training iteration.

[0056] Size represents the amount of communication, which can be any amount of communication during a single training session.

[0057] AlgoType represents a specific collection communication algorithm, such as allreduce, allgather, etc.

[0058] Min represents the minimum value.

[0059] The Perf(performance) method name represents the performance statistics results.

[0060] Perf(S1_AlgoType1, Comm1) represents the communication volume 1 corresponding to the statistical set communication algorithm 1 and the performance of the selected communication group 1; similarly, the specific explanations of Perf(S2_AlgoType2, Comm2) and Perf(Sn_AlgoTypen, Commn) are similar.

[0061] n in{1:N} means n ranges from 1 to N.

[0062] Based on this multi-objective function problem, this specification provides a data communication optimization method. Specifically, this method selects a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. This allows for the selection of a suitable target communication group for data processing based on different data processing communication volumes and the corresponding communication group selection threshold. When applied to distributed training scenarios of deep learning models, this data communication optimization method can greatly improve the communication performance and model training efficiency during the model training process.

[0063] The theoretical basis of the data communication optimization method provided in the embodiments of this specification can be derived from the simplified solution of the Pareto optimal solution. For example, the Pareto optimal solution only needs to find a relatively better solution set of communication groups (Comm_hybridPayload) of the hybrid protocol stack. Combining the empirical basis that multiple communication groups have no additional performance (overhead), the data communication optimization method provided in the embodiments of this specification only needs to be simplified to solving for two types of communication groups, Comms, namely bandwidth-sensitive and latency-sensitive. The specific implementation formula is as follows:

[0064] Min(Perf(SN_AlgoTypeN,Comm_hybridPayload)=Min(Perf(Sm_AlgoTypem,Comm_bw)+Min(Perf(Sn_AlgoTypen,Comm_lat)

[0065] m+n=N

[0066] Here, Perf(SN_AlgoTypeN, Comm_hybridPayload) represents the communication volume N corresponding to the statistical set communication algorithm N, and the performance of the communication group corresponding to the hybrid protocol stack; Perf(Sm_AlgoTypem, Comm_bw) represents the communication volume m corresponding to the statistical set communication algorithm m, and the performance of the communication group corresponding to the bandwidth optimization; similarly, Perf(Sn_AlgoTypen, Comm_lat) represents the communication volume n corresponding to the statistical set communication algorithm n, and the performance of the communication group corresponding to the latency optimization.

[0067] m represents the number of times the bandwidth-optimized communication group is selected, n represents the number of times the latency-optimized communication group is selected, and N represents the number of communications involved in one training iteration, which is the sum of the number of times the bandwidth-optimized communication group and the number of times the latency-optimized communication group is selected.

[0068] This specification provides a data communication optimization method, and also relates to a data communication optimization device, a data communication optimization system, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0069] See Figure 1 , Figure 1 The illustration shows a specific scenario in which a data communication optimization method according to an embodiment of this specification is applied to the distributed training of a deep learning model.

[0070] Figure 1 The system includes a client 102 and a server 104. The client 102 can be understood as a laptop, desktop computer, tablet computer, etc., and the server 104 can be understood as a physical server or cloud server. For ease of understanding, the following embodiments will be described in detail using the client 102 as a desktop computer and the server 104 as a physical server.

[0071] In practice, the client 102 sends a data processing request to the server 104. This data processing request can be understood as a model training request for a target deep learning model (such as an image classification model or a text recognition model), and the data processing request carries the training data corresponding to the training of the target deep learning model.

[0072] After receiving the data processing request, server 104 determines the target communication group for executing the data processing request based on its deployed data communication optimization system, and executes the data processing request according to the target communication group, so as to achieve efficient distributed training of the target deep learning model through better communication performance. The data communication optimization system includes adaptation nodes, algorithm nodes, scheduling nodes, prediction nodes, and network nodes, and each node provides different communication functions. Each node can be understood as a software unit in a device.

[0073] In practical applications, after the server 104 performs distributed training on the target deep learning model, it can return the trained target deep learning model to the client 102.

[0074] The following is in conjunction with the appendix Figure 2 The data communication optimization method described above will be further explained. Figure 2 A flowchart of a data communication optimization method provided in one embodiment of this specification is shown, which specifically includes the following steps.

[0075] Step 202: Determine the data processing request.

[0076] The data processing request carries the data to be processed and the target set communication algorithm corresponding to the data to be processed.

[0077] One or more data communication optimization methods provided in the embodiments of this specification are applied to a data communication optimization system, and these data communication optimization methods can be applied to the distributed training of deep learning models to improve their communication performance and training efficiency. Of course, in practical applications, these data communication optimization methods can be applied not only to the distributed training of deep learning models, but also to any applicable scenario, such as other distributed scenarios: distributed data reading and writing scenarios, distributed data storage scenarios, etc.

[0078] Specifically, the data processing request varies depending on the specific application scenario of this data communication optimization method. For example, if the data communication optimization method is applied to a distributed training scenario for deep learning models, the data processing request can be understood as a model training request for the deep learning model. If the data communication optimization method is applied to a distributed data reading and writing scenario, the data processing request can be understood as a data reading and writing request.

[0079] For ease of understanding, the following embodiments all use the application of this data communication optimization method to the distributed training scenario of deep learning models, and the data processing request is taken as an example of the model training request for deep learning models, which will be described in detail below.

[0080] Therefore, determining the data processing request can be understood as determining the model training request for the target machine learning model.

[0081] The target machine learning model can be understood as the deep learning model mentioned above.

[0082] In the case where the data processing request is a model training request for a target machine learning model, the data to be processed can be understood as the training data of the target machine learning model, such as training samples and their corresponding training labels. The target set communication algorithm corresponding to this data includes, but is not limited to, all-reduce, all-gather, send, or recv algorithms. Set communication algorithms can be understood as a method for data transmission between different nodes. Specifically, distributed training is usually performed on a computing cluster, where each node performs a portion of the computation. There are data dependencies and sharing between the computations of different nodes, necessitating data transmission between them. Therefore, communication in distributed training is mostly based on set communication algorithms.

[0083] Step 204: Based on the target set communication algorithm, determine the data processing communication volume of the data to be processed and the corresponding communication group selection threshold.

[0084] The data processing communication volume of the data to be processed can be understood as the communication volume determined when processing the data to be processed according to the target set communication algorithm; the communication group selection threshold can be understood as the pre-calculated communication group selection threshold corresponding to the target set communication algorithm.

[0085] Therefore, after determining the data processing request, the data to be processed can be processed according to the target set communication algorithm carried in the data processing request, the data processing communication volume of the data to be processed can be determined, and the communication group selection threshold corresponding to the target set communication algorithm can be determined.

[0086] In practice, after receiving or acquiring a data processing request, the call chain of the original communication library corresponding to the data processing request is obtained. This allows subsequent invocation of the original set communication algorithm within the original communication library based on the call chain. By leveraging the symbol characteristics of the original communication library, the original set communication algorithm can be hijacked completely without any noticeable interference. This enables the subsequent determination of the target set communication algorithm from the hijacked original set communication algorithm. The specific implementation method is as follows:

[0087] After determining the data processing request, and before determining the data processing communication volume of the data to be processed and the corresponding communication group selection threshold according to the target set communication algorithm, the method further includes:

[0088] Obtain the call chain of the original communication library corresponding to the data processing request, and call the original set communication algorithm in the original communication library according to the call chain, wherein the target set communication algorithm is any one of the original set communication algorithms.

[0089] The original communication library can be understood as the nccl communication library, which can implement collective communication and point-to-point communication, focusing on communication with the GPU. This original communication library includes, but is not limited to, collective communication algorithms such as allreduce, allgather, send, or recv. The original communication library corresponding to the data processing request can be understood as the original communication library that can only process data processing requests using a specific collective communication algorithm. Then, the call chain can be understood as the chain of calls to the nccl communication library.

[0090] Taking the nccl communication library as an example, obtaining the call chain of the original communication library corresponding to the data processing request, and then calling the original set communication algorithm in the original communication library according to the call chain, can be understood as obtaining the call chain of the nccl communication library corresponding to the data processing request and calling the original set communication algorithm in the nccl communication library according to the call chain. For example, the call communication entry point of the nccl communication library can be hijacked for initialization, and the distributed training acceleration engine of the data communication optimization system applied by this data communication optimization method can be inserted to optimize the code.

[0091] In one or more embodiments of this specification, after obtaining the call chain of the original communication library corresponding to the data processing request, and calling the original set communication algorithm in the original communication library according to the call chain, multiple communication groups can be constructed based on the original set communication algorithm. The communication protocols in different communication groups are different. When processing the data to be processed, the communication performance and training efficiency when executing the data processing request vary depending on the execution of different communication groups. Therefore, constructing multiple communication groups facilitates the selection of different communication groups to achieve optimal data processing based on the data processing communication volume of different data to be processed. The specific implementation method is as follows:

[0092] After obtaining the call chain of the original communication library corresponding to the data processing request, and calling the original set communication algorithm in the original communication library according to the call chain, the method further includes:

[0093] If a set communication algorithm performance table is found to exist, the set communication algorithm performance table is invoked, and at least two communication groups are constructed based on the set communication algorithm performance table, wherein the communication protocols of each of the at least two communication groups are different.

[0094] The set communication algorithm performance table can be understood as a hash table. The hash table stores data consisting of keys and values. The keys in the set communication algorithm performance table involve the communication volume of the data to be processed, the set communication algorithm type, the data type, and the communication group type, while the values ​​are the specific communication performance.

[0095] Specifically, the performance table of the ensemble communication algorithm can be constructed using the formula perf_map[nNode_mPerNode_Msg_bw] = xxx. Here, nNode_mPerNode_Msg_bw is the key, representing the number of GPU nodes (node), with each node having mPerNode GPUs. msg represents the current communication volume, bw indicates that the communication group is of bandwidth-optimized type (bw can be replaced with lat to indicate that the communication group is of latency-optimized type), and xxx is the value, representing the performance data.

[0096] For example, Perf_map[nNode_mPerNode_Msg_allreduce_bw]=100;

[0097] Perf_map[nNode_mPerNode_Msg_allreduce_lat] = 200; that is, the performance of the communication group optimized by the allreduce algorithm for a certain target traffic is 100; the performance of the communication group optimized by the allreduce algorithm for a certain target traffic is 200.

[0098] Taking the set communication algorithms allreduce and allgather as examples, the performance table of the set communication algorithm corresponding to the above formula perf_map[nNode_mPerNode_Msg_bw]=xxx can be constructed as follows:

[0099] Perf_map[nNode_mPerNode_Msg_allreduce_bw]=xxx;

[0100] Perf_map[nNode_mPerNode_Msg_allreduce_lat]=xxx;

[0101] Perf_map[nNode_mPerNode_Msg_allgather_bw]=xxx;

[0102] Perf_map[nNode_mPerNode_Msg_allgather_lat]=xxx;

[0103] As can be seen from the above formula, the performance table of the constructed communication set algorithm can reflect the performance of the bw and lat communication groups corresponding to the allreduce algorithm, as well as the performance of the bw and lat communication groups corresponding to the allgather algorithm.

[0104] Of course, in practical applications, if other set communication algorithms exist, the above formula can be used to determine and update the performance table of the set communication algorithm.

[0105] A communication group can be understood as an instantiation of a specific communication library class object. For example, the at least two communication groups include:

[0106] Bandwidth-optimized communication group and latency-optimized communication group.

[0107] In the case that at least two communication groups are latency-optimized communication groups and bandwidth-optimized communication groups, the latency-optimized communication group will involve the low-latency optimization protocol LL128, and the bandwidth-optimized communication group will involve the bandwidth optimization protocol simple, etc. Among them, the bandwidth-optimized communication group is a communication group that optimizes communication for bandwidth, and the latency-optimized communication group is a communication group that optimizes communication for latency.

[0108] Among them, the bandwidth-optimized communication group is a communication group built for the nccl simple protocol to optimize bandwidth, and the delay-optimized communication group is a communication group built for the nccl LL128 protocol to optimize delay.

[0109] Specifically, the construction process for building at least two communication groups is as follows: based on various parameters of the data to be processed in the data processing request, such as communication volume and collective communication algorithm, the key in the collective communication algorithm performance table is provided; the value in the collective communication algorithm performance table is obtained based on the key; and at least two communication groups are constructed based on the key and value in the collective communication algorithm performance table. The key in the collective communication algorithm performance table involves the communication volume, collective communication algorithm type, data type, and communication group type of the data to be processed, and the value is the specific communication performance.

[0110] Specifically, taking perf_map[nNode_mPerNode_Msg] = xxx as an example, nNode_mPerNode_Msg is the key, representing node GPU nodes, each node has mPerNode GPUs, msg represents the current communication volume, and xxx is the value, representing performance data.

[0111] In one or more embodiments of this specification, if a performance table for the collective communication algorithm does not exist, it is necessary to construct multiple communication groups corresponding to the original collective communication algorithm based on the data to be processed, so that a suitable target communication group can be selected from multiple communication groups for optimal data processing. The specific implementation method is as follows:

[0112] After obtaining the call chain of the original communication library corresponding to the data processing request, and calling the original set communication algorithm in the original communication library according to the call chain, the method further includes:

[0113] If it is determined that there is no performance table for the set communication algorithm, at least two communication groups corresponding to the original set communication algorithm are constructed based on the data to be processed, wherein the communication protocols of each of the at least two communication groups are different.

[0114] In practical applications, if there is no performance table for the set communication algorithm, at least two communication groups corresponding to the original set communication algorithm are randomly constructed. That is, multiple communication groups are constructed under the execution parameters in the data processing request received by nccl. The execution parameters may include the data to be processed in the data processing request.

[0115] For example, taking the original set communication algorithms, including but not limited to allreduce, allgather, send, or recv, as an example, bandwidth-optimized communication groups and latency-optimized communication groups corresponding to the original set communication algorithms such as allreduce, allgather, send, or recv are constructed according to the data to be processed.

[0116] In one or more embodiments of this specification, if it is determined that there is no performance table for the collective communication algorithm, after constructing at least two communication groups based on the data to be processed, it is also necessary to construct a reasonable and accurate performance table for the collective communication algorithm by pre-testing (i.e., performing performance statistics) based on the data type of the data to be processed. This allows for the subsequent determination of the communication group selection threshold corresponding to each collective communication algorithm based on the performance table. The specific implementation method is as follows:

[0117] After constructing at least two communication groups based on the data to be processed, the process further includes:

[0118] Determine the data type of the data to be processed, and the preset communication volume set corresponding to the data type;

[0119] The target communication volume is determined sequentially from the preset communication volume set. The target communication volume is processed according to at least two communication groups corresponding to the original set communication algorithm. The execution performance set of at least two communication groups corresponding to the original set communication algorithm is determined. The set communication algorithm performance table is constructed according to the execution performance set.

[0120] The data type of the data to be processed can be understood as character type, floating-point type, etc.; then the preset communication volume set corresponding to the data type can be understood as a preset character type preset communication volume set or a preset floating-point type preset communication volume set. In addition, the communication volume in the preset communication volume set corresponding to any data type can be understood as any size from 8 bytes to 512 megabytes. For example, the preset communication volume set is: 8 bytes, 9 bytes... 512 megabytes, etc.

[0121] The target communication volume can be understood as any communication volume in the preset communication volume set, such as 8 bytes or 9 bytes. Taking the target communication volume as 8 bytes or 9 bytes, and at least two communication groups as the first communication group and the second communication group as examples, the target communication volume is determined sequentially from the preset communication volume set. The target communication volume is then processed according to the at least two communication groups corresponding to the original set communication algorithm. The execution performance set of the at least two communication groups corresponding to the original set communication algorithm is determined. This can be understood as follows: determining the target communication volume of 8 bytes from the preset communication volume set, executing the target communication volume of 8 bytes in the first and second communication groups corresponding to the original set communication algorithm, and determining the first execution performance of the first communication group and the second execution performance of the second communication group corresponding to the original set communication algorithm; similarly, determining the target communication volume of 9 bytes from the preset communication volume set, and using the above method, determining the third execution performance of the first communication group and the fourth execution performance of the second communication group corresponding to the original set communication algorithm; and determining the execution performance set through the first, second, third, and fourth execution performances.

[0122] The execution performance set includes the execution performance of each communication group corresponding to the original collective communication algorithm. Taking the structure of the above collective communication algorithm performance table as Perf_map[nNode_mPerNode_Msg_allreduce_bw]=xxx as an example, for instance, if a target communication volume of 8 bytes is determined from the preset communication volume set, and the target communication volume of 8 bytes is executed in the first communication group corresponding to the original collective communication algorithm, the first execution performance of the first communication group corresponding to the original collective communication algorithm can be determined. This can be understood as determining a target communication volume of 8 bytes from the preset communication volume set, executing the target communication volume of 8 bytes in the bandwidth-optimized communication group corresponding to the allreduce algorithm, and determining the first execution performance as XXX. The collective communication algorithm performance table can then be constructed based on the above information.

[0123] In practical applications, the data type of the data to be processed in the data processing request in the current communication scenario needs to be obtained. If there are multiple types, it is necessary to iterate through them.

[0124] For example, if the data to be processed has two data types, floating-point and character, then it is necessary to first iterate through the performance table of the set communication algorithm for the floating-point data type, and then iterate through the performance table of the set communication algorithm for the character data type. The loop iterates from a preset communication volume set. If the preset communication volume in the preset communication volume set exceeds the range of the preset communication volume set, the loop is exited. Otherwise, if the preset communication volume in the preset communication volume set is within the range of the preset communication volume set, a specific data volume of the current data type is simulated, and the bandwidth-optimized communication group and the delay-optimized communication group of the specified set communication algorithm are executed. The corresponding performance is statistically analyzed, the execution performance set is determined, and the set communication algorithm performance table is constructed or updated. For a specific implementation of the iteration, please refer to the example above.

[0125] Therefore, after determining the performance table of the set communication algorithms, the communication group selection threshold corresponding to each original set communication algorithm can be determined based on this table. This allows for the identification of the corresponding communication group selection threshold for the target set communication algorithm selected from the original set communication algorithms, enabling the selection of a suitable target communication group for subsequent data processing based on this threshold. The specific implementation method is as follows:

[0126] Based on the performance table of the set communication algorithm, determine the communication group selection threshold corresponding to the original set communication algorithm.

[0127] The communication group selection threshold can be understood as the communication performance boundary value of at least two communication groups corresponding to each original set communication algorithm. For example, if the original set communication algorithms are allreduce, allgather, send, and recv, the corresponding at least two communication groups are the bandwidth optimization communication group and the latency optimization communication group.

[0128] In practical applications, the performance table of the collective communication algorithm stores the communication group selection thresholds for different collective communication algorithms. The communication group selection threshold corresponding to each original collective communication algorithm can be determined. For example, the communication group selection threshold for the allreduce algorithm is 64 Mbps, the communication group selection threshold for the collection algorithm is 45 Mbps, the communication group selection threshold for the sending algorithm is 78 Mbps, and the communication group selection threshold for the receiving algorithm is 54 Mbps.

[0129] Based on this, the data communication optimization method of the embodiments of this specification can construct at least two communication groups and construct a set communication algorithm performance table, and determine a communication selection threshold so that the target communication group can be determined from at least two communication groups according to the data processing communication volume and the communication group selection threshold.

[0130] Step 206: Determine the target communication group from at least two communication groups based on the data processing traffic and the communication group selection threshold.

[0131] The target communication group can be understood as any one of the at least two communication groups. That is, when the data processing request is communicated through the target communication group, the communication performance will be better than when the data processing request is communicated through other communication groups among the at least two communication groups.

[0132] In one or more embodiments of this specification, given a determined data processing traffic volume and a communication group selection threshold, a target communication group corresponding to the data processing traffic volume can be quickly determined from at least two communication groups based on the communication group selection threshold. The specific implementation is as follows:

[0133] The step of determining the target communication group from at least two communication groups based on the data processing traffic and the communication group selection threshold includes:

[0134] The data processing traffic is compared with the communication group selection threshold, and the target communication group is determined from at least two communication groups based on the comparison result.

[0135] Taking a communication group selection threshold of 64 Mbps and at least two communication groups including a bandwidth-optimized communication group and a latency-optimized communication group as an example, the data processing communication volume is compared with the communication group selection threshold, and the target communication group is determined from at least two communication groups based on the comparison results.

[0136] Specifically, by comparing the data processing traffic volume with the communication group selection threshold, when the data processing traffic volume is 128 Mbps, it can be determined that the data processing traffic volume of 128 Mbps is greater than the communication group selection threshold of 64 Mbps. At this time, based on the comparison results, the bandwidth-optimized communication group can be selected as the target communication group from the bandwidth-optimized communication group and the delay-optimized communication group.

[0137] In the data communication optimization method provided in the embodiments of this specification, when the data processing communication volume and the communication group selection threshold are determined, the target communication group corresponding to the data processing communication volume can be quickly determined from at least two communication groups based on the communication group selection threshold.

[0138] Step 208: Execute the data processing request according to the target communication group.

[0139] In one or more embodiments of this specification, the data processing request is still taken as a model training request for the target machine learning model as an example. The specific implementation of executing the data processing request according to the target communication group is as follows:

[0140] The step of executing the data processing request according to the target communication group includes:

[0141] Based on the target communication group, execute the model training request for the target machine learning model.

[0142] The data communication optimization method provided in the embodiments of this specification, when applied to a distributed training scenario of a deep learning model, can execute a model training request for the target machine learning model according to the selected target communication group, so as to improve the communication performance and model training efficiency during the model training process.

[0143] Based on the detailed description of the above embodiments, the data communication optimization method will be explained in general with a specific example: For example, if the target set communication algorithm carried in the data processing request is the allreduce algorithm, a set communication algorithm performance table is determined to exist. The set communication algorithm performance table is called to construct a bandwidth-optimized communication group and a latency-optimized communication group. The data processing communication volume of the data to be processed determined by the reduction algorithm is 100 Mbps, and the corresponding communication group selection threshold is 64 Mbps. The data processing communication volume of the data to be processed is greater than the communication group selection threshold. Therefore, the bandwidth-optimized communication group is selected as the target communication group. The communication group object corresponding to the target communication group is called to execute the data processing request (i.e., to perform distributed model training of the target deep learning model). Then, the trained target deep learning model is returned to the client.

[0144] The data communication optimization method provided in this specification selects a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. This allows for the selection of a suitable target communication group for data processing based on the different data processing communication volumes and the corresponding communication group selection threshold. When applied to distributed training scenarios of deep learning models, this data communication optimization method can greatly improve the communication performance and model training efficiency during the model training process.

[0145] See Figure 3 , Figure 3 A flowchart illustrating the specific processing steps of a data communication optimization method according to an embodiment of this specification is shown.

[0146] Step 302: Enter the Python code to be delivered.

[0147] The Python code is user-side code, such as code for an artificial intelligence training scenario, which is the data to be processed in the above embodiment.

[0148] Step 304: API hijacking.

[0149] Specifically, API hijacking can be understood as hijacking the ncclCommInitRankFunc initialization symbol of the nccl communication library and inserting it into the distributed training acceleration engine of the data communication optimization system applied by the data communication optimization method to optimize the code.

[0150] Step 306: Determine if a cache exists. If yes, proceed to step 308; otherwise, proceed to step 312.

[0151] The Cache primarily stores the performance table of the collective communication algorithm under the current cluster size. The table structure is shown below, storing the communication volume corresponding to the communication group selection threshold (msg_gate) for different collective communication algorithms:

[0152] allreduce_msg_gate=msg_gate1;

[0153] allgather_msg_gate=msg_gate2;

[0154] send_msg_gate = msg_gate3;

[0155] recv_msg_gate = msg_gate4;

[0156] Let's take the first formula, allreduce_msg_gate = msg_gate1, as an example to explain the specific meaning of this formula. The specific meanings of the other formulas are similar to those of the first formula. The left-hand side, allreduce_msg_gate, represents the communication group selection threshold for the Allreduce algorithm, and the right-hand side is the specific threshold communication volume.

[0157] Step 308: Load cache.

[0158] Loadcache can be understood as calling the set communication algorithm performance table when it is determined that a cache exists; for a detailed explanation of the set communication algorithm performance table, please refer to the description of the above embodiments, which will not be repeated here.

[0159] Step 310: Build a communication group.

[0160] Specifically, building a communication group can be understood as the specific implementation method of building a communication group based on the performance table of the called set communication algorithm in the above embodiments.

[0161] Step 312: Build a communication group.

[0162] Specifically, when it is determined that there is no cache, a communication group is constructed; the specific implementation method for constructing the communication group is the same as the specific implementation method for constructing the communication group when it is determined that there is no set communication algorithm performance table in the above embodiment.

[0163] Step 314: Pre-check.

[0164] Specifically, proceed to Pre-check step 31402 - step 31412.

[0165] Step 31402: Obtain the data type.

[0166] Specifically, obtaining the data type can be understood as obtaining the data type of the data to be processed in the above embodiments.

[0167] Step 31404: Determine whether the preset communication volume corresponding to a certain data type is >512 MB (i.e., msg > 512 MB). If yes, proceed to step 31406; otherwise, proceed to step 31408.

[0168] Here, msg(message) can be understood as the preset communication volume corresponding to the data type of the data to be processed in the above embodiments.

[0169] Step 31406: Exit the loop.

[0170] Specifically, if the preset communication volume corresponding to a certain data type is determined to be greater than 512 MB, the loop will be broken and the execution of Pre-check will end.

[0171] Step 31408: Execute the bandwidth optimization communication group and the latency optimization communication group.

[0172] Specifically, when the preset communication volume corresponding to a certain data type is less than 512 MB, the execution of bandwidth-optimized communication groups and latency-optimized communication groups is performed to statistically analyze performance.

[0173] Step 31410: Perf_time[msg,type,allreduce].

[0174] In this context, Perf_time can be understood as the performance time statistics in the performance table of the set communication algorithm in the above embodiment, and type can be understood as the data type of the data to be processed.

[0175] Specifically, Perf_time[msg,type,allreduce] can be understood as calculating communication performance based on the data type of the data to be processed, the preset communication volume corresponding to a certain data type, and the set communication algorithm.

[0176] Step 31412: msg* = 2.

[0177] Specifically, the formula can be understood as doubling the preset communication volume corresponding to a certain data type, that is, returning the doubled preset communication volume corresponding to a certain data type to step 31404 and continuing the loop to complete the execution of Pre-check.

[0178] Furthermore, Pre-check can be understood as the specific implementation process of constructing the performance table of the set communication algorithm in the above embodiments.

[0179] Step 316: Build the performance table.

[0180] This performance table can be understood as a performance table for set communication algorithms.

[0181] Specifically, constructing the performance table can be understood as the specific implementation result of constructing the performance table of the set communication algorithm in the above embodiments.

[0182] Step 318: Communication threshold.

[0183] The communication volume threshold can be understood as the communication group selection threshold in the above embodiments.

[0184] Step 320: Start training.

[0185] Step 322: Execute the set communication algorithm.

[0186] Specifically, executing a set communication algorithm can be understood as determining a target set communication algorithm and then performing the subsequent model training process based on that target set communication algorithm.

[0187] Step 324: Obtain communication volume.

[0188] Specifically, obtaining communication volume can be understood as determining the data processing communication volume of the data to be processed and the data type of the data to be processed in the above embodiments.

[0189] Step 326: Determine whether the communication volume is greater than the communication volume threshold. If yes, proceed to step 328; otherwise, proceed to step 330.

[0190] Step 328: Comm_bw selection.

[0191] Specifically, if the data processing traffic exceeds the traffic threshold, the bandwidth-optimized communication group, Comm_bw, is selected.

[0192] Step 330: Comm_lat selection.

[0193] Specifically, if the data processing communication volume is determined to be less than the communication volume threshold, the delay-optimized communication group, Comm_lat, is selected.

[0194] Step 332: Communication execution.

[0195] Specifically, communication execution can be understood as performing model training based on the target communication group (such as a bandwidth-optimized communication group or a latency-optimized communication group) in the above embodiments.

[0196] Step 334: End.

[0197] Specifically, "end" can be understood as ending the training cycle.

[0198] The data communication optimization method provided in the embodiments of this specification addresses the performance of bandwidth and latency under different data volumes. It implements an optimization method that allows multiple communication groups to coexist, including bandwidth-optimized communication groups and latency-optimized communication groups. This ensures that the complex workload of a single training session can achieve a better solution for multiple objectives. When faced with different communication data volumes, appropriate communication groups can be selected for communication, thereby improving communication performance and model training efficiency.

[0199] Corresponding to the above method embodiments, this specification also provides embodiments of a data communication optimization device. Figure 4 A schematic diagram of a data communication optimization device according to one embodiment of this specification is shown. Figure 4 As shown, the device includes:

[0200] The request determination module 402 is configured to determine a data processing request, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed;

[0201] The data determination module 404 is configured to determine the data processing communication volume of the data to be processed and the corresponding communication group selection threshold according to the target set communication algorithm.

[0202] The communication group determination module 406 is configured to determine a target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold, wherein the at least two communication groups correspond to the aggregate communication algorithm;

[0203] The request execution module 408 is configured to execute the data processing request based on the target communication group.

[0204] Optionally, the device further includes:

[0205] The algorithm calling module is configured as follows:

[0206] Obtain the call chain of the original communication library corresponding to the data processing request, and call the original set communication algorithm in the original communication library according to the call chain, wherein the target set communication algorithm is any one of the original set communication algorithms.

[0207] Optionally, the device further includes:

[0208] The communication group building module is configured as follows:

[0209] If a set communication algorithm performance table is found to exist, the set communication algorithm performance table is invoked, and at least two communication groups are constructed based on the set communication algorithm performance table, wherein the communication protocols of each of the at least two communication groups are different.

[0210] If it is determined that there is no performance table for the set communication algorithm, at least two communication groups corresponding to the original set communication algorithm are constructed based on the data to be processed, wherein the communication protocols of each of the at least two communication groups are different.

[0211] Optionally, the device further includes:

[0212] The performance statistics module is configured as follows:

[0213] Determine the data type of the data to be processed, and the preset communication volume set corresponding to the data type;

[0214] The target communication volume is determined sequentially from the preset communication volume set, and the target communication volume is processed according to at least two communication groups corresponding to the original set communication algorithm. The execution performance set of at least two communication groups corresponding to the original set communication algorithm is then determined.

[0215] Construct a performance table for the set communication algorithm based on the execution performance set.

[0216] Optionally, the device further includes:

[0217] The threshold determination module is configured as follows:

[0218] Based on the performance table of the set communication algorithm, determine the communication group selection threshold corresponding to the original set communication algorithm.

[0219] Optionally, the communication group determination module 406 is further configured to:

[0220] The data processing traffic is compared with the communication group selection threshold, and the target communication group is determined from at least two communication groups based on the comparison result.

[0221] Optionally, the communication group determination module 406 is further configured to:

[0222] The at least two communication groups include:

[0223] Bandwidth-optimized communication group and latency-optimized communication group.

[0224] Optionally, the communication group determination module 402 is further configured to:

[0225] Determine the model training request for the target machine learning model;

[0226] Accordingly, the request execution module 408 is further configured to:

[0227] Based on the target communication group, execute the model training request for the target machine learning model.

[0228] The data communication optimization device provided in this specification can select a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. This allows for the selection of a suitable target communication group for data processing based on the different data processing communication volumes and the corresponding communication group selection threshold. When applied to distributed training scenarios of deep learning models, this data communication optimization device can greatly improve the communication performance and model training efficiency during the model training process.

[0229] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the data communication optimization apparatus embodiments are basically similar to the data communication optimization method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data communication optimization method embodiments.

[0230] Corresponding to the above method embodiments, this specification also provides embodiments of a data communication optimization system. Figure 5 A schematic diagram of a data communication optimization system according to one embodiment of this specification is shown. Figure 5 As shown, the system includes: adaptation nodes, algorithm nodes, prediction nodes, and scheduling nodes, wherein:

[0231] The adapter node 502 is configured to determine a data processing request sent from the network node, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed;

[0232] The algorithm node 504 is configured to determine the target set communication algorithm and, based on the target set communication algorithm, determine the data processing communication volume of the data to be processed.

[0233] The scheduling node 506 is configured to determine a target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold, wherein the at least two communication groups correspond to the target set communication algorithm, and the at least two communication groups and the communication group selection threshold are determined by the prediction node 608.

[0234] The scheduling node 506 is also configured to execute the data processing request according to the target communication group.

[0235] The data communication optimization system provided in this specification selects a target communication group from multiple communication groups based on the data processing communication volume of the data to be processed carried in the data processing request and the communication group selection threshold. This allows for the selection of a suitable target communication group for data processing based on different data processing communication volumes and the corresponding communication group selection threshold. When applied to distributed training scenarios of deep learning models, this data communication optimization system can greatly improve the communication performance and model training efficiency during the model training process.

[0236] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the data communication optimization system embodiment is basically similar to the data communication optimization method embodiment, so the description is relatively simple; relevant parts can be referred to in the description of the data communication optimization method embodiment.

[0237] See Figure 6 , Figure 6 An architecture diagram of a data communication optimization system provided in one embodiment of this specification is shown.

[0238] The specific architecture of a data communication optimization system provided in one embodiment of this specification can be divided into 5 layers, from top to bottom: adaptation layer 602, algorithm layer 604, scheduling layer 606, prediction layer 608, and network layer 610.

[0239] Adaptation Layer 602: Primarily implemented for the user interface side, it features a unified communication acceleration framework. This framework comprises three main components: the first component, the second component, and the third component. It can be quickly adapted by registering with the backend of an open-source Python machine learning library using a single line of code.

[0240] The first component mainly consists of Python-side collection communication algorithm interfaces, such as the creation and destruction of communication groups. Ultimately, it calls the underlying second component interface to intercept Python's call chain to the original communication library.

[0241] The second component mainly consists of the backend code for the communication group implemented in C++, including the definition of different collection communication algorithms such as allreduce, which is used to replace the original collection communication algorithm side call implementation of the original communication library;

[0242] The third component mainly consists of the network-side interface implementation of the nccl interface, including the implementation of sockets and direct memory access communication interfaces, which is used to replace the native communication-side interface implementation of the original communication library.

[0243] Algorithm layer 604: This layer mainly implements the collection communication algorithm in the backend of the second component. This layer will call the original communication library in the lower layer according to the user's actual call. In addition, this layer also needs to implement different communication traffic interception for the communication group selection of the next layer.

[0244] Scheduling layer 606: Primarily responsible for selecting thresholds for the comm_bw bandwidth-optimized communication group, the comm_lat latency-optimized communication group, and the msg_gate communication group. The selection logic is mainly based on the actual measurement results from the prediction layer. If the communication volume size is less than the communication group selection threshold, then the latency-optimized communication group is selected as the target communication group to execute the corresponding collective communication algorithm; otherwise, the bandwidth-optimized communication group is selected as the target communication group to execute the corresponding collective communication algorithm.

[0245] Comm_bw is a communication group construction tool for the simple protocol with bandwidth optimization.

[0246] Comm_lat is used for constructing communication groups for latency optimization in the LL128 protocol;

[0247] The existence of multiple communication groups only incurs an additional overhead of 20 MB of video memory, without any additional occupation of other resources. Therefore, if only one communication group is actually communicating at the same time, the performance will not be affected by the other communication groups not being called. This detail makes the multi-communication group scheme corresponding to the data communication optimization method provided in the embodiments of this specification superior.

[0248] Prediction layer 608: mainly targets the interface of the original communication library of the network layer for hijacking, such as hijacking the execution of the interface functions of the original communication library for pre-check; the function of pre-check is mainly to perform pre-communication stress test for a given amount of data, construct multiple communication groups under the execution parameters received by the original communication library, execute and record specific performance indicators and update the global performance statistics table.

[0249] Network Layer 610: This layer mainly implements the raw communication library, specifically socket communication or direct memory access communication. This layer exposes some APIs for native data types. For example, the raw workload will call the initial interface function from open-source Python machine learning libraries and other artificial intelligence frameworks to initialize the construction of the communication group.

[0250] An embodiment of this specification is illustrated by combining an architecture diagram of a data communication optimization system provided in one embodiment of this specification with a flowchart of the specific processing procedure of a data communication optimization method provided in one embodiment of this specification.

[0251] In the adaptation layer, the Python code to be delivered is input, the data processing request is determined, and the unified communication acceleration framework includes three major components: the first component, the second component, and the third component. The adaptation is quick, and the call chain of the original communication library corresponding to the data processing request is obtained. The original set communication algorithm in the original communication library is called according to the call chain.

[0252] In the prediction layer, API hijacking is performed to determine if a cache exists. If a performance table for the collective communication algorithm is found to exist, the cache is loaded and a communication group is constructed. If a performance table for the collective communication algorithm is found not to exist, a pre-check is performed to construct a performance table. This mainly targets the interface of the original communication library in the network layer for hijacking, such as hijacking the execution of the interface functions of the original communication library for pre-check. The function of pre-check is mainly to perform pre-communication stress testing for a given amount of data. Under the execution parameters received by the original communication library, multiple comms communication groups are constructed, executed, and specific performance indicators are recorded and updated to the global performance statistics table.

[0253] In the scheduling layer, the data processing communication volume of the data to be processed and the corresponding communication group selection threshold are determined according to the target set communication algorithm. Based on the data processing communication volume and the communication group selection threshold, the target communication group is determined from at least two communication groups. This mainly involves selecting a bandwidth-optimized communication group, a latency-optimized communication group, and a communication group selection threshold. The selection logic is mainly based on the actual measurement results of the prediction layer.

[0254] In the algorithm layer, the set communication algorithm is executed to obtain communication traffic. The algorithm layer will call the implementation of the original communication library in the lower layer according to the actual call of the user. In addition, this layer also needs to implement different communication traffic interception.

[0255] This specification combines an architecture diagram of a data communication optimization system provided in one embodiment with a flowchart of the specific processing procedure of a data communication optimization method provided in one embodiment to illustrate one embodiment of this specification. Based on the lossless performance effect of multiple communication groups, and by fully utilizing the communication characteristics of bandwidth and latency, a bandwidth-latency hybrid optimization scheme based on multiple communication groups corresponding to the data communication optimization method provided in this embodiment of the specification is realized. The data communication optimization method provided in this embodiment of the specification is essentially a protocol stack implementation with hybrid effective payload, thereby achieving better performance of distributed training under a specific cluster network.

[0256] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0257] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0258] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0259] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0260] The processor 720 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data communication optimization method.

[0261] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computing device embodiments are basically similar to the data communication optimization method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data communication optimization method embodiments.

[0262] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data communication optimization method.

[0263] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. In particular, the computer-readable storage medium embodiments are relatively simple in description because they are fundamentally similar to the data communication optimization method embodiments; relevant parts can be referred to in the description of the data communication optimization method embodiments.

[0264] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data communication optimization method.

[0265] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the computer program embodiments are basically similar to the data communication optimization method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the data communication optimization method embodiments.

[0266] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0267] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0268] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous. Secondly, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification.

[0269] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0270] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data communication optimization method, comprising: A data processing request is determined, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed; If it is determined that there is no performance table for the collective communication algorithm, the performance table for the collective communication algorithm is constructed by pre-stress testing at least two communication groups based on the data type of the data to be processed. The communication group is an instantiation of a communication library class object. The performance table for the collective communication algorithm stores the communication group selection threshold for different collective communication algorithms. The collective communication algorithm is a processing method for data transmission between different nodes. The data processing communication volume of the data to be processed is determined by performing data processing on the data to be processed according to the target set communication algorithm, and the communication group selection threshold corresponding to the target set communication algorithm is determined according to the set communication algorithm performance table. Based on the data processing communication volume and the communication group selection threshold, a target communication group is determined from at least two communication groups, wherein the at least two communication groups correspond to the target set communication algorithm; The data processing request is executed according to the target communication group.

2. The data communication optimization method according to claim 1, further comprising, after determining the data processing request and before determining the data processing communication volume of the data to be processed and the corresponding communication group selection threshold according to the target set communication algorithm: Obtain the call chain of the original communication library corresponding to the data processing request, and call the original set communication algorithm in the original communication library according to the call chain. The original communication library is the original communication library that uses a specific set communication algorithm to process the data processing request, and the target set communication algorithm is any one of the original set communication algorithms.

3. The data communication optimization method according to claim 2, after obtaining the call chain of the original communication library corresponding to the data processing request and calling the original set communication algorithm in the original communication library according to the call chain, further includes: If a set communication algorithm performance table is found to exist, the set communication algorithm performance table is invoked, and at least two communication groups are constructed based on the set communication algorithm performance table, wherein the communication protocols of each of the at least two communication groups are different.

4. The data communication optimization method according to claim 2, after obtaining the call chain of the original communication library corresponding to the data processing request and calling the original set communication algorithm in the original communication library according to the call chain, further includes: If it is determined that there is no performance table for the set communication algorithm, at least two communication groups corresponding to the original set communication algorithm are constructed based on the data to be processed, wherein the communication protocols of each of the at least two communication groups are different.

5. The data communication optimization method according to claim 4, wherein, in the case that a collective communication algorithm performance table does not exist, constructing a collective communication algorithm performance table for the at least two communication groups through pre-stress testing based on the data type of the data to be processed includes: Determine the data type of the data to be processed, and the preset communication volume set corresponding to the data type; The target communication volume is determined sequentially from the preset communication volume set, and the target communication volume is processed according to at least two communication groups corresponding to the original set communication algorithm. The execution performance set of at least two communication groups corresponding to the original set communication algorithm is then determined. Construct a performance table for the set communication algorithm based on the execution performance set.

6. The data communication optimization method according to claim 1, wherein determining the target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold comprises: The data processing traffic is compared with the communication group selection threshold, and the target communication group is determined from at least two communication groups based on the comparison result.

7. The data communication optimization method according to claim 1, wherein the at least two communication groups include: Bandwidth-optimized communication group and latency-optimized communication group.

8. The data communication optimization method according to claim 1, wherein determining the data processing request includes: Determine the model training request for the target machine learning model; Accordingly, executing the data processing request based on the target communication group includes: Based on the target communication group, execute the model training request for the target machine learning model.

9. A data communication optimization device, comprising: The request determination module is configured to determine a data processing request, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed; The performance statistics module is configured to construct a performance table for at least two communication groups by pre-stress testing, based on the data type of the data to be processed, when it is determined that there is no performance table for the collective communication algorithm. The communication group is an instantiation of a communication library class object. The performance table for the collective communication algorithm stores the communication group selection thresholds for different collective communication algorithms. The collective communication algorithm is a processing method for data transmission between different nodes. The data determination module is configured to determine the data processing communication volume of the data to be processed by performing data processing on the data to be processed according to the target set communication algorithm, and to determine the communication group selection threshold corresponding to the target set communication algorithm according to the set communication algorithm performance table. A communication group determination module is configured to determine a target communication group from at least two communication groups based on the data processing communication volume and the communication group selection threshold, wherein the at least two communication groups correspond to the target set communication algorithm; The request execution module is configured to execute the data processing request based on the target communication group.

10. A data communication optimization system, comprising an adaptation node, an algorithm node, a prediction node, and a scheduling node, wherein: The adapter node is configured to determine a data processing request sent from a network node, wherein the data processing request carries data to be processed and a target set communication algorithm corresponding to the data to be processed; The algorithm node is configured to, in the event that no set communication algorithm performance table exists, construct a set communication algorithm performance table for at least two communication groups through pre-stress testing based on the data type of the data to be processed. The communication group is an instantiation of a communication library class object. The set communication algorithm performance table stores communication group selection thresholds for different set communication algorithms. The set communication algorithm is a processing method for data transmission across different nodes. The node then determines the target set communication algorithm, processes the data to be processed according to the target set communication algorithm to determine the data processing communication volume of the data to be processed, and determines the communication group selection threshold corresponding to the target set communication algorithm based on the set communication algorithm performance table. The scheduling node is configured to determine a target communication group from at least two communication groups based on the data processing traffic and the communication group selection threshold, wherein the at least two communication groups correspond to the target set communication algorithm, and the at least two communication groups and the communication group selection threshold are determined by the prediction node. The scheduling node is also configured to execute the data processing request based on the target communication group.

11. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the data communication optimization method according to any one of claims 1 to 8.

12. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the data communication optimization method according to any one of claims 1 to 8.

13. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the data communication optimization method according to any one of claims 1 to 8.

Images