Communication energy efficiency optimization method, apparatus, device, and storage medium

Abstract

Embodiments of the present application provide a communication energy efficiency optimization method, device, equipment and storage medium, and relate to the technical field of intelligent computing clusters. The method selects a reference chip set from the intelligent computing cluster, obtains reference transmission parameters of the reference chip set, and calculates a transmission distribution ratio based on the reference transmission parameters. In the model training process of distributed data parallel, the to-be-synchronized tensor is divided into an NPU data block and a CPU data block based on the transmission distribution ratio, the CPU data block is transmitted to a DRAM buffer, a compressor is used to generate a compressed data block corresponding to the CPU data block, the compressed data block is transmitted to the DRAM buffer of the corresponding receiving end chip through a CPU communication path, and the NPU data block is transmitted to the HBM memory of the receiving end chip through an NPU communication path. The transmission distribution ratio is determined in combination with the actual transmission capacity of the hardware, the to-be-synchronized tensor division can accurately match the transmission capacity of the two types of hardware, and the training efficiency of the intelligent computing cluster is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent computing clusters, and in particular to a communication energy efficiency optimization method, device, equipment and storage medium. BACKGROUND

[0002] With the rapid development of AI large models in natural language processing, computer vision and multi-modal tasks, intelligent computing clusters are increasingly widely used in scientific research, industry and various fields of society. However, training a large language model with hundreds of billions of parameters once can consume up to several megawatt-hours of total energy, and the energy consumption problem has become a key problem restricting the efficient operation of intelligent computing clusters. Therefore, reducing the overall communication energy consumption by reducing the communication data volume and shortening the communication time is one of the important ways to improve the energy efficiency of the cluster.

[0003] In related technologies, each AI node of an intelligent computing cluster usually includes two types of core processors, NPU and CPU. In the model training process, NPU undertakes the main AI computing tasks, such as matrix multiplication, convolution operation, and high-computing-intensive operations such as forward and backward propagation, and CPU is responsible for general computing, task scheduling, data preprocessing, and auxiliary computing. However, under this division mode, the CPU channel is mostly in an idle state during the training phase and cannot participate in the cross-node data transmission process, resulting in an insignificant energy consumption reduction effect. SUMMARY

[0004] The main purpose of the embodiments of the present application is to propose a communication energy efficiency optimization method, device, equipment and storage medium to improve the training efficiency of intelligent computing cluster AI training.

[0005] To achieve the above purpose, a first aspect of the embodiments of the present application proposes a communication energy efficiency optimization method, comprising:

[0006] Applied to a sending end chip, the method comprises:

[0007] Obtaining a transmission allocation ratio, the transmission allocation ratio being obtained by selecting a reference chip group from an intelligent computing cluster, obtaining a reference transmission parameter of the reference chip group, and calculating the reference transmission parameter, the reference chip group comprising a reference sending chip and a reference receiving chip;

[0008] In the process of distributed data parallel model training, obtaining a to-be-synchronized tensor, and dividing the to-be-synchronized tensor into an NPU data block and a CPU data block based on the transmission allocation ratio;

[0009] Transmitting the CPU data block to a DRAM buffer area, generating a compressed data block corresponding to the CPU data block by using a selected compressor, and transmitting the compressed data block to the DRAM buffer area of the corresponding receiving end chip through a CPU communication path;

[0010] transmitting the NPU data block to an HBM memory of the receiving chip through an NPU communication path.

[0011] In some embodiments, the transmission distribution ratio is calculated according to the reference transmission parameters, comprising:

[0012] obtaining the reference transmission parameters of the reference chip set, including a CPU bandwidth of at least the CPU communication path, an NPU bandwidth of the NPU communication path, a copy bandwidth between an HBM memory and a DRAM buffer in the reference sending chip or the reference receiving chip;

[0013] calculating the transmission distribution ratio according to at least the CPU bandwidth, the NPU bandwidth and the copy bandwidth.

[0014] In some embodiments, the calculating the transmission distribution ratio according to at least the CPU bandwidth, the NPU bandwidth and the copy bandwidth, comprising:

[0015] setting a ratio parameter, and obtaining reference NPU data and reference CPU data according to the ratio parameter and a data size of a reference data packet;

[0016] calculating a total communication time according to the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth and the copy bandwidth;

[0017] constructing a total energy consumption based on the total communication time, minimizing the total energy consumption to obtain a target optimization function, solving the target optimization function to obtain an optimal solution of the ratio parameter as the transmission distribution ratio.

[0018] In some embodiments, the calculating a total communication time according to the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth and the copy bandwidth, comprising:

[0019] calculating an NPU time delay according to the reference NPU data and the NPU bandwidth, calculating an external transmission time delay according to the reference CPU data, a compression ratio of the compressor and the CPU bandwidth, calculating an internal transmission time delay according to the ratio parameter, the data size and the copy bandwidth, and calculating a CPU time delay according to the external transmission time delay and the internal transmission time delay;

[0020] obtaining a compression time of the reference data packet, and obtaining a compression time delay according to the ratio parameter and the compression time;

[0021] calculating a CPU total time delay according to the CPU time delay and the compression time delay, and setting a maximum value of the CPU total time delay and the NPU time delay as the total communication time.

[0022] In some embodiments, the total communication duration is constructed based on the total communication duration, comprising:

[0023] Obtaining a compression power parameter, a CPU power parameter of the CPU communication path, an NPU power parameter of the NPU communication path, a CPU idle power and an NPU idle power;

[0024] Calculating the difference between the total communication duration and the CPU delay and the NPU delay respectively, to obtain CPU idle time and NPU idle time;

[0025] According to the product of the compression delay and the compression power parameter, the compression energy consumption is obtained, according to the product of the CPU delay and the CPU power parameter, the CPU transmission energy consumption is obtained, according to the product of the CPU idle power and the CPU idle time, the CPU idle energy consumption is obtained, according to the product of the NPU delay and the NPU power parameter, the NPU transmission energy consumption is obtained, according to the product of the NPU idle power and the NPU idle time, the NPU idle energy consumption is obtained;

[0026] The total energy consumption is obtained by accumulating the compression energy consumption, the CPU transmission energy consumption, the CPU idle energy consumption, the NPU transmission energy consumption and the NPU idle energy consumption.

[0027] In some embodiments, the target optimization function is solved to obtain the optimal solution of the proportion parameter as the transmission allocation proportion, comprising:

[0028] The target optimization function is solved by using a convex function optimization solving method, and the solution corresponding to the case that the sum of the CPU delay and the compression delay is consistent with the NPU delay is set as the mutation point of the proportion parameter;

[0029] The energy consumption values corresponding to the case that the proportion parameter is two end point values and the mutation point are calculated respectively, and the proportion parameter corresponding to the minimum energy consumption value is selected as the transmission allocation proportion.

[0030] To achieve the above-mentioned purpose, a second aspect of the embodiments of the present application proposes a communication energy efficiency optimization method, applied to a receiving end chip, the method comprising:

[0031] Obtaining a compressed data block transmitted by a sending end chip to a DRAM buffer through a CPU communication path;

[0032] Obtaining an NPU data block transmitted by the sending end chip to an HBM memory through an NPU communication path, the compressed data block and the NPU data block being obtained by the sending end chip according to the communication energy efficiency optimization method of any one of the first aspect;

[0033] Decompress the compressed data block to obtain a decompressed data block, transmit the decompressed data block to the HBM memory, and merge the decompressed data block and the NPU data block to obtain target data corresponding to the to-be-synchronized tensor.

[0034] To achieve the above object, a third aspect of the embodiment of the present application proposes a communication energy efficiency optimization device, applied to a sending end chip, the device comprises:

[0035] A parameter acquisition module is configured to acquire a transmission allocation ratio, the transmission allocation ratio being calculated based on a reference transmission parameter of a reference chip set selected from an intelligent algorithm cluster, the reference chip set comprising a reference sending chip and a reference receiving chip;

[0036] A data allocation module is configured to acquire a to-be-synchronized tensor during a distributed data parallel model training process, and divide the to-be-synchronized tensor into an NPU data block and a CPU data block based on the transmission allocation ratio;

[0037] A CPU data transmission module is configured to transmit the CPU data block to a DRAM buffer, generate a compressed data block corresponding to the CPU data block by using a selected compressor, and transmit the compressed data block to a DRAM buffer of a corresponding receiving end chip through a CPU communication path;

[0038] An NPU data transmission module is configured to transmit the NPU data block to an HBM memory of the receiving end chip through an NPU communication path.

[0039] To achieve the above object, a fourth aspect of the embodiment of the present application proposes an electronic device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method of the first aspect or the second aspect when executing the computer program.

[0040] To achieve the above object, a fifth aspect of the embodiment of the present application proposes a storage medium, which is a storage medium, the storage medium storing a computer program, and the computer program being executed by a processor to implement the method of the first aspect or the second aspect.

[0041] The communication energy efficiency optimization method, apparatus, device, and storage medium proposed in this application obtain a transmission allocation ratio. This ratio is calculated by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on these parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During distributed data parallel model training, a tensor to be synchronized is obtained. Based on the transmission allocation ratio, the tensor to be synchronized is divided into NPU data blocks and CPU data blocks. The CPU data blocks are transmitted to a DRAM buffer. A selected compressor generates compressed data blocks corresponding to the CPU data blocks. The compressed data blocks are transmitted through the CPU communication path to the DRAM buffer of the corresponding receiving chip. The NPU data blocks are transmitted through the NPU communication path to the HBM memory of the receiving chip. This application embodiment determines the transmission allocation ratio through benchmark chipset calibration and combined with the actual hardware transmission capabilities. This ensures that the ratio of the tensor to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. For example, when using only the NPU path, a large amount of non-critical data will consume the high bandwidth resources of HBM; when using only the CPU path, the data transmission speed is limited by the interaction efficiency between DRAM and CPU, thus maximizing the performance advantages of both paths. Simultaneously, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On one hand, NPU data blocks are directly transmitted to the HBM memory of the receiving chip through the NPU communication path, adapting to the critical tensor data that needs high-frequency access in AI training, ensuring rapid synchronization of core data, and reducing waiting time during training iterations; on the other hand, CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving DRAM buffer through the CPU communication path, effectively alleviating the bandwidth pressure between DRAM and CPU. Furthermore, parallel transmission can avoid the efficiency loss caused by serial data transmission. Therefore, this application embodiment, through hardware-adapted tensor partitioning, dual-path parallel transmission, and data compression strategies, can solve the problems of resource waste and speed bottlenecks existing in a single transmission path, enabling the hardware resources of the intelligent computing cluster, such as NPU, CPU, HBM, and DRAM, to form efficient collaboration, ultimately significantly improving the data synchronization efficiency of AI training, accelerating the overall training process, and achieving an improvement in the AI ​​training efficiency of the intelligent computing cluster. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the communication process of computing nodes in the intelligent computing cluster provided in the embodiments of this application.

[0043] Figure 2 This is a flowchart of the communication energy efficiency optimization method provided in the embodiments of this application.

[0044] Figure 3is a flowchart provided by an embodiment of the present application for calculating a transmission allocation ratio according to a reference transmission parameter.

[0045] Figure 4 is a flowchart provided by an embodiment of the present application for calculating a transmission allocation ratio according to at least a CPU bandwidth, an NPU bandwidth and a copy bandwidth.

[0046] Figure 5 is a flowchart provided by an embodiment of the present application for calculating a total communication duration according to a reference NPU data, a reference CPU data, a CPU bandwidth, an NPU bandwidth and a copy bandwidth.

[0047] Figure 6 is a flowchart provided by an embodiment of the present application for constructing a total energy consumption based on the total communication duration.

[0048] Figure 7 is a flowchart provided by an embodiment of the present application for a communication energy efficiency optimization method applied to a receiving end chip.

[0049] Figure 8 is a structure block diagram of a communication energy efficiency optimization device provided by another embodiment of the present application.

[0050] Figure 9 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not intended to limit the present application.

[0052] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application, and are not intended to limit the present application.

[0054] First, several terms involved in the present application are analyzed:

[0055] Artificial Intelligence (AI): is a new technical science that studies, develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; Artificial intelligence is a branch of computer science, artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems, etc. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0056] With the rapid development of AI large models in natural language processing, computer vision and multi-modal tasks, the application of intelligent computing clusters in scientific research, industry and society is becoming more and more widespread. However, according to public data statistics, the total energy consumption of training a trillion-parameter-level large language model can reach several megawatt-hours. It can be seen that the energy consumption problem has become a key problem restricting the efficient operation of intelligent computing clusters, which not only brings high operating costs, but also aggravates carbon emissions.

[0057] In distributed AI training, especially in the parallel training scenario of large-scale models, parameter gradients or activation values need to be frequently synchronized between different computing nodes. According to existing measurement results, such communication often accounts for 20% to 40% of the training time, and in large-scale clusters it can even become a bottleneck. Since the communication process will occupy the high-speed interconnection channel and keep the device active, its energy consumption proportion cannot be ignored. Therefore, reducing the amount of communication data, shortening the communication time, and thus reducing the overall communication energy consumption, is one of the important ways to improve the energy efficiency of the cluster.

[0058] In the related art, each AI node of the intelligent computing cluster usually contains two types of core processors, NPU and CPU. In the model training process, NPU undertakes the main AI computing tasks, such as matrix multiplication, convolution operation, and high-power intensive operations such as forward and backward propagation, and CPU is responsible for general computing, task scheduling, data preprocessing and auxiliary computing. In the current implementation process of AI training, in order to be easy to use and compatible with the software stack, only the NPU dedicated high-speed communication channel equipped in the AI server, such as HCCS, NVLink, GPU RDMA, etc., is used for cross-node parameter synchronization, and the regular network channel of the CPU end of the server, such as PCIe-based NIC, RoCE, Ethernet RDMA, etc. This approach simplifies development, but also means that the CPU channel is mostly idle during the training phase and cannot participate in cross-node data transmission. If the original training communication data is divided equally into two channels for transmission, due to the fact that the bandwidth of the CPU channel is usually significantly lower than that of the NPU dedicated channel, this gap causes the equal division method not only unable to improve the overall synchronization speed, but also may lengthen the critical path. Therefore, the unreasonable allocation ratio also cannot effectively reduce the energy consumption.

[0059] Based on this, the embodiment of the application provides a communication energy efficiency optimization method, device, equipment and storage medium, which is applied to a compressed training scene of an intelligent computing cluster, a transmission distribution ratio is determined by benchmark chip set calibration and combined with actual transmission capacity of hardware, it is ensured that the ratio of the to-be-synchronized tensor divided into NPU data blocks and CPU data blocks can accurately match the transmission capacity of the two types of hardware, and the short board effect caused by the mismatch of hardware performance in the traditional single path transmission is avoided. For example, when only the NPU path is used, a large amount of non-critical data will occupy the high-bandwidth resource of the HBM; when only the CPU path is used, the data transmission speed is limited by the interaction efficiency of the DRAM and the CPU, so that the performance advantages of the two types of paths are maximized. At the same time, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On the one hand, the NPU data blocks are directly transmitted to the HBM memory of the receiving end chip through the NPU communication path, which is suitable for the critical tensor data that need to be accessed frequently in AI training, guarantees the rapid synchronization of core data, and reduces the waiting time in the training iteration; on the other hand, the CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving end DRAM buffer through the CPU communication path, which effectively relieves the bandwidth pressure between the DRAM and the CPU. In addition, parallel transmission can also avoid the efficiency loss caused by serial data transmission. Therefore, through the hardware-adapted tensor division, double-path parallel transmission and data compression strategy, the embodiment of the application can solve the problems of resource waste and speed bottleneck existing in the single transmission path, make the hardware resources such as NPU, CPU, HBM and DRAM of the intelligent computing cluster form efficient cooperation, and finally significantly improve the data synchronization efficiency of AI training and accelerate the overall training process, so as to realize the improvement of the AI training efficiency of the intelligent computing cluster.

[0060] The embodiment of the application provides a communication energy efficiency optimization method, device, equipment and storage medium, which is applied to a compressed training scene of an intelligent computing cluster, a transmission distribution ratio is determined by benchmark chip set calibration and combined with actual transmission capacity of hardware, it is ensured that the ratio of the to-be-synchronized tensor divided into NPU data blocks and CPU data blocks can accurately match the transmission capacity of the two types of hardware, and the short board effect caused by the mismatch of hardware performance in the traditional single path transmission is avoided. For example, when only the NPU path is used, a large amount of non-critical data will occupy the high-bandwidth resource of the HBM; when only the CPU path is used, the data transmission speed is limited by the interaction efficiency of the DRAM and the CPU, so that the performance advantages of the two types of paths are maximized. At the same time, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On the one hand, the NPU data blocks are directly transmitted to the HBM memory of the receiving end chip through the NPU communication path, which is suitable for the critical tensor data that need to be accessed frequently in AI training, guarantees the rapid synchronization of core data, and reduces the waiting time in the training iteration; on the other hand, the CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving end DRAM buffer through the CPU communication path, which effectively relieves the bandwidth pressure between the DRAM and the CPU. In addition, parallel transmission can also avoid the efficiency loss caused by serial data transmission. Therefore, through the hardware-adapted tensor division, double-path parallel transmission and data compression strategy, the embodiment of the application can solve the problems of resource waste and speed bottleneck existing in the single transmission path, make the hardware resources such as NPU, CPU, HBM and DRAM of the intelligent computing cluster form efficient cooperation, and finally significantly improve the data synchronization efficiency of AI training and accelerate the overall training process, so as to realize the improvement of the AI training efficiency of the intelligent computing cluster.

[0061] The embodiment of the application can acquire and process related data based on artificial intelligence technology. Artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0062] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other major directions.

[0063] The communication energy efficiency optimization method provided by the embodiment of the present application relates to the technical field of intelligent computing clusters. The communication energy efficiency optimization method provided by the embodiment of the present application can be executed by a terminal or a server end, and can also be a computer program running in a terminal or a server end to execute. For example, the computer program can be a native program or a software module in the operating system; it can be a native application (Application, APP), that is, a program that needs to be installed in the operating system to run, such as a client supporting communication energy efficiency optimization, that is, a program that only needs to be downloaded into a browser environment to run; it can also be a small program that can be embedded into any APP. In short, the above computer program can be any form of application program, module or plug-in. Among them, the terminal communicates with the server through the network. The communication energy efficiency optimization method can be executed by the terminal or the server alone, or by the terminal and the server cooperatively.

[0064] In some embodiments, the terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, or a smart watch, etc. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms; it can also be a service node in a blockchain system, the service nodes in the blockchain system form a peer-to-peer (Peer To Peer, P2P) network among them, and the P2P protocol is an application layer protocol running on the transmission control protocol (Transmission Control Protocol, TCP) protocol. The terminal and the server can be connected through Bluetooth, universal serial bus (Universal Serial Bus, USB) or network communication connection methods, which are not limited in the embodiment.

[0065] The application is operable in a multitude of generic or specific computer system environments or configurations. Examples of well known computing systems, environments, and / or configurations that can be suitable for use with the application include personal computers, server computers, handheld or laptop devices, tablet devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, and the like, that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in local and remote computer storage media including memory storage devices.

[0066] A communication energy efficiency optimization method in an embodiment of the application is described below.

[0067] Reference is first made to Figure 1 , Figure 1 is a schematic diagram of a communication process of a computing node in an intelligent computing cluster provided by an embodiment of the application. Each AI computing node is a chip. Reference is made to Figure 1 A large number of AI computing nodes are deployed in an intelligent computing cluster to run cooperatively to support training tasks of large-scale artificial intelligence models. Each AI node internally includes two types of core processors: one is an NPU (Neural Processing Unit) processor for high-performance computing of deep learning, and the other is a CPU (Central Processing Unit) processor for general computing. In the training process, the NPU processor is responsible for performing main AI computing, such as matrix multiplication, convolution operation, and forward and backward propagation of high-computing-intensive tasks; and the CPU processor is responsible for general computing, task scheduling, data preprocessing, and auxiliary computing.

[0068] In an embodiment, as shown in Figure 1 , a high-speed HBM (High Bandwidth Memory) memory is equipped in the NPU processor as its local storage, and all model parameters, activation values, and intermediate computing results required for AI training are stored in the HBM memory. When the NPU processors of different chips communicate with each other, a data path can be established directly between the HBM memories of different computing nodes as an NPU communication path, so as to realize high-speed data transmission from the HBM memory of the AI computing node 1 to the HBM memory of the AI computing node 2. Such an NPU communication path has extremely high bandwidth, usually hundreds of GB / s, and is suitable for use in large-scale gradient synchronization, activation exchange, and other scenarios.

[0069] In an embodiment, referring to Figure 1 As shown, the CPU processor uses a conventional DRAM (Dynamic Random-Access Memory) memory as local storage, which has a higher access delay and a bandwidth generally lower than HBM, but a larger capacity. When the CPU processors in the same chip communicate, the DRAM buffer of the AI computing node 1 can directly transmit data to the DRAM buffer of the AI computing node 2 through a network interface such as RoCE based on RDMA or InfiniBand, and this communication link is called a CPU communication path. The bandwidth of this CPU communication path is generally significantly smaller than that of the NPU communication path, but it can also undertake the task of data synchronization.

[0070] In an embodiment, if it is necessary to transmit data originally stored in the HBM memory through the CPU communication path, an additional memory copying step must be performed, which is: first, copy the data from the HBM memory to the DRAM buffer on the AI computing node 1; then, the CPU processor performs a compression operation on the data in the DRAM buffer to reduce the data volume; the compressed data is transmitted to the DRAM buffer of the AI computing node 2 through the CPU communication path. At the receiving end, the CPU processor of the AI computing node 2 decompresses the data, and finally copies the decompressed data back to the HBM memory of the AI computing node 2 to participate in subsequent NPU processor calculations. Since the resident occupation of the DRAM buffer in the AI training task is generally low, sufficient DRAM buffer space can be reserved as a data buffer for the CPU communication path to ensure the smooth execution of the above process.

[0071] As can be seen from the above, the core idea of the embodiments of the present application is to simultaneously enable the NPU communication path and the CPU communication path in the cross-node communication of distributed training, determine the transmission allocation ratio in combination with the actual transmission capacity of the hardware, split the data to be transmitted into two parts at a suitable ratio: one part is transmitted directly through the NPU communication path, and the other part is transmitted through the CPU communication path after compression, so as to ensure that the ratio of the NPU data block and the CPU data block of the tensor to be synchronized can accurately match the transmission capacity of the two types of hardware, avoiding the short board effect caused by the mismatch of hardware performance in the traditional single path transmission. In this way, not only can the bandwidth resources of the CPU communication path that are originally idle in the training stage be fully utilized, but also the data volume of the slow channel can be reduced by compression, so that the two channels can complete the task as much as possible in parallel. Compared with the scheme of using only a single communication path, the double-channel parallel communication can significantly shorten the communication time under the same total data volume, and reduce the overall energy consumption by reducing the idle waiting time of the device, thereby realizing higher communication efficiency and energy efficiency performance.

[0072] It can be understood that the above Figure 1In the communication process of the AI computing node 1 and the AI computing node 2, the AI computing node 1 can be a sending end chip, and the AI computing node 2 can be a receiving end chip. If the AI computing node 2 needs to send data to the AI computing node 3, the AI computing node 2 becomes a sending end chip corresponding to the AI computing node 3.

[0073] The communication energy efficiency optimization method of the embodiment of the application is described below, which is applied to the scene of training compression of an intelligent computing cluster.

[0074] In an embodiment, Figure 2 is an optional flowchart of the communication energy efficiency optimization method provided by the embodiment of the application, Figure 2 The method in the above embodiment is applied to a sending end chip, and can include but is not limited to steps 110 to 140. It can be understood that the order of steps 110 to 140 in the above embodiment is not limited, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs. Figure 2

[0075] Step 110: Obtain a transmission allocation ratio.

[0076] In an embodiment, the intelligent computing cluster can set the same transmission allocation ratio for all computing nodes participating in the distributed AI training process, thereby reducing the computing complexity. The transmission allocation ratio can be calculated in advance. First, a pair of sending end chip and receiving end chip is randomly selected from the intelligent computing cluster to form a reference chip set, the reference chip set including a reference sending end chip and a reference receiving end chip, then the reference transmission parameters of the reference chip set are obtained, and the transmission allocation ratio is calculated according to the reference transmission parameters, and the parameter is applied to the entire intelligent computing cluster. If a computing node is used as a sending end chip, the amount of data divided to the NPU communication path and the CPU communication path is determined according to the transmission allocation ratio.

[0077] In an embodiment, refer to Figure 3 , Figure 3 is a flowchart of calculating the transmission allocation ratio according to the reference transmission parameters provided by the embodiment of the application, and specifically includes the following steps:

[0078] Step 310: Obtain the reference transmission parameters of the reference chip set, including the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and the DRAM buffer in the reference sending end chip or the reference receiving end chip.

[0079] ​In one embodiment, a test data packet is generated, which can be consistent with the actual load of training tensor data in a real-world scenario. This test data packet is then used for benchmark calibration. The test data packet is transmitted within a benchmark chipset to test the CPU bandwidth corresponding to each CPU communication path. NPU bandwidth of NPU communication path Simultaneously, either the reference transmitting chip or the reference receiving chip is selected arbitrarily to test the copy bandwidth between the HBM memory and the DRAM buffer. It is understood that the copy bandwidth here can be the uplink bandwidth for transferring data from the HBM memory to the DRAM buffer in the reference transmitting chip, or the downlink bandwidth between the DRAM buffer and the HBM memory in the reference receiving chip, or the average of the uplink and downlink bandwidths. This embodiment does not specifically limit this.

[0080] Simultaneously, CPU power parameters corresponding to the CPU communication path transmission process are obtained through a power meter. NPU power parameters of the NPU communication path CPU idle power during idle time in the reference transmitter chip And the NPU idle power of the NPU processor in the reference receiver chip during idle time. .

[0081] In addition, since data compression is also required in the CPU processor, it is necessary to obtain the compression time corresponding to different data volumes using different compressors under different compression parameters R (such as compression ratio) in the CPU processor. and compression power parameters Understandably, the relationships between data size, compressor, compression parameters, compression time, and compression power parameters can be solved in advance. However, the parameters may differ under each condition and need to be measured separately.

[0082] Thus, for the test data packet, the CPU bandwidth related to the energy consumption parameter calibration can be obtained. NPU bandwidth Copy bandwidth CPU power parameters NPU power parameters CPU idle power NPU idle power Compression time and compression power parameters These data are used as baseline transmission parameters. It's understandable that these baseline transmission parameters can be obtained in advance through offline testing and can be read directly as needed later.

[0083] Step 320: calculating the transmission distribution ratio according to at least the CPU bandwidth, the NPU bandwidth and the copy bandwidth.

[0084] In an embodiment, referring to Figure 4 , Figure 4 is a flowchart provided by the embodiment of the application for calculating the transmission distribution ratio according to at least the CPU bandwidth, the NPU bandwidth and the copy bandwidth, and specifically includes the following steps:

[0085] Step 410: setting a ratio parameter, and obtaining the reference NPU data and the reference CPU data according to the ratio parameter and the data size of the reference data packet.

[0086] In an embodiment, the ratio parameter is assumed to be , the number of the test data packets is , and the data size of each test data packet is Therefore, the reference NPU data is , and the reference CPU data is That is, the data corresponding to the number of in the test data packet is transmitted from the NPU communication path, and the data corresponding to the number of in the test data packet is transmitted from the CPU communication path.

[0087] Step 420: calculating the total communication time according to the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth and the copy bandwidth.

[0088] In an embodiment, referring to Figure 5 , Figure 5 is a flowchart provided by the embodiment of the application for calculating the total communication time according to the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth and the copy bandwidth, and specifically includes the following steps:

[0089] Step 510: calculating the NPU time delay according to the reference NPU data and the NPU bandwidth, calculating the external transmission time delay according to the reference CPU data, the compression ratio of the compressor and the CPU bandwidth, calculating the internal transmission time delay according to the ratio parameter, the data size and the copy bandwidth, and calculating the CPU time delay according to the external transmission time delay and the internal transmission time delay.

[0090] In an embodiment, the NPU time delay calculated according to the reference NPU data and the NPU bandwidth is , which is expressed as:

[0091]

[0092] The external transmission time delay calculated according to the reference CPU data, the compression ratio of the compressor and the CPU bandwidth is expressed as: , the internal transmission delay calculated according to the proportion parameter, the data size and the copy bandwidth is represented as: Here, the internal transmission delay is the uplink delay of transmitting data from the HBM memory in the reference sending chip to the DRAM buffer and the downlink delay of transmitting data from the DRAM buffer in the reference receiving chip to the HBM memory.

[0093] Therefore, the CPU delay calculated according to the external transmission delay and the internal transmission delay is , which is represented as:

[0094]

[0095] Step 520: Obtain the compression duration of the reference data packet, and obtain the compression delay according to the proportion parameter and the compression duration.

[0096] In an embodiment, the compression delay obtained according to the proportion parameter and the compression duration is represented as: .

[0097] Step 530: Calculate the CPU total delay according to the CPU delay and the compression delay, and set the maximum value of the CPU total delay and the NPU delay as the total communication duration.

[0098] In an embodiment, the CPU total delay calculated according to the CPU delay and the compression delay is represented as , and the maximum value of the CPU total delay and the NPU delay is set as the total communication duration, so the total communication duration T is represented as:

[0099]

[0100] Step 430: Construct the total energy consumption based on the total communication duration, minimize the total energy consumption to obtain a target optimization function, solve the target optimization function, and obtain the optimal solution of the proportion parameter as the transmission allocation proportion.

[0101] In an embodiment, referring to Figure 6 , Figure 6 is a flowchart provided by the embodiment of the application for constructing the total energy consumption based on the total communication duration, and specifically includes the following steps:

[0102] Step 610: Obtain the compression power parameter, the CPU power parameter of the CPU communication path, the NPU power parameter of the NPU communication path, the CPU idle power and the NPU idle power.

[0103] Step 620: Calculate the difference between the total communication duration and the CPU delay and the NPU delay, respectively, to obtain the CPU idle time and the NPU idle time.

[0104] In an embodiment, the CPU idle time is obtained according to the difference between the total communication duration and the CPU delay as Based on the difference between the total communication duration and the NPU latency, the NPU idle time is calculated as follows: .

[0105] Step 630: Obtain compression energy consumption by multiplying compression latency and compression power parameters; obtain CPU transmission energy consumption by multiplying CPU latency and CPU power parameters; obtain CPU idle energy consumption by multiplying CPU idle power and CPU idle time; obtain NPU transmission energy consumption by multiplying NPU latency and NPU power parameters; and obtain NPU idle energy consumption by multiplying NPU idle power and NPU idle time.

[0106] In one embodiment, based on compression delay With compression power parameters The compression energy consumption obtained by multiplying the products is According to CPU latency CPU power parameters The CPU transfer energy consumption obtained by multiplying is According to CPU idle power CPU idle time The CPU idle power consumption obtained by multiplying is According to NPU latency With NPU power parameters The NPU transmission power consumption obtained by multiplying the products is According to NPU idle power NPU idle time The NPU idle power consumption obtained by multiplying the product of the two is: .

[0107] Step 640: Accumulate the compression energy consumption, CPU transmission energy consumption, CPU idle energy consumption, NPU transmission energy consumption, and NPU idle energy consumption to obtain the total energy consumption.

[0108] In one embodiment, for the scaling parameter The total energy consumption is obtained by summing up the compression energy consumption, CPU transmission energy consumption, CPU idle energy consumption, NPU transmission energy consumption, and NPU idle energy consumption. Represented as:

[0109]

[0110]

[0111] Next, the optimization objective is to minimize the total energy consumption. Therefore, minimizing the total energy consumption yields the objective optimization function. , is represented as:

[0112]

[0113]

[0114] Next, the target optimization function is solved to obtain the optimal solution of the proportion parameter as the transmission allocation proportion.

[0115] In an embodiment, for a given compressor and its compression parameter R and compression duration , the target optimization function is a convex function with respect to , so the target optimization function is solved by using a convex function optimization solving method, the solution corresponding to the case where the sum of the CPU latency and the compression latency is consistent with the NPU latency is set as the mutation point of the proportion parameter, and the energy consumption values corresponding to the two end point values and the mutation point of the proportion parameter are calculated, and the proportion parameter corresponding to the minimum energy consumption value is selected as the transmission allocation proportion.

[0116] Among them, for a convex function, the minimum value of a convex function on a closed interval must appear at the end point of the interval, or at the stable point where the derivative is equal to zero (that is, the “bottom” of the function). Therefore, the minimum value of the total energy consumption must be taken at the two end points or , and the mutation point of the max maximum function . In this embodiment, the point where the sum of the CPU latency and the compression latency is consistent with the NPU latency is set as the mutation point, that is, the mutation point is defined as the point in the function that just satisfies , which is expressed as:

[0117]

[0118] Only when the traffic data is allocated at the value corresponding to the mutation point, the duration of the CPU communication path and the NPU communication path is exactly equal.

[0119] Therefore, based on , the analytical expression of with respect to the known parameters can be obtained, and after simplification, it is expressed as:

[0120]

[0121]

[0122] Therefore, the energy consumption values when , and are calculated, and the proportion parameter corresponding to the minimum energy consumption value is selected as the transmission allocation proportion. It can be understood that if , it means that there is no need to allocate data to the CPU communication path, and if This means that all data is transmitted using the CPU communication path. This means according to This ratio allocates data to the CPU communication path for transmission. Even here... This is also different from the single transmission in related technologies. This is because the embodiment of this application determines the most suitable transmission allocation ratio based on the actual hardware situation, and it is a value calculated based on the actual situation.

[0123] In one embodiment, due to The computational cost is small, and it can be calculated directly using a one-dimensional fine-grained grid or a bipartite search without resorting to mutation points. Multiple values ​​are selected in a fully covered interval, and energy consumption values ​​are calculated for each value. The convexity of the function is utilized to ensure that the global minimum is found quickly.

[0124] In one embodiment, during the calculation of the transmission allocation ratio, the compression parameters R and compression duration are adjusted for different compressors. Joint optimization is performed to select the optimal transmission allocation ratio. The compressor with the smallest value is selected as the compressor used subsequently. In other words, this embodiment can further select the compressor with the best energy consumption optimization effect based on the transmission allocation ratio, thereby further reducing energy consumption. Since the number of compressor types is generally less than ten, only a small amount of calculation is required, and this embodiment does not limit the actual use.

[0125] Step 120: During the distributed data parallel model training process, obtain the tensor to be synchronized, and divide the tensor to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio.

[0126] In one embodiment, after determining the transmission allocation ratio, energy efficiency optimization for compressed communication is performed based on this ratio during the specific distributed data-parallel model training process. This requires modifying the PyTorch communication layer to achieve dual-channel data transmission. For training jobs using distributed data parallelism, a custom communication hook can be registered to replace the default gradient AllReduce function. At the sending chip, the communication hook first obtains the tensor to be synchronized, and then, based on the pre-calculated transmission allocation ratio... It is divided into two parts, namely NPU data blocks and CPU data blocks.

[0127] Step 130: transmitting the CPU data block to the DRAM buffer, generating a compressed data block corresponding to the CPU data block by using the selected compressor, and transmitting the compressed data block to the DRAM buffer of the corresponding receiving end chip through the CPU communication path.

[0128] In an embodiment, in order to make the compression and transmission of the CPU communication path proceed in parallel, with reference to Figure 1 , the sending end chip first migrates the CPU data block to the DRAM buffer of the sending end chip itself, compresses the CPU data block by using the selected compressor, generates a compressed data block corresponding to the CPU data block, and then transmits the compressed data block to the DRAM buffer of the corresponding receiving end chip through the CPU communication path. It can be understood that the selected compressor here is obtained by jointly optimizing the compression parameter R and the compression time length when calculating the transmission allocation ratio in advance.

[0129] Step 140: transmitting the NPU data block to the HBM memory of the receiving end chip through the NPU communication path.

[0130] In an embodiment, the NPU data block is stored on the HBM memory of the NPU processor, and the communication primitive of the NPU communication path is immediately put into the device flow for execution, and is transmitted to the HBM memory of the receiving end chip through the NPU communication path, so that the two paths are overlapped and parallel in the time axis.

[0131] In an embodiment, with reference to Figure 7 , Figure 7 is a flowchart of a communication energy efficiency optimization method applied to a receiving end chip provided by the embodiment of the application, and specifically includes the following steps:

[0132] Step 710: obtaining a compressed data block transmitted to the DRAM buffer by the sending end chip through the CPU communication path.

[0133] In an embodiment, with reference to Figure 1 , the receiving end chip obtains the compressed data block through the CPU communication path, and the compressed data block is stored in the corresponding DRAM buffer of the receiving end chip.

[0134] Step 720: obtaining an NPU data block transmitted to the HBM memory by the sending end chip through the NPU communication path.

[0135] In an embodiment, at the same time, the receiving end chip also obtains the NPU data block through the NPU communication path, and the NPU data block is stored in the corresponding HBM memory of the receiving end chip.

[0136] Step 730: decompressing the compressed data block to obtain a decompressed data block, transmitting the decompressed data block to the HBM memory, and merging the decompressed data block and the NPU data block to obtain target data corresponding to the tensor to be synchronized.

[0137] In an embodiment, for the receiving chip, the compressed data block in the DRAM buffer needs to be decompressed by using the CPU processor to obtain a decompressed data block. Due to the difference in data arrival time and position of the two communication paths of the NPU and the CPU, the compressed data block received by the CPU communication path must first fall into the DRAM buffer and still temporarily reside in the DRAM buffer after decompression, waiting for the data arrived on the HBM memory by the NPU communication path to be merged. In general AI training, since all parameters are stored on the HBM memory of the NPU processor, the occupation of the host DRAM buffer is usually low, mainly for data loading and a small amount of control caching, so a large DRAM buffer can be reserved as a temporary storage pool for the CPU communication path without affecting the training process. Next, after the two-way data is ready, the receiving chip transmits the decompressed data block to the HBM memory, and merges the decompressed data block and the NPU data block, so as to obtain the target data corresponding to the tensor to be synchronized.

[0138] Since the cross-node synchronization of AI training is usually in the form of a large packet of hundreds of GB, the communication time is mainly limited by the bandwidth rather than the end-to-end delay, and the delay difference between the CPU and NPU channels in the order of microseconds to milliseconds has negligible effect on the total time. It is only necessary to ensure that the splitting ratio and the buffer size meet the stable throughput balance. In summary, by proportionally splitting the tensor to be synchronized according to the transmission distribution ratio obtained by pre-calibration, and cooperating with the recombination process of the receiving chip, the idle bandwidth of the CPU communication path can be effectively converted into significant synchronization time and energy consumption reduction while maintaining the stability of the training.

[0139] In an embodiment, the optimization effect of the communication energy efficiency optimization method of the embodiment of the application is tested by using a simulation tool. The hardware and parameter settings used are as follows: the NPU bandwidth is =112GB / s, the CPU bandwidth is =35GB / s, and the bidirectional copy bandwidth between the HBM and the DRAM is =55GB / s. The power parameters of the node are determined as follows: the CPU power parameter of the CPU communication path is =238W, the CPU idle power is =180W, the NPU power parameter of the NPU communication path is =842W, the NPU idle power is =716W, and the compression power parameter of the compression and decompression process is = 336W, and the single cross-node test data volume is D = 316GB.

[0140] First, the total energy consumption of communication in the baseline scheme using only a single NPU communication path in the related art is E = 2883.5J. In this case, all data is directly transmitted between the HBM memories of the two nodes, and although the bandwidth is high, the NPU processor is in an active state for a long time, and the CPU processor is completely idle, resulting in that the energy efficiency cannot be further optimized.

[0141] If the double-channel parallel communication of the embodiment of the present application is introduced, but without data compression, a part of the data is allocated to the CPU channel path for transmission, so that the channel bandwidth of the originally idle CPU processor is utilized. Although the bandwidth of the CPU channel is low, through reasonable data proportion allocation, the transmission can be completed in parallel with the NPU channel, which significantly reduces the overall energy consumption. The total energy consumption under the non-compression configuration is reduced to 2678.9J, which is reduced by about 7.1% compared with the single NPU communication path.

[0142] And when the compression technology is further introduced in the double-channel CPU path in the embodiment of the present application, the energy consumption performance is closely related to the performance of the compressor. The compression ratio, compression / decompression time and the CPU active power caused thereby jointly determine the final energy efficiency result. Experiments show that the most effective compression is the SZx compressor, which has a moderate compression ratio (R = 3.0) and short compression and decompression time (only 1.2s for full data), so that the additional overhead of the CPU communication path is low, thereby realizing the double optimization of communication time and energy consumption while maintaining a high compression effect. When the SZx compressor is used, the total energy consumption is further reduced to 2633.8J, which is the optimal configuration under the experimental conditions.

[0143] Therefore, the communication energy efficiency optimization method of the embodiment of the present application can utilize two cross-node channels in parallel, i.e. the NPU communication path and the CPU communication path, in the distributed AI training scene: in order to minimize the communication energy consumption, the to-be-synchronized tensor to be synchronized each time across the nodes is split into two parallel transmissions according to the transmission allocation proportion. One is transmitted without compression through the NPU communication path, and the other is compressed and then transmitted through the CPU communication path; at the same time, the compressed data is decompressed by the receiving end and participates in the reduction operation. This process can compensate for the bandwidth disadvantage of the slow channel (CPU communication path) with the compression ratio without increasing the waiting time of the critical path, and can hide the compression / decompression time delay with the help of flow parallel and double buffering mechanism. The core is to control the allocation of the two data through the adjustable fixed allocation proportion, and to jointly optimize the compression ratio R and the total compression and decompression time T omp of different compressors.

[0144] By incorporating the idle CPU network channel (such as RoCE, Ethernet RDMA) into the communication link, the total communication bandwidth can be theoretically improved, and the congestion risk of the NPU dedicated channel (such as NVLink, HCCS) can be reduced; at the same time, the parallel transmission of the double channel can shorten the single synchronization time, especially in large-scale All-Reduce operation, data shunting can effectively alleviate the bandwidth bottleneck. In addition, the efficient compression and decompression process is introduced into the CPU communication path to reduce the data transmission amount of the slow channel, thereby reducing the total communication time and idle energy consumption. Activating the CPU channel will not significantly increase the overall power consumption. Combined with the measured bandwidth and power parameters, the optimal transmission allocation ratio and the optimal compressor selection are determined to optimize the energy efficiency of AI training communication, reduce the idle waiting energy consumption of the device, and meet the green computing trend.

[0145] The technical scheme provided in the embodiments of the present application comprises the following steps: obtaining a transmission distribution ratio, selecting a reference chip set from the intelligent computing cluster, obtaining a reference transmission parameter of the reference chip set, and calculating the reference transmission parameter to obtain the reference chip set, which comprises a reference sending chip and a reference receiving chip; obtaining a to-be-synchronized tensor during a distributed data parallel model training process; dividing the to-be-synchronized tensor into an NPU data block and a CPU data block based on the transmission distribution ratio; transmitting the CPU data block to a DRAM buffer; generating a compressed data block corresponding to the CPU data block by using the selected compressor; transmitting the compressed data block to the DRAM buffer of the corresponding receiving end chip through a CPU communication path; and transmitting the NPU data block to the HBM memory of the receiving end chip through an NPU communication path. The embodiments of the present application calibrate the reference chip set, determine the transmission distribution ratio in combination with the actual transmission capacity of the hardware, ensure that the ratio of the NPU data block and the CPU data block obtained by dividing the to-be-synchronized tensor can accurately match the transmission capacity of the two types of hardware, and avoid the short board effect caused by the mismatch between the hardware performance in the traditional single path transmission. For example, when only the NPU path is used, a large amount of non-critical data will occupy the high bandwidth resources of the HBM; when only the CPU path is used, the data transmission speed is limited by the interaction efficiency between the DRAM and the CPU, thereby maximizing the performance advantages of the two types of paths. At the same time, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On the one hand, the NPU data block is directly transmitted to the HBM memory of the receiving end chip through the NPU communication path, which is suitable for the critical tensor data that needs to be accessed frequently in AI training, guarantees the rapid synchronization of core data, and reduces the waiting time in the training iteration; on the other hand, the CPU data block is first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving end DRAM buffer through the CPU communication path, which effectively relieves the bandwidth pressure between the DRAM and the CPU. In addition, parallel transmission can also avoid the efficiency loss caused by serial data transmission. Therefore, the embodiments of the present application can solve the problems of resource waste and speed bottleneck existing in the single transmission path by using the hardware-adapted tensor division, double-path parallel transmission and data compression strategy, make the hardware resources such as NPU, CPU, HBM and DRAM of the intelligent computing cluster form efficient cooperation, and finally significantly improve the data synchronization efficiency of AI training, accelerate the overall training process, and realize the improvement of the AI training efficiency of the intelligent computing cluster.

[0146] The embodiments of the present application also provide a communication energy efficiency optimization device, which can implement the above-mentioned communication energy efficiency optimization method, and the device is referred to as Figure 8 The device comprises:

[0147] The parameter obtaining module 810 is configured to obtain a transmission allocation ratio, the transmission allocation ratio being obtained by selecting a reference chip set from the intelligent computing cluster, obtaining a reference transmission parameter of the reference chip set, and calculating the reference transmission parameter, and the reference chip set including a reference sending chip and a reference receiving chip.

[0148] The data distribution module 820 is configured to, in the model training process of distributed data parallelism, obtain a to-be-synchronized tensor, and distribute the to-be-synchronized tensor into an NPU data block and a CPU data block based on the transmission allocation ratio.

[0149] The CPU data transmission module 830 is configured to transmit the CPU data block to a DRAM buffer, generate a compressed data block corresponding to the CPU data block by using the selected compressor, and transmit the compressed data block to a DRAM buffer of a corresponding receiving end chip through a CPU communication path.

[0150] The NPU data transmission module 840 is configured to transmit the NPU data block to an HBM memory of the receiving end chip through an NPU communication path.

[0151] The specific implementation of the communication energy efficiency optimization apparatus of the embodiment is basically the same as the specific implementation of the communication energy efficiency optimization method described above, and will not be repeated here.

[0152] The embodiment of the present application further provides an electronic device, comprising:

[0153] at least one memory;

[0154] at least one processor;

[0155] at least one program;

[0156] The program is stored in the memory, and the processor executes the at least one program to implement the communication energy efficiency optimization method provided in the present application. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0157] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0158] The processor 901 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is used to execute related programs to implement the technical solutions provided in the embodiments of the present application.

[0159] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 902 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present specification are implemented by software or firmware, the related program codes are stored in the memory 902 and are invoked and executed by the processor 901 to perform the communication energy efficiency optimization method of the embodiments of the present application.

[0160] The input / output interface 903 is configured to realize information input and output.

[0161] The communication interface 904 is configured to realize the communication interaction between the device and other devices. The communication can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0162] The bus 905 is configured to transmit information between various components (such as the processor 901, the memory 902, the input / output interface 903, and the communication interface 904) of the device.

[0163] The processor 901, the memory 902, the input / output interface 903, and the communication interface 904 are connected to each other through the bus 905 to realize the communication connection between them inside the device.

[0164] The embodiments of the present application also provide a storage medium. The storage medium is a storage medium, and the storage medium stores a computer program. The computer program is executed by the processor to realize the above-mentioned communication energy efficiency optimization method.

[0165] The memory is a non-transitory storage medium, which can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory, and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor. These remote memories can be connected to the processor through a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0166] The communication energy efficiency optimization method, device, equipment and storage medium provided by the embodiments of the present application can obtain a transmission allocation ratio, select a reference chip set from the intelligent computing cluster, obtain a reference transmission parameter of the reference chip set, and calculate the reference transmission parameter to obtain the reference chip set including a reference sending chip and a reference receiving chip. In the distributed data parallel model training process, a to-be-synchronized tensor is obtained, the to-be-synchronized tensor is divided into an NPU data block and a CPU data block based on the transmission allocation ratio, the CPU data block is transmitted to a DRAM buffer, a compressor is used to generate a compressed data block corresponding to the CPU data block, the compressed data block is transmitted to the DRAM buffer of the corresponding receiving end chip through a CPU communication path, and the NPU data block is transmitted to the HBM memory of the receiving end chip through an NPU communication path. The embodiments of the present application calibrate the reference chip set, determine the transmission allocation ratio in combination with the actual transmission capacity of the hardware, ensure that the ratio of the to-be-synchronized tensor divided into the NPU data block and the CPU data block can accurately match the transmission capacity of the two types of hardware, and avoid the short board effect caused by the mismatch of the hardware performance in the traditional single path transmission. For example, when only the NPU path is used, a large amount of non-critical data will occupy the high bandwidth resource of the HBM; when only the CPU path is used, the data transmission speed is limited by the interaction efficiency of the DRAM and the CPU, thereby maximizing the performance advantages of the two types of paths. At the same time, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On the one hand, the NPU data block is directly transmitted to the HBM memory of the receiving end chip through the NPU communication path, which is suitable for the critical tensor data that needs to be accessed frequently in AI training, guarantees the rapid synchronization of core data, and reduces the waiting time in the training iteration; on the other hand, the CPU data block is first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving end DRAM buffer through the CPU communication path, which effectively relieves the bandwidth pressure between the DRAM and the CPU. In addition, parallel transmission can also avoid the efficiency loss caused by serial data transmission. Therefore, through the hardware-adapted tensor division, double-path parallel transmission and data compression strategy, the embodiments of the present application can solve the problems of resource waste and speed bottleneck existing in the single transmission path, make the NPU, CPU, HBM, DRAM and other hardware resources of the intelligent computing cluster form efficient cooperation, and finally significantly improve the data synchronization efficiency of AI training and accelerate the overall training process, thereby improving the AI training efficiency of the intelligent computing cluster.

[0167] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0168] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0169] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0170] Those skilled in the art can understand that all or some steps in the above disclosed method, functional modules / units in the system and device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0171] The terms "first", "second", "third", "fourth" and the like in the description of the present application and the above-mentioned figures (if any) are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0172] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association between the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases of only A, only B and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents that the associated objects before and after are in an "or" relationship. "At least one of the following" or similar expressions means any combination of these items, including any combination of single item or multiple items. For example, at least one of a, b or c, can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0173] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0174] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0175] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0176] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various program storage media.

[0177] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, which are not limited to the scope of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A communication energy efficiency optimization method, characterized in that, Applied to a transmitting chip, the method includes: The transmission allocation ratio is obtained by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During the distributed data parallel model training process, the tensor to be synchronized is obtained, and the tensor to be synchronized is divided into NPU data blocks and CPU data blocks based on the transmission allocation ratio. The CPU data block is transmitted to the DRAM buffer, a compressed data block corresponding to the CPU data block is generated using the selected compressor, and the compressed data block is transmitted to the DRAM buffer of the corresponding receiving chip through the CPU communication path. The NPU data block is transmitted to the HBM memory of the receiving chip via the NPU communication path; The transmission allocation ratio is calculated based on the reference transmission parameters, including: obtaining the reference transmission parameters in the reference chipset, including at least the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and DRAM buffer in the reference transmitting chip or the reference receiving chip; setting the ratio parameter, and obtaining the reference NPU data and reference CPU data based on the ratio parameter and the data size of the reference data packet; calculating the total communication duration based on the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth; constructing the total energy consumption based on the total communication duration, minimizing the total energy consumption to obtain the objective optimization function, solving the objective optimization function, and obtaining the optimal solution of the ratio parameter as the transmission allocation ratio.

2. The communication energy efficiency optimization method according to claim 1, characterized in that, The calculation of the total communication duration based on the baseline NPU data, the baseline CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth includes: The NPU latency is calculated based on the baseline NPU data and the NPU bandwidth. The external transmission latency is calculated based on the baseline CPU data, the compression ratio of the compressor, and the CPU bandwidth. The internal transmission latency is calculated based on the ratio parameter, the data size, and the copy bandwidth. The CPU latency is calculated based on the external transmission latency and the internal transmission latency. Obtain the compression duration of the baseline data packet, and obtain the compression delay based on the ratio parameter and the compression duration; The total CPU latency is calculated based on the CPU latency and the compression latency, and the maximum value of the total CPU latency and the NPU latency is set as the total communication duration.

3. The communication energy efficiency optimization method according to claim 2, characterized in that, The process of constructing the total energy consumption based on the total communication duration includes: Obtain compression power parameters, CPU power parameters of the CPU communication path, NPU power parameters of the NPU communication path, CPU idle power, and NPU idle power; Calculate the difference between the total communication time and the CPU latency and the NPU latency to obtain the CPU idle time and NPU idle time; Compression energy consumption is obtained by multiplying the compression latency and the compression power parameter; CPU transmission energy consumption is obtained by multiplying the CPU latency and the CPU power parameter; CPU idle energy consumption is obtained by multiplying the CPU idle power and the CPU idle time; NPU transmission energy consumption is obtained by multiplying the NPU latency and the NPU power parameter; and NPU idle energy consumption is obtained by multiplying the NPU idle power and the NPU idle time. The total energy consumption is obtained by summing the compression energy consumption, the CPU transmission energy consumption, the CPU idle energy consumption, the NPU transmission energy consumption, and the NPU idle energy consumption.

4. The communication energy efficiency optimization method according to claim 3, characterized in that, Solving the objective optimization function to obtain the optimal solution for the proportional parameter as the transmission allocation ratio includes: The objective optimization function is solved using a convex function optimization method. The solution corresponding to the point where the sum of the CPU latency and the compressed latency is consistent with the NPU latency is set as the abrupt change point of the proportional parameter. Calculate the energy consumption values ​​corresponding to the two endpoint values ​​and the abrupt change point of the ratio parameter, and select the ratio parameter corresponding to the smallest energy consumption value as the transmission allocation ratio.

5. A communication energy efficiency optimization method, characterized in that, Applied to a receiver chip, the method includes: Acquire compressed data blocks transmitted from the sending chip to the DRAM buffer via the CPU communication path; Obtain NPU data blocks transmitted from the transmitting chip to HBM memory via the NPU communication path, wherein the compressed data blocks and the NPU data blocks are obtained by the transmitting chip according to any one of claims 1 to 4; The compressed data block is decompressed to obtain a decompressed data block, which is then transferred to the HBM memory. The decompressed data block and the NPU data block are then merged to obtain the target data corresponding to the tensor to be synchronized.

6. A communication energy efficiency optimization device, characterized in that, The device, applied to a transmitting chip, includes: Parameter acquisition module: used to acquire the transmission allocation ratio, which is obtained by selecting a benchmark chipset from the intelligent computing cluster, acquiring the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. Data allocation module: used to obtain tensors to be synchronized during the distributed data parallel model training process, and divide the tensors to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio; CPU data transmission module: used to transmit the CPU data block to the DRAM buffer, generate a compressed data block corresponding to the CPU data block using the selected compressor, and transmit the compressed data block to the DRAM buffer of the corresponding receiving chip through the CPU communication path; NPU data transmission module: used to transmit the NPU data block to the HBM memory of the receiving chip through the NPU communication path; The transmission allocation ratio is calculated based on the reference transmission parameters, including: obtaining the reference transmission parameters in the reference chipset, including at least the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and DRAM buffer in the reference transmitting chip or the reference receiving chip; setting the ratio parameter, and obtaining the reference NPU data and reference CPU data based on the ratio parameter and the data size of the reference data packet; calculating the total communication duration based on the reference NPU data, the reference CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth; constructing the total energy consumption based on the total communication duration, minimizing the total energy consumption to obtain the objective optimization function, solving the objective optimization function, and obtaining the optimal solution of the ratio parameter as the transmission allocation ratio.

7. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the communication energy efficiency optimization method according to any one of claims 1 to 5.

8. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the communication energy efficiency optimization method according to any one of claims 1 to 5.

Images