Communication energy efficiency optimization method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a communication energy efficiency optimization method and device, equipment and a storage medium, and relates to the technical field of intelligent computing clusters. The method comprises the following steps: selecting a reference chipset from an intelligent calculation cluster, acquiring a reference transmission parameter of the reference chipset, calculating according to the reference transmission parameter to obtain a transmission distribution proportion, dividing a tensor to be synchronized into an NPU data block and a CPU data block based on the transmission distribution proportion during a distributed data parallel model training process, and performing data synchronization on the NPU data block and the CPU data block. And transmitting the CPU data block to the DRAM buffer area, generating a compressed data block corresponding to the CPU data block by using the selected compressor, transmitting the compressed data block to the DRAM buffer area of the corresponding receiving end chip through the CPU communication path, and transmitting the NPU data block to the HBM memory of the receiving end chip through the NPU communication path. The transmission distribution proportion is determined by combining the actual transmission capacity of hardware, it is ensured that to-be-synchronized tensor division can be accurately matched with the transmission capacity of two types of hardware, and the intelligent computing cluster training efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent computing cluster technology, and in particular to communication energy efficiency optimization methods, apparatus, devices and storage media. Background Technology

[0002] With the rapid development of large-scale AI models in natural language processing, computer vision, and multimodal tasks, intelligent computing clusters are increasingly being used in scientific research, industry, and various social fields. However, training a large language model with hundreds of billions of parameters can consume several megawatt-hours of energy, making energy consumption a key issue restricting the efficient operation of intelligent computing clusters. Therefore, reducing the amount of communication data and shortening the communication time to reduce overall communication energy consumption is one of the important ways to improve the energy efficiency of the cluster.

[0003] In related technologies, each AI node in an intelligent computing cluster typically contains two core processors: an NPU and a CPU. During model training, the NPU handles the main AI computational tasks, such as matrix multiplication, convolution operations, and computationally intensive operations like forward and backward propagation. The CPU, on the other hand, is responsible for general computing, task scheduling, data preprocessing, and auxiliary computations. However, in this division of labor, CPU channels are mostly idle during the training phase and do not participate in cross-node data transfer, resulting in limited energy savings. Summary of the Invention

[0004] The main objective of this application is to propose a communication energy efficiency optimization method, apparatus, device, and storage medium to improve the training efficiency of AI training in intelligent computing clusters.

[0005] To achieve the above objectives, a first aspect of this application proposes a communication energy efficiency optimization method, comprising: Applied to a transmitting chip, the method includes: The transmission allocation ratio is obtained by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During the distributed data parallel model training process, the tensor to be synchronized is obtained, and the tensor to be synchronized is divided into NPU data blocks and CPU data blocks based on the transmission allocation ratio. The CPU data block is transmitted to the DRAM buffer, a compressed data block corresponding to the CPU data block is generated using the selected compressor, and the compressed data block is transmitted to the DRAM buffer of the corresponding receiving chip through the CPU communication path. The NPU data block is transmitted to the HBM memory of the receiving chip via the NPU communication path.

[0006] In some embodiments, the transmission allocation ratio is calculated based on the reference transmission parameters, including: The reference transmission parameters are obtained, including at least the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and the DRAM buffer in the reference transmitting chip or the reference receiving chip. The transmission allocation ratio is calculated based at least on the CPU bandwidth, the NPU bandwidth, and the copy bandwidth.

[0007] In some embodiments, calculating the transmission allocation ratio based at least on the CPU bandwidth, the NPU bandwidth, and the copy bandwidth includes: Set a scaling parameter, and obtain the baseline NPU data and baseline CPU data based on the scaling parameter and the data size of the baseline data packet; The total communication duration is calculated based on the baseline NPU data, the baseline CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth. The total energy consumption is constructed based on the total communication duration. The target optimization function is obtained by minimizing the total energy consumption. The optimal solution of the ratio parameter is obtained by solving the target optimization function and serving as the transmission allocation ratio.

[0008] In some embodiments, calculating the total communication duration based on the baseline NPU data, the baseline CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth includes: The NPU latency is calculated based on the baseline NPU data and the NPU bandwidth. The external transmission latency is calculated based on the baseline CPU data, the compression ratio of the compressor, and the CPU bandwidth. The internal transmission latency is calculated based on the ratio parameter, the data size, and the copy bandwidth. The CPU latency is calculated based on the external transmission latency and the internal transmission latency. Obtain the compression duration of the baseline data packet, and obtain the compression delay based on the ratio parameter and the compression duration; The total CPU latency is calculated based on the CPU latency and the compression latency, and the maximum value of the total CPU latency and the NPU latency is set as the total communication duration.

[0009] In some embodiments, constructing the total energy consumption based on the total communication duration includes: Obtain compression power parameters, CPU power parameters of the CPU communication path, NPU power parameters of the NPU communication path, CPU idle power, and NPU idle power; Calculate the difference between the total communication time and the CPU latency and the NPU latency to obtain the CPU idle time and NPU idle time; Compression energy consumption is obtained by multiplying the compression latency and the compression power parameter; CPU transmission energy consumption is obtained by multiplying the CPU latency and the CPU power parameter; CPU idle energy consumption is obtained by multiplying the CPU idle power and the CPU idle time; NPU transmission energy consumption is obtained by multiplying the NPU latency and the NPU power parameter; and NPU idle energy consumption is obtained by multiplying the NPU idle power and the NPU idle time. The total energy consumption is obtained by summing the compression energy consumption, the CPU transmission energy consumption, the CPU idle energy consumption, the NPU transmission energy consumption, and the NPU idle energy consumption.

[0010] In some embodiments, solving the objective optimization function to obtain the optimal solution of the proportional parameter as the transmission allocation ratio includes: The objective optimization function is solved using a convex function optimization method. The solution corresponding to the point where the sum of the CPU latency and the compressed latency is consistent with the NPU latency is set as the abrupt change point of the proportional parameter. Calculate the energy consumption values ​​corresponding to the two endpoint values ​​and the abrupt change point of the ratio parameter, and select the ratio parameter corresponding to the smallest energy consumption value as the transmission allocation ratio.

[0011] To achieve the above objectives, a second aspect of this application proposes a communication energy efficiency optimization method applied to a receiver chip, the method comprising: Acquire compressed data blocks transmitted from the sending chip to the DRAM buffer via the CPU communication path; Obtain NPU data blocks transmitted from the transmitting chip to HBM memory via the NPU communication path, wherein the compressed data blocks and the NPU data blocks are obtained by the transmitting chip according to the communication energy efficiency optimization method described in any one of the first aspects; The compressed data block is decompressed to obtain a decompressed data block, which is then transferred to the HBM memory. The decompressed data block and the NPU data block are then merged to obtain the target data corresponding to the tensor to be synchronized.

[0012] To achieve the above objectives, a third aspect of this application provides a communication energy efficiency optimization device applied to a transmitting chip, the device comprising: Parameter acquisition module: used to acquire the transmission allocation ratio, which is obtained by selecting a benchmark chipset from the intelligent computing cluster, acquiring the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. Data allocation module: used to obtain tensors to be synchronized during the distributed data parallel model training process, and divide the tensors to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio; CPU data transmission module: used to transmit the CPU data block to the DRAM buffer, generate a compressed data block corresponding to the CPU data block using the selected compressor, and transmit the compressed data block to the DRAM buffer of the corresponding receiving chip through the CPU communication path; NPU data transmission module: used to transmit the NPU data block to the HBM memory of the receiving chip through the NPU communication path.

[0013] To achieve the above objectives, a fourth aspect of the present application provides an electronic device, the electronic device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method described in the first or second aspect.

[0014] To achieve the above objectives, a fifth aspect of the present application provides a storage medium storing a computer program that, when executed by a processor, implements the method described in the first or second aspect.

[0015] The communication energy efficiency optimization method, apparatus, device, and storage medium proposed in this application obtain a transmission allocation ratio. This ratio is calculated by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on these parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During distributed data parallel model training, a tensor to be synchronized is obtained. Based on the transmission allocation ratio, the tensor to be synchronized is divided into NPU data blocks and CPU data blocks. The CPU data blocks are transmitted to a DRAM buffer. A selected compressor generates compressed data blocks corresponding to the CPU data blocks. The compressed data blocks are transmitted through the CPU communication path to the DRAM buffer of the corresponding receiving chip. The NPU data blocks are transmitted through the NPU communication path to the HBM memory of the receiving chip. This application embodiment determines the transmission allocation ratio through benchmark chipset calibration and combined with the actual hardware transmission capabilities. This ensures that the ratio of the tensor to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. For example, when using only the NPU path, a large amount of non-critical data will consume the high bandwidth resources of HBM; when using only the CPU path, the data transmission speed is limited by the interaction efficiency between DRAM and CPU, thus maximizing the performance advantages of both paths. Simultaneously, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On one hand, NPU data blocks are directly transmitted to the HBM memory of the receiving chip through the NPU communication path, adapting to the critical tensor data that needs high-frequency access in AI training, ensuring rapid synchronization of core data, and reducing waiting time during training iterations; on the other hand, CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving DRAM buffer through the CPU communication path, effectively alleviating the bandwidth pressure between DRAM and CPU. Furthermore, parallel transmission can avoid the efficiency loss caused by serial data transmission. Therefore, this application embodiment, through hardware-adapted tensor partitioning, dual-path parallel transmission, and data compression strategies, can solve the problems of resource waste and speed bottlenecks existing in a single transmission path, enabling the hardware resources of the intelligent computing cluster, such as NPU, CPU, HBM, and DRAM, to form efficient collaboration, ultimately significantly improving the data synchronization efficiency of AI training, accelerating the overall training process, and achieving an improvement in the AI ​​training efficiency of the intelligent computing cluster. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the communication process of computing nodes in the intelligent computing cluster provided in the embodiments of this application.

[0017] Figure 2 This is a flowchart of the communication energy efficiency optimization method provided in the embodiments of this application.

[0018] Figure 3This is a flowchart of calculating the transmission allocation ratio based on the reference transmission parameters, provided in an embodiment of this application.

[0019] Figure 4 This is a flowchart provided in the embodiments of this application for calculating the transmission allocation ratio based at least on CPU bandwidth, NPU bandwidth, and copy bandwidth.

[0020] Figure 5 This is a flowchart provided in an embodiment of the present application for calculating the total communication time based on the baseline NPU data, baseline CPU data, CPU bandwidth, NPU bandwidth, and copy bandwidth.

[0021] Figure 6 This is a flowchart of constructing total energy consumption based on the total communication duration, provided in an embodiment of this application.

[0022] Figure 7 This is a flowchart of a communication energy efficiency optimization method for a receiver chip provided in an embodiment of this application.

[0023] Figure 8 This is a structural block diagram of a communication energy efficiency optimization device provided in another embodiment of this application.

[0024] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] It should be noted that although functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0028] First, let's analyze some of the terms used in this application: Artificial Intelligence (AI) is a new branch of computer science that studies, develops, and applies theories, methods, technologies, and systems to simulate, extend, and expand human intelligence. It aims to understand the essence of intelligence and produce intelligent machines that can react in a way similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thought. Furthermore, AI utilizes digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceiving the environment, acquiring knowledge, and using that knowledge to achieve optimal results.

[0029] With the rapid development of large AI models in natural language processing, computer vision, and multimodal tasks, intelligent computing clusters are being used more and more widely in scientific research, industry, and society. However, according to publicly available data, training a large language model with hundreds of billions of parameters can consume several megawatt-hours of energy. It is evident that energy consumption has become a key issue restricting the efficient operation of intelligent computing clusters, which not only brings high operating costs but also exacerbates carbon emissions.

[0030] In distributed AI training, especially in parallel training scenarios for large-scale models, parameter gradients or activation values ​​need to be frequently synchronized between different computing nodes. According to existing measurements, this type of communication often accounts for 20% to 40% of training time, and can even become a bottleneck in large-scale clusters. Since the communication process occupies high-speed interconnect channels and keeps devices active, its energy consumption is also significant. Therefore, reducing the amount of communication data and shortening the communication duration, thereby reducing overall communication energy consumption, is one of the important ways to improve cluster energy efficiency.

[0031] In related technologies, each AI node in an intelligent computing cluster typically contains two core processors: an NPU and a CPU. During model training, the NPU handles the main AI computational tasks, such as matrix multiplication, convolution operations, and computationally intensive operations like forward and backward propagation. The CPU, on the other hand, handles general computation, task scheduling, data preprocessing, and auxiliary computations. Currently, AI training implementations often utilize only the dedicated high-speed communication channels of the NPU within the AI ​​server, such as HCCS, NVLink, and GPU RDMA, for cross-node parameter synchronization, for ease of use and software stack compatibility, rather than leveraging the server CPU's conventional network channels, such as PCIe-based NICs, RoCE, and Ethernet RDMA. While this simplifies development, it also means that the CPU channels are mostly idle during the training phase, failing to participate in cross-node data transmission. If the original training communication data is evenly distributed between the two channels, the bandwidth of the CPU channels is typically significantly lower than that of the dedicated NPU channels. This difference means that even distribution not only fails to improve overall synchronization speed but may also lengthen the critical path. Therefore, an unreasonable allocation ratio also fails to effectively reduce energy consumption.

[0032] Based on this, embodiments of this application provide a communication energy efficiency optimization method, apparatus, device, and storage medium applied to intelligent computing cluster training compression scenarios. By calibrating a benchmark chipset and combining it with the actual hardware transmission capabilities, the transmission allocation ratio is determined to ensure that the ratio of tensors to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. For example, when using only the NPU path, a large amount of non-critical data will occupy the high bandwidth resources of HBM; when using only the CPU path, the data transmission speed is limited by the interaction efficiency between DRAM and CPU, thus maximizing the performance advantages of both paths. Simultaneously, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On one hand, NPU data blocks are directly transmitted to the HBM memory of the receiving chip through the NPU communication path, adapting to the critical tensor data that needs high-frequency access in AI training, ensuring rapid synchronization of core data, and reducing waiting time during training iterations; on the other hand, CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving DRAM buffer through the CPU communication path, effectively alleviating the bandwidth pressure between DRAM and CPU. Furthermore, parallel transmission can avoid the efficiency loss caused by serial data transmission. Therefore, the embodiments of this application, through hardware-adapted tensor partitioning, dual-path parallel transmission, and data compression strategies, can solve the problems of resource waste and speed bottlenecks existing in a single transmission path, enabling the hardware resources of the intelligent computing cluster, such as NPU, CPU, HBM, and DRAM, to form efficient collaboration, ultimately significantly improving the data synchronization efficiency of AI training, accelerating the overall training process, and realizing the improvement of AI training efficiency of the intelligent computing cluster.

[0033] This application provides a communication energy efficiency optimization method, apparatus, device, and storage medium, which are specifically described through the following embodiments. First, the communication energy efficiency optimization method in this application embodiment is described.

[0034] This application's embodiments can acquire and process relevant data based on artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0035] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0036] The communication energy efficiency optimization method provided in this application relates to the field of intelligent computing cluster technology. The communication energy efficiency optimization method provided in this application can be executed by a terminal or a server, or it can be executed by a computer program running on a terminal or a server. For example, the computer program can be a native program or software module in an operating system; it can be a native application (APP), i.e., a program that needs to be installed in the operating system to run, such as a client that supports communication energy efficiency optimization, i.e., a program that only needs to be downloaded to a browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be any form of application, module, or plugin. The terminal communicates with the server through a network. The communication energy efficiency optimization method can be executed by the terminal or the server alone, or by the terminal and the server collaboratively.

[0037] In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, or smartwatch, etc. The server can be a standalone server, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms; it can also be a service node in a blockchain system, where the service nodes form a peer-to-peer (P2P) network. The P2P protocol is an application layer protocol running on top of the Transmission Control Protocol (TCP). The terminal and server can connect via Bluetooth, Universal Serial Bus (USB), or a network, etc., and this embodiment does not impose any limitations.

[0038] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0039] The communication energy efficiency optimization method in the embodiments of this application is described below.

[0040] First refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the communication process of computing nodes in an intelligent computing cluster provided in this application embodiment. Each AI computing node is a chip. (Refer to...) Figure 1 In an intelligent computing cluster, a large number of AI computing nodes are deployed to work collaboratively to support the training of large-scale artificial intelligence models. Each AI node contains two types of core processors: an NPU (Neural Processing Unit) processor for high-performance computing in deep learning, and a CPU (Central Processing Unit) processor for general-purpose computing. During training, the NPU processor is responsible for performing the main AI calculations, such as matrix multiplication, convolution operations, and computationally intensive tasks like forward and backward propagation; while the CPU processor is responsible for general-purpose computing, task scheduling, data preprocessing, and auxiliary calculations.

[0041] In one embodiment, reference is made to Figure 1 As shown, the NPU processor is equipped with high-speed HBM (High Bandwidth Memory) as its local storage. All model parameters, activation values, and intermediate computation results required for AI training are stored in HBM memory. When NPU processors from different chips communicate, a data path can be directly established between the HBM memories of different computing nodes as the NPU communication path, thereby enabling high-speed data transfer from the HBM memory of AI computing node 1 to the HBM memory of AI computing node 2. This NPU communication path has extremely high bandwidth, typically hundreds of GB / s, making it suitable for use in scenarios such as large-scale gradient synchronization and activation swapping.

[0042] In one embodiment, reference is made to Figure 1 As shown, the CPU processor uses conventional DRAM (Dynamic Random-Access Memory) as local storage. Its access latency is higher and its bandwidth is typically lower than HBM, but its capacity is larger. When communicating between CPUs on the same chip, the DRAM buffer of AI computing node 1 can directly transfer data to the DRAM buffer of AI computing node 2 via a network interface, such as RoCE or InfiniBand based on RDMA. This communication link is called the CPU communication path. The bandwidth of this CPU communication path is generally significantly smaller than that of the NPU communication path, but it can still handle data synchronization tasks.

[0043] In one embodiment, if data originally stored in HBM memory needs to be transferred via the CPU communication path, an additional memory copying step is required. Specifically, the data is first copied from HBM memory to a DRAM buffer on AI compute node 1. Then, the CPU processor compresses the data in the DRAM buffer to reduce its size. The compressed data is then transferred to the DRAM buffer of AI compute node 2 via the CPU communication path. At the receiving end, the CPU processor of AI compute node 2 decompresses the data and finally copies the decompressed data back to the HBM memory of AI compute node 2 for subsequent NPU computation. Since the resident occupancy of the DRAM buffer is typically low during AI training tasks, sufficient DRAM buffer space can be reserved as a data buffer for the CPU communication path, ensuring the smooth execution of the above process.

[0044] As described above, the core idea of ​​this application's embodiments is to simultaneously enable both the NPU and CPU communication paths in cross-node communication during distributed training. The transmission allocation ratio is determined based on the actual hardware transmission capabilities, and the data to be transmitted is split into two parts at an appropriate ratio: one part is transmitted directly through the NPU communication path, and the other part is transmitted after compression through the CPU communication path. This ensures that the ratio of the tensor to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. This fully utilizes the idle bandwidth resources of the CPU communication path during the training phase and reduces the data volume of the slow channel through compression, allowing the two channels to complete tasks in parallel as much as possible. Compared to a scheme using only a single communication path, dual-channel parallel communication can significantly shorten communication time with the same total data volume and reduce overall energy consumption by reducing device idle waiting time, thereby achieving higher communication efficiency and energy efficiency.

[0045] It is understandable that the above Figure 1During communication between AI computing node 1 and AI computing node 2, AI computing node 1 can act as a transmitting chip, while AI computing node 2 can act as a receiving chip. If AI computing node 2 needs to send data to AI computing node 3, then AI computing node 2 becomes the transmitting chip corresponding to AI computing node 3.

[0046] The following describes a communication energy efficiency optimization method according to an embodiment of this application, applied to a scenario of intelligent computing cluster training compression.

[0047] In one embodiment, Figure 2 This is an optional flowchart of the communication energy efficiency optimization method provided in the embodiments of this application. Figure 2 The method described herein, when applied to a transmitting chip, may include, but is not limited to, steps 110 to 140. It is also understood that this embodiment... Figure 2 The order of steps 110 to 140 is not specifically limited. The order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0048] Step 110: Obtain the transmission allocation ratio.

[0049] In one embodiment, during a distributed AI training process, the intelligent computing cluster can set the same transmission allocation ratio for all participating computing nodes, thereby reducing computational complexity. The transmission allocation ratio can be pre-calculated. First, a pair of transmitting and receiving chips is randomly selected from the intelligent computing cluster to form a baseline chip set, which includes a baseline transmitting chip and a baseline receiving chip. Then, the baseline transmission parameters of the baseline chip set are obtained, and the transmission allocation ratio is calculated based on these parameters. This parameter is then applied to the entire intelligent computing cluster. If a computing node acts as a transmitting chip, the amount of data allocated to the NPU communication path and the CPU communication path is determined according to this transmission allocation ratio.

[0050] In one embodiment, reference is made to Figure 3 , Figure 3 This is a flowchart of calculating the transmission allocation ratio based on the reference transmission parameters provided in this application embodiment, specifically including the following steps: Step 310: Obtain the reference transmission parameters in the reference chipset, including at least the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and the DRAM buffer in the reference transmitting chip or the reference receiving chip.

[0051] In one embodiment, a test data packet is generated, which can be consistent with the actual load of training tensor data in a real-world scenario. This test data packet is then used for benchmark calibration. The test data packet is transmitted within a benchmark chipset to test the CPU bandwidth corresponding to each CPU communication path. NPU bandwidth of NPU communication path Simultaneously, either the reference transmitting chip or the reference receiving chip is selected arbitrarily to test the copy bandwidth between the HBM memory and the DRAM buffer. It is understood that the copy bandwidth here can be the uplink bandwidth for transferring data from the HBM memory to the DRAM buffer in the reference transmitting chip, or the downlink bandwidth between the DRAM buffer and the HBM memory in the reference receiving chip, or the average of the uplink and downlink bandwidths. This embodiment does not specifically limit this.

[0052] Simultaneously, CPU power parameters corresponding to the CPU communication path transmission process are obtained through a power meter. NPU power parameters of the NPU communication path CPU idle power during idle time in the reference transmitter chip And the NPU idle power of the NPU processor in the reference receiver chip during idle time. .

[0053] In addition, since data compression is also required in the CPU processor, it is necessary to obtain the compression time corresponding to different data volumes using different compressors under different compression parameters R (such as compression ratio) in the CPU processor. and compression power parameters Understandably, the relationships between data size, compressor, compression parameters, compression time, and compression power parameters can be solved in advance. However, the parameters may differ under each condition and need to be measured separately.

[0054] Thus, for the test data packet, the CPU bandwidth related to the energy consumption parameter calibration can be obtained. NPU bandwidth Copy bandwidth CPU power parameters NPU power parameters CPU idle power NPU idle power Compression time and compression power parameters These data are used as baseline transmission parameters. It's understandable that these baseline transmission parameters can be obtained in advance through offline testing and can be read directly as needed later.

[0055] Step 320: Calculate the transmission allocation ratio based at least on CPU bandwidth, NPU bandwidth, and copy bandwidth.

[0056] In one embodiment, reference is made to Figure 4 , Figure 4 This is a flowchart provided in an embodiment of the present application for calculating the transmission allocation ratio based at least on CPU bandwidth, NPU bandwidth, and copy bandwidth, specifically including the following steps: Step 410: Set the scaling parameters, and obtain the baseline NPU data and baseline CPU data based on the scaling parameters and the data size of the baseline data packet.

[0057] In one embodiment, the scaling parameter is assumed to be ,in, The number of test data packets is Therefore, the baseline NPU data is The baseline CPU data is In other words, the test data packet contains... The corresponding amount of data is transmitted from the NPU communication path, while... The corresponding amount of data is transmitted from the CPU communication path.

[0058] Step 420: Calculate the total communication duration based on the baseline NPU data, baseline CPU data, CPU bandwidth, NPU bandwidth, and copy bandwidth.

[0059] In one embodiment, reference is made to Figure 5 , Figure 5 This is a flowchart provided in an embodiment of the present application for calculating the total communication time based on baseline NPU data, baseline CPU data, CPU bandwidth, NPU bandwidth, and copy bandwidth, specifically including the following steps: Step 510: Calculate the NPU latency based on the baseline NPU data and NPU bandwidth; calculate the external transmission latency based on the baseline CPU data, the compressor's compression ratio, and the CPU bandwidth; calculate the internal transmission latency based on the ratio parameter, data size, and copy bandwidth; and calculate the CPU latency based on the external and internal transmission latencies.

[0060] In one embodiment, the NPU latency is calculated based on baseline NPU data and NPU bandwidth. Represented as:

[0061] The external transmission delay, calculated based on baseline CPU data, the compressor's compression ratio, and CPU bandwidth, is expressed as follows: The internal transmission delay, calculated based on the proportional parameters, data size, and copy bandwidth, is expressed as follows: The internal transmission delay here is the uplink delay of transmitting data from the HBM memory in the reference transmitting chip to the DRAM buffer plus the downlink delay of transmitting data from the DRAM buffer in the reference receiving chip to the HBM memory.

[0062] Therefore, the CPU latency is calculated based on the external transmission latency and the internal transmission latency. Represented as:

[0063] Step 520: Obtain the compression duration of the baseline data packet, and obtain the compression delay based on the ratio parameter and the compression duration.

[0064] In one embodiment, the compression delay obtained based on the scaling parameter and compression duration is expressed as: .

[0065] Step 530: Calculate the total CPU latency based on the CPU latency and compression latency, and set the maximum value of the total CPU latency and NPU latency as the total communication duration.

[0066] In one embodiment, the total CPU latency calculated based on CPU latency and compression latency is expressed as: The maximum value of the CPU total latency and the NPU latency is set as the total communication duration. Therefore, the total communication duration T is expressed as:

[0067] Step 430: Construct the total energy consumption based on the total communication duration, minimize the total energy consumption to obtain the objective optimization function, solve the objective optimization function, and obtain the optimal solution of the proportional parameter as the transmission allocation ratio.

[0068] In one embodiment, reference is made to Figure 6 , Figure 6 This is a flowchart of constructing total energy consumption based on the total communication duration provided in this application embodiment, specifically including the following steps: Step 610: Obtain compression power parameters, CPU power parameters of CPU communication path, NPU power parameters of NPU communication path, CPU idle power, and NPU idle power.

[0069] Step 620: Calculate the difference between the total communication time and the CPU latency and NPU latency to obtain the CPU idle time and NPU idle time.

[0070] In one embodiment, the CPU idle time is obtained based on the difference between the total communication duration and the CPU latency. Based on the difference between the total communication duration and the NPU latency, the NPU idle time is calculated as follows: .

[0071] Step 630: Obtain compression energy consumption by multiplying compression latency and compression power parameters; obtain CPU transmission energy consumption by multiplying CPU latency and CPU power parameters; obtain CPU idle energy consumption by multiplying CPU idle power and CPU idle time; obtain NPU transmission energy consumption by multiplying NPU latency and NPU power parameters; and obtain NPU idle energy consumption by multiplying NPU idle power and NPU idle time.

[0072] In one embodiment, based on compression delay With compression power parameters The compression energy consumption obtained by multiplying the products is According to CPU latency CPU power parameters The CPU transmission power consumption obtained by multiplying is According to CPU idle power CPU idle time The CPU idle power consumption obtained by multiplying is According to NPU latency With NPU power parameters The NPU transmission power consumption obtained by multiplying the products is According to NPU idle power NPU idle time The NPU idle power consumption obtained by multiplying the product of the two is: .

[0073] Step 640: Accumulate the compression energy consumption, CPU transmission energy consumption, CPU idle energy consumption, NPU transmission energy consumption, and NPU idle energy consumption to obtain the total energy consumption.

[0074] In one embodiment, for the scaling parameter The total energy consumption is obtained by summing up the compression energy consumption, CPU transmission energy consumption, CPU idle energy consumption, NPU transmission energy consumption, and NPU idle energy consumption. Represented as:

[0075]

[0076] Next, the optimization objective is to minimize the total energy consumption. Therefore, minimizing the total energy consumption yields the objective optimization function. , is represented as:

[0077]

[0078] Next, the objective optimization function is solved to obtain the optimal solution of the proportional parameter as the transmission allocation ratio.

[0079] In one embodiment, for a given compressor and its compression parameters R and compression duration... The objective function is about Since the function is a convex function, the objective optimization function is solved using the convex function optimization method. The solution corresponding to the sum of CPU latency and compression latency being consistent with NPU latency is set as the mutation point of the proportional parameter. The energy consumption values ​​corresponding to the two endpoint values ​​of the proportional parameter and the mutation point are calculated respectively. The proportional parameter corresponding to the minimum energy consumption value is selected as the transmission allocation ratio.

[0080] For convex functions, the minimum value of a convex function on a closed interval must occur at the endpoints of the interval, or at a stationary point where its derivative equals zero (i.e., the "trough" of the function). Therefore, total energy consumption... The minimum value must be at both endpoints. or And the abrupt change point of the max maxima function. The value is obtained from the location. In this embodiment, the point where the sum of CPU latency and compression latency is consistent with NPU latency is set as the mutation point, that is, the mutation point. Defined as The function exactly satisfies The point is represented as:

[0081] Only when traffic data is allocated based on the value corresponding to the mutation point will the duration of the CPU communication path and the NPU communication path be exactly equal.

[0082] Therefore, based on It can be obtained The analytical expression for the known parameters, after simplification, is expressed as:

[0083]

[0084] Therefore, calculate separately , as well as The energy consumption value at any given time is used to select the proportional parameter corresponding to the minimum energy consumption value as the transmission allocation ratio. It can be understood that if it is... This means that no data needs to be allocated to the CPU communication path. This means that all data is transmitted using the CPU communication path. This means according to This ratio allocates data to the CPU communication path for transmission. Even here... This is also different from the single transmission in related technologies. This is because the embodiment of this application determines the most suitable transmission allocation ratio based on the actual hardware situation, and it is a value calculated based on the actual situation.

[0085] In one embodiment, due to The computational cost is small, and it can be calculated directly using a one-dimensional fine-grained grid or a bipartite search without resorting to mutation points. Multiple values ​​are selected in a fully covered interval, and energy consumption values ​​are calculated for each value. The convexity of the function is utilized to ensure that the global minimum is found quickly.

[0086] In one embodiment, during the calculation of the transmission allocation ratio, the compression parameters R and compression duration are adjusted for different compressors. Joint optimization is performed to select the optimal transmission allocation ratio. The compressor with the smallest value is selected as the compressor used subsequently. In other words, this embodiment can further select the compressor with the best energy consumption optimization effect based on the transmission allocation ratio, thereby further reducing energy consumption. Since the number of compressor types is generally less than ten, only a small amount of calculation is required, and this embodiment does not limit the actual use.

[0087] Step 120: During the distributed data parallel model training process, obtain the tensor to be synchronized, and divide the tensor to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio.

[0088] In one embodiment, after determining the transmission allocation ratio, energy efficiency optimization for compressed communication is performed based on this ratio during the specific distributed data-parallel model training process. This requires modifying the PyTorch communication layer to achieve dual-channel data transmission. For training jobs using distributed data parallelism, a custom communication hook can be registered to replace the default gradient AllReduce function. At the sending chip, the communication hook first obtains the tensor to be synchronized, and then, based on the pre-calculated transmission allocation ratio... It is divided into two parts, namely NPU data blocks and CPU data blocks.

[0089] Step 130: Transfer the CPU data block to the DRAM buffer, generate the compressed data block corresponding to the CPU data block using the selected compressor, and transfer the compressed data block to the DRAM buffer of the corresponding receiving chip through the CPU communication path.

[0090] In one embodiment, to allow the compression and transmission of the CPU communication path to proceed in parallel, refer to Figure 1The transmitting chip immediately moves the CPU data block to its own DRAM buffer, compresses it using a selected compressor to generate a compressed data block, and then transmits the compressed data block to the corresponding receiving chip's DRAM buffer via the CPU communication path. It's understandable that the selected compressor is pre-optimized by combining the compression parameters R and compression duration when calculating the transmission allocation ratio. Obtained.

[0091] Step 140: Transmit the NPU data block to the HBM memory of the receiving chip through the NPU communication path.

[0092] In one embodiment, NPU data blocks are stored in the HBM memory of the NPU processor. The communication primitives of the NPU communication path are immediately queued into the device stream for execution and transmitted to the HBM memory of the receiving chip through the NPU communication path, so that the two paths overlap and run in parallel on the time axis.

[0093] In one embodiment, reference is made to Figure 7 , Figure 7 This is a flowchart of a communication energy efficiency optimization method for a receiver chip provided in this application embodiment, which specifically includes the following steps: Step 710: Obtain the compressed data block transmitted from the sending chip to the DRAM buffer via the CPU communication path.

[0094] In one embodiment, reference is made to Figure 1 The receiving chip obtains compressed data blocks through the CPU communication path, and the compressed data blocks are stored in the DRAM buffer corresponding to the receiving chip.

[0095] Step 720: Obtain the NPU data block transmitted from the sending chip to the HBM memory via the NPU communication path.

[0096] In one embodiment, the receiving chip also acquires NPU data blocks through the NPU communication path, and the NPU data blocks are stored in the HBM memory corresponding to the receiving chip.

[0097] Step 730: Decompress the compressed data block to obtain a decompressed data block, transfer the decompressed data block to HBM memory, and merge the decompressed data block and NPU data block to obtain the target data corresponding to the tensor to be synchronized.

[0098] In one embodiment, the receiving chip needs to use the CPU processor to decompress the compressed data blocks in the DRAM buffer to obtain decompressed data blocks. Due to the difference in arrival time and location of data between the NPU and CPU communication paths, the compressed data blocks received by the CPU communication path must first fall into the DRAM buffer and remain temporarily stored there after decompression, awaiting merging with the data arriving from the NPU communication path in the HBM memory. In typical AI training, since all parameters are stored in the HBM memory of the NPU processor, the host DRAM buffer usage is usually low, mainly used for data loading and a small amount of control caching. Therefore, a larger DRAM buffer can be reserved as a temporary storage pool for the CPU communication path without affecting the training process. Next, once both data streams are complete, the receiving chip transmits the decompressed data blocks to the HBM memory and merges the decompressed data blocks and NPU data blocks to obtain the target data corresponding to the tensor to be synchronized.

[0099] Since cross-node synchronization in AI training typically involves large packets on the order of hundreds of gigabytes, communication time is primarily constrained by bandwidth rather than end-to-end latency. The latency difference between the CPU and NPU channels, ranging from microseconds to milliseconds, has a negligible impact on the total duration. It is sufficient to ensure that the segmentation ratio and buffer size meet a stable throughput balance. In summary, by proportionally segmenting the tensors to be synchronized using a pre-calibrated transmission allocation ratio, and coordinating this with the receiver chip's reassembly process, training stability can be maintained while effectively converting the idle bandwidth of the CPU communication path into considerable synchronization time and energy savings.

[0100] In one embodiment, the optimization effect of the communication energy efficiency optimization method of this application embodiment is tested using simulation tools. The hardware and parameter settings used are as follows: NPU bandwidth is... =112GB / s, CPU bandwidth is =35GB / s, the bidirectional copy bandwidth between HBM and DRAM is =55GB / s. The node power parameter measurement specifically includes: CPU power parameters of the CPU communication path. =238W, CPU idle power =180W, NPU power parameters of NPU communication path =842W, NPU idle power =716W, the compression power parameters during the compression and decompression processes are as follows: =336W, and the test data volume for a single cross-node test is D=316GB.

[0101] Firstly, in the baseline scheme using only a single NPU communication path in related technologies, the total communication energy consumption is E=2883.5J. In this case, all data is directly transmitted between the HBM memory of the two nodes. Although the bandwidth is high, the NPU processor is active for a long time, while the CPU processor is completely idle, resulting in the inability to further optimize energy efficiency.

[0102] If the dual-channel parallel communication of this application embodiment is introduced, but without data compression, a portion of the data is allocated to the CPU channel path for transmission, thus utilizing the previously idle channel bandwidth of the CPU processor. Although the CPU channel bandwidth is lower, through reasonable data allocation, transmission can be completed in parallel with the NPU channel, significantly reducing overall energy consumption. The total energy consumption under the uncompressed configuration is reduced to 2678.9J, which is approximately 7.1% lower than that of a single NPU communication path.

[0103] In this application embodiment, when compression technology is further introduced into the dual-channel CPU path, energy consumption is closely related to the performance of the compressor. The compression ratio, compression / decompression time, and the resulting active CPU power consumption jointly determine the final energy efficiency result. Experiments show that the SZx compressor is the most efficient, with a moderate compression ratio (R=3.0) and short compression and decompression time (only 1.2s for full data), resulting in low overhead in the CPU communication path. This achieves dual optimization of communication time and energy consumption while maintaining high compression performance. When using the SZx compressor, the total energy consumption is further reduced to 2633.8J, becoming the optimal configuration under the experimental conditions.

[0104] Therefore, the communication energy efficiency optimization method of this application embodiment can utilize two cross-node channels in parallel in a distributed AI training scenario: the NPU communication path and the CPU communication path. With the goal of minimizing communication energy consumption, the tensors to be synchronized across nodes each time are split into two parallel transmission paths according to the transmission allocation ratio. One path transmits uncompressed data via the NPU communication path, while the other path compresses the data before transmission via the CPU communication path. Simultaneously, the receiving end decompresses the compressed data and participates in the reduction operation. This process can compensate for the bandwidth disadvantage of the slow channel (CPU communication path) with the compression ratio without increasing the critical path waiting time, while hiding the compression / decompression latency through pipelining and double buffering mechanisms. Its core lies in controlling the distribution of data between the two paths through an adjustable fixed allocation ratio, and adjusting the compression ratio R and the total compression / decompression time T of different compressors. omp Joint optimization was carried out.

[0105] By incorporating idle CPU network channels (such as RoCE and Ethernet RDMA) into the communication link, the total communication bandwidth can theoretically be increased, reducing the congestion risk of NPU dedicated channels (such as NVLink and HCCS). Simultaneously, dual-channel parallel transmission can shorten the single synchronization time, especially in large-scale All-Reduce operations, where data offloading can effectively alleviate bandwidth bottlenecks. Furthermore, introducing efficient compression and decompression processes into the CPU communication path reduces the amount of data transmitted through slow channels, thereby reducing total communication time and idle energy consumption. Activating CPU channels does not significantly increase overall power consumption. By combining measured bandwidth and power parameters to determine the optimal transmission allocation ratio and the optimal compressor selection, energy efficiency optimization for AI training communication is achieved, reducing device idle waiting energy consumption, which aligns with the trend of green computing.

[0106] The technical solution provided in this application obtains a transmission allocation ratio. This ratio is calculated by selecting a benchmark chipset from the intelligent computing cluster, acquiring the benchmark transmission parameters of the benchmark chipset, and calculating based on these parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During distributed data parallel model training, a tensor to be synchronized is acquired. Based on the transmission allocation ratio, the tensor is divided into NPU data blocks and CPU data blocks. The CPU data blocks are transmitted to a DRAM buffer. A selected compressor generates compressed data blocks corresponding to the CPU data blocks. These compressed data blocks are transmitted through the CPU communication path to the DRAM buffer of the corresponding receiving chip. The NPU data blocks are transmitted through the NPU communication path to the HBM memory of the receiving chip. This application embodiment determines the transmission allocation ratio through benchmark chipset calibration and combined with the actual hardware transmission capabilities. This ensures that the ratio of the tensor to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. For example, when using only the NPU path, a large amount of non-critical data will consume the high bandwidth resources of HBM; when using only the CPU path, the data transmission speed is limited by the interaction efficiency between DRAM and CPU, thus maximizing the performance advantages of both paths. Simultaneously, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On one hand, NPU data blocks are directly transmitted to the HBM memory of the receiving chip through the NPU communication path, adapting to the critical tensor data that needs high-frequency access in AI training, ensuring rapid synchronization of core data, and reducing waiting time during training iterations; on the other hand, CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving DRAM buffer through the CPU communication path, effectively alleviating the bandwidth pressure between DRAM and CPU. Furthermore, parallel transmission can avoid the efficiency loss caused by serial data transmission. Therefore, this application embodiment, through hardware-adapted tensor partitioning, dual-path parallel transmission, and data compression strategies, can solve the problems of resource waste and speed bottlenecks existing in a single transmission path, enabling the hardware resources of the intelligent computing cluster, such as NPU, CPU, HBM, and DRAM, to form efficient collaboration, ultimately significantly improving the data synchronization efficiency of AI training, accelerating the overall training process, and achieving an improvement in the AI ​​training efficiency of the intelligent computing cluster.

[0107] This application also provides a communication energy efficiency optimization device that can implement the above-mentioned communication energy efficiency optimization method, referring to... Figure 8 The device includes: Parameter acquisition module 810: used to acquire the transmission allocation ratio. The transmission allocation ratio is obtained by selecting a benchmark chipset from the intelligent computing cluster, acquiring the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip.

[0108] Data allocation module 820: used to acquire tensors to be synchronized during the distributed data parallel model training process, and divide the tensors to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio.

[0109] CPU data transmission module 830: Used to transmit CPU data blocks to DRAM buffer, generate compressed data blocks corresponding to CPU data blocks using the selected compressor, and transmit the compressed data blocks to the DRAM buffer of the corresponding receiving chip through the CPU communication path.

[0110] NPU data transmission module 840: Used to transmit NPU data blocks to the HBM memory of the receiving chip via the NPU communication path.

[0111] The specific implementation of the communication energy efficiency optimization device in this embodiment is basically the same as the specific implementation of the communication energy efficiency optimization method described above, and will not be repeated here.

[0112] This application also provides an electronic device, including: At least one memory; At least one processor; At least one program; The program is stored in a memory, and the processor executes the at least one program to implement the communication energy efficiency optimization method described above in this application. The electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), in-vehicle computers, etc.

[0113] Please see Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 901 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 902 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 902 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called and executed by the processor 901 using the communication energy efficiency optimization method of the embodiments of this application. The input / output interface 903 is used to implement information input and output; The communication interface 904 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 905 transmits information between various components of the device (e.g., processor 901, memory 902, input / output interface 903, and communication interface 904); The processor 901, memory 902, input / output interface 903, and communication interface 904 are connected to each other within the device via bus 905.

[0114] This application embodiment also provides a storage medium that stores a computer program, which, when executed by a processor, implements the above-described communication energy efficiency optimization method.

[0115] Memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0116] The communication energy efficiency optimization method, apparatus, device, and storage medium proposed in this application obtain a transmission allocation ratio. This ratio is calculated by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on these parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During distributed data parallel model training, a tensor to be synchronized is obtained. Based on the transmission allocation ratio, the tensor to be synchronized is divided into NPU data blocks and CPU data blocks. The CPU data blocks are transmitted to a DRAM buffer. A selected compressor generates compressed data blocks corresponding to the CPU data blocks. The compressed data blocks are transmitted through the CPU communication path to the DRAM buffer of the corresponding receiving chip. The NPU data blocks are transmitted through the NPU communication path to the HBM memory of the receiving chip. This application embodiment determines the transmission allocation ratio through benchmark chipset calibration and combined with the actual hardware transmission capabilities. This ensures that the ratio of the tensor to be synchronized into NPU data blocks and CPU data blocks accurately matches the transmission capabilities of the two types of hardware, avoiding the bottleneck effect caused by hardware performance mismatch in traditional single-path transmission. For example, when using only the NPU path, a large amount of non-critical data will consume the high bandwidth resources of HBM; when using only the CPU path, the data transmission speed is limited by the interaction efficiency between DRAM and CPU, thus maximizing the performance advantages of both paths. Simultaneously, the CPU communication path and the NPU communication path adopt a parallel transmission mode. On one hand, NPU data blocks are directly transmitted to the HBM memory of the receiving chip through the NPU communication path, adapting to the critical tensor data that needs high-frequency access in AI training, ensuring rapid synchronization of core data, and reducing waiting time during training iterations; on the other hand, CPU data blocks are first transmitted to the DRAM buffer, compressed by the compressor, and then transmitted to the receiving DRAM buffer through the CPU communication path, effectively alleviating the bandwidth pressure between DRAM and CPU. Furthermore, parallel transmission can avoid the efficiency loss caused by serial data transmission. Therefore, this application embodiment, through hardware-adapted tensor partitioning, dual-path parallel transmission, and data compression strategies, can solve the problems of resource waste and speed bottlenecks existing in a single transmission path, enabling the hardware resources of the intelligent computing cluster, such as NPU, CPU, HBM, and DRAM, to form efficient collaboration, ultimately significantly improving the data synchronization efficiency of AI training, accelerating the overall training process, and achieving an improvement in the AI ​​training efficiency of the intelligent computing cluster.

[0117] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0118] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0121] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0122] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0123] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0124] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0125] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0126] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0127] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A communication energy efficiency optimization method, characterized in that, Applied to a transmitting chip, the method includes: The transmission allocation ratio is obtained by selecting a benchmark chipset from the intelligent computing cluster, obtaining the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. During the distributed data parallel model training process, the tensor to be synchronized is obtained, and the tensor to be synchronized is divided into NPU data blocks and CPU data blocks based on the transmission allocation ratio. The CPU data block is transmitted to the DRAM buffer, a compressed data block corresponding to the CPU data block is generated using the selected compressor, and the compressed data block is transmitted to the DRAM buffer of the corresponding receiving chip through the CPU communication path. The NPU data block is transmitted to the HBM memory of the receiving chip via the NPU communication path.

2. The communication energy efficiency optimization method according to claim 1, characterized in that, The transmission allocation ratio is calculated based on the reference transmission parameters, including: The reference transmission parameters are obtained, including at least the CPU bandwidth of the CPU communication path, the NPU bandwidth of the NPU communication path, and the copy bandwidth between the HBM memory and the DRAM buffer in the reference transmitting chip or the reference receiving chip. The transmission allocation ratio is calculated based at least on the CPU bandwidth, the NPU bandwidth, and the copy bandwidth.

3. The communication energy efficiency optimization method according to claim 2, characterized in that, The calculation of the transmission allocation ratio based at least on the CPU bandwidth, the NPU bandwidth, and the copy bandwidth includes: Set a scaling parameter, and obtain the baseline NPU data and baseline CPU data based on the scaling parameter and the data size of the baseline data packet; The total communication duration is calculated based on the baseline NPU data, the baseline CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth. The total energy consumption is constructed based on the total communication duration. The target optimization function is obtained by minimizing the total energy consumption. The optimal solution of the ratio parameter is obtained by solving the target optimization function and serving as the transmission allocation ratio.

4. The communication energy efficiency optimization method according to claim 3, characterized in that, The calculation of the total communication duration based on the baseline NPU data, the baseline CPU data, the CPU bandwidth, the NPU bandwidth, and the copy bandwidth includes: The NPU latency is calculated based on the baseline NPU data and the NPU bandwidth. The external transmission latency is calculated based on the baseline CPU data, the compression ratio of the compressor, and the CPU bandwidth. The internal transmission latency is calculated based on the ratio parameter, the data size, and the copy bandwidth. The CPU latency is calculated based on the external transmission latency and the internal transmission latency. Obtain the compression duration of the baseline data packet, and obtain the compression delay based on the ratio parameter and the compression duration; The total CPU latency is calculated based on the CPU latency and the compression latency, and the maximum value of the total CPU latency and the NPU latency is set as the total communication duration.

5. The communication energy efficiency optimization method according to claim 4, characterized in that, The process of constructing the total energy consumption based on the total communication duration includes: Obtain compression power parameters, CPU power parameters of the CPU communication path, NPU power parameters of the NPU communication path, CPU idle power, and NPU idle power; Calculate the difference between the total communication time and the CPU latency and the NPU latency to obtain the CPU idle time and NPU idle time; Compression energy consumption is obtained by multiplying the compression latency and the compression power parameter; CPU transmission energy consumption is obtained by multiplying the CPU latency and the CPU power parameter; CPU idle energy consumption is obtained by multiplying the CPU idle power and the CPU idle time; NPU transmission energy consumption is obtained by multiplying the NPU latency and the NPU power parameter; and NPU idle energy consumption is obtained by multiplying the NPU idle power and the NPU idle time. The total energy consumption is obtained by summing the compression energy consumption, the CPU transmission energy consumption, the CPU idle energy consumption, the NPU transmission energy consumption, and the NPU idle energy consumption.

6. The communication energy efficiency optimization method according to claim 5, characterized in that, Solving the objective optimization function to obtain the optimal solution for the proportional parameter as the transmission allocation ratio includes: The objective optimization function is solved using a convex function optimization method. The solution corresponding to the point where the sum of the CPU latency and the compressed latency is consistent with the NPU latency is set as the abrupt change point of the proportional parameter. Calculate the energy consumption values ​​corresponding to the two endpoint values ​​and the abrupt change point of the ratio parameter, and select the ratio parameter corresponding to the smallest energy consumption value as the transmission allocation ratio.

7. A communication energy efficiency optimization method, characterized in that, Applied to a receiver chip, the method includes: Acquire compressed data blocks transmitted from the sending chip to the DRAM buffer via the CPU communication path; Obtain NPU data blocks transmitted from the transmitting chip to HBM memory via the NPU communication path, wherein the compressed data blocks and the NPU data blocks are obtained by the transmitting chip according to any one of claims 1 to 6; The compressed data block is decompressed to obtain a decompressed data block, which is then transferred to the HBM memory. The decompressed data block and the NPU data block are then merged to obtain the target data corresponding to the tensor to be synchronized.

8. A communication energy efficiency optimization device, characterized in that, The device, applied to a transmitting chip, includes: Parameter acquisition module: used to acquire the transmission allocation ratio, which is obtained by selecting a benchmark chipset from the intelligent computing cluster, acquiring the benchmark transmission parameters of the benchmark chipset, and calculating based on the benchmark transmission parameters. The benchmark chipset includes a benchmark transmitting chip and a benchmark receiving chip. Data allocation module: used to obtain tensors to be synchronized during the distributed data parallel model training process, and divide the tensors to be synchronized into NPU data blocks and CPU data blocks based on the transmission allocation ratio; CPU data transmission module: used to transmit the CPU data block to the DRAM buffer, generate a compressed data block corresponding to the CPU data block using the selected compressor, and transmit the compressed data block to the DRAM buffer of the corresponding receiving chip through the CPU communication path; NPU data transmission module: used to transmit the NPU data block to the HBM memory of the receiving chip through the NPU communication path.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the communication energy efficiency optimization method according to any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the communication energy efficiency optimization method according to any one of claims 1 to 7.

Images