Communication method and apparatus, electronic device, computer-readable storage medium, and computer program product

Abstract

The application provides a communication method, device, electronic equipment, computer readable storage medium and computer program product; the method comprises the following steps: distributing to-be-processed data to a plurality of target computing cards, each target computing card comprising a plurality of computing cores and at least one communication core, each computing core being allocated a data slice in the to-be-processed data; for each target computing card, determining a target core corresponding to the target computing card from the at least one communication core, and constructing a first communication network based on the computing cores and the target core; controlling each computing core to perform reduction processing on the data slices through the first communication network to obtain a first processing result of the target computing card; constructing a second communication network based on the target cores, and controlling each target core to perform global reduction processing on the first processing results of the target computing cards through the second communication network to obtain a second processing result. Through the application, the collection communication efficiency between computing cards can be improved.

Description

Technical Field

[0001] This application relates to the field of semiconductor technology, and more particularly to a communication method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid expansion of artificial intelligence models and high-performance computing applications, the demand for graphics processing units (GPUs) computing power is constantly increasing. To address this issue, multi-chip module (MCM) technology is employed, integrating multiple smaller computing chips into a single computing card using advanced packaging technologies such as silicon interposers. This architecture presents new challenges for aggregate communication (such as global reduction) in large-scale distributed training.

[0003] In related technologies, the use of inter-card links between various computing cards for aggregated communication results in insufficient utilization of high-speed intra-card links, leading to low communication efficiency between computing cards. Summary of the Invention

[0004] This application provides a communication method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve the efficiency of aggregated communication between computing cards.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides a communication method, the method comprising:

[0007] In response to receiving data to be processed, the data to be processed is allocated to multiple target computing cards. Each target computing card includes multiple computing chips and at least one communication chip. Each computing chip is allocated a data fragment of the data to be processed. The communication chip has inter-card communication functionality.

[0008] For each target computing card, a target chip corresponding to the target computing card is determined from the at least one communication chip, and a first communication network is constructed based on each computing chip and the target chip;

[0009] Each computing chip is controlled to perform reduction processing on each data fragment through the first communication network to obtain the first processing result of the target computing card, and the first processing result is sent to the target chip corresponding to the target computing card through the first communication network.

[0010] A second communication network is constructed based on each of the target chips, and each target chip is controlled to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain a second processing result;

[0011] For each target chip, the target chip is controlled to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

[0012] This application provides a communication device, including:

[0013] A data allocation module is used to allocate the data to be processed to multiple target computing cards in response to receiving data to be processed. Each target computing card includes multiple computing cores and at least one communication core. Each computing core is allocated a data fragment of the data to be processed, and the communication core has inter-card communication function.

[0014] A chip determination module is used to determine, for each target computing card, the target chip corresponding to the target computing card from the at least one communication chip, and to construct a first communication network based on each computing chip and the target chip;

[0015] The first processing module is used to control each of the computing chips to perform reduction processing on each of the data fragments through the first communication network to obtain the first processing result of the target computing card, and send the first processing result to the target chip corresponding to the target computing card through the first communication network.

[0016] The second processing module is used to construct a second communication network based on each of the target chips, and control each of the target chips to perform global reduction processing on the first processing result of each of the target computing cards through the second communication network to obtain a second processing result;

[0017] The data broadcasting module is used to control each target chip to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

[0018] This application provides an electronic device, the electronic device comprising:

[0019] Memory is used to store executable instructions or computer programs.

[0020] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the communication method provided in the embodiments of this application.

[0021] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which are executed by a processor to implement the communication method provided in this application.

[0022] This application provides a computer program product, including computer-executable instructions or a computer program, which, when executed by a processor, implements the communication method provided in this application.

[0023] The embodiments of this application have the following beneficial effects:

[0024] In this embodiment, the data to be processed is distributed to multiple target computing cards. Each target computing card includes multiple computing cores and at least one communication core. Each computing core is allocated a data fragment from the data to be processed. Next, for each target computing card, a target core corresponding to the target computing card is determined from the at least one communication core. A first communication network is constructed based on each computing core and the target core. Each computing core is controlled to perform reduction processing on each data fragment through the first communication network to obtain a first processing result from the target computing card. This decomposes the data aggregation task between computing cards into local aggregation subtasks executed independently within each computing card, achieving localized processing of communication tasks and significantly reducing the amount of data in cross-card communication. Then, a second communication network is constructed based on each target core, and each target core is controlled to perform global reduction processing on the first processing result from each target computing card through the second communication network to obtain a second processing result. Finally, each target core is controlled to broadcast the second processing result to each computing core in the target computing card corresponding to the target core through the first communication network. In this way, the data aggregation stage in the aggregated communication task can be limited to be completed in the high-speed intracard link, making full use of the high bandwidth resources of the intracard under the multi-core chip packaging architecture, so that the low-speed inter-card link only carries a small amount of aggregated data, thereby improving the efficiency of aggregated communication between computing cards. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the network architecture of the communication method provided in the embodiments of this application;

[0026] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0027] Figure 3 This is a first flowchart illustrating the communication method provided in an embodiment of this application;

[0028] Figure 4 This is a second flowchart illustrating the communication method provided in an embodiment of this application;

[0029] Figure 5This is a third flowchart illustrating the communication method provided in an embodiment of this application;

[0030] Figure 6 This is a fourth flowchart illustrating the communication method provided in the embodiments of this application;

[0031] Figure 7 This is a schematic diagram of the first structure of inter-card communication provided in an embodiment of this application;

[0032] Figure 8 This is a schematic diagram of the first link bandwidth provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram of the second structure of inter-card communication provided in an embodiment of this application;

[0034] Figure 10 This is a schematic diagram of the second link bandwidth provided in an embodiment of this application;

[0035] Figure 11 This is a schematic diagram of the third structure for inter-card communication provided in an embodiment of this application;

[0036] Figure 12 This is a schematic diagram of the fourth structure for inter-card communication provided in an embodiment of this application;

[0037] Figure 13 This is a fifth structural diagram of inter-card communication provided in the embodiments of this application.

[0038] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0041] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0042] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0043] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0044] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0045] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0046] 1) Multi-Chip Module (MCM): This is a semiconductor packaging technology that decomposes the system function into multiple independent chips and integrates the chips onto a substrate or interposer through packaging technology to package them into a system-on-a-chip.

[0047] 2) Collective Communication: This is a many-to-many or one-to-many communication mode in the field of parallel computing. It refers to a global data exchange operation that is jointly participated in and completed by all members (or a subset) within a process group.

[0048] 3) Reduction: This involves performing feature extraction or aggregation calculations on the data slices held by each computing chip, such as summation, averaging, finding the maximum or minimum value, etc.

[0049] 4) Chip: A chip is an integrated circuit die that is pre-manufactured and tested independently and has a specific function. The computing chip is responsible for performing data processing and numerical calculation tasks, while the communication chip is a chip with inter-card communication network interface (I / O) and data routing interaction functions.

[0050] 5) Computing Card: This refers to a computing acceleration card that includes a multi-core packaged architecture, including but not limited to graphics processing unit (GPU), tensor processor card, or neural network processor card.

[0051] This application provides a communication method, apparatus, electronic device, computer-readable storage medium, and computer program product that can improve the efficiency of aggregated communication between computing cards.

[0052] The following describes exemplary applications of the electronic device provided in the embodiments of this application. The electronic device provided in the embodiments of this application can be implemented as a server. Exemplary applications when the device is implemented as a server will be described below.

[0053] See Figure 1 , Figure 1 This is a schematic diagram of the network architecture of the communication method provided in the embodiments of this application, exemplified. Figure 1 The system involves server 200, network 300, and terminal 400. Terminal 400 connects to server 200 through network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0054] During communication, terminal 400 sends the data to be processed to server 200. Upon receiving the data, server 200 distributes it to multiple target computing cards. Each target computing card includes multiple computing cores and at least one communication core. Each computing core is allocated a data fragment from the data to be processed, and the communication core has inter-card communication functionality. For each target computing card, a target core corresponding to the target computing card is determined from at least one communication core, and a first communication network is constructed based on each computing core and the target core. Each computing core is controlled to perform reduction processing on each data fragment through the first communication network to obtain a first processing result for the target computing card, and then sends the first processing result to the target core corresponding to the target computing card through the first communication network. A second communication network is constructed based on each target core, and each target core is controlled to perform global reduction processing on the first processing results of each target computing card through the second communication network to obtain a second processing result. For each target core, the second processing result is broadcast to each computing core in the target computing card corresponding to the target core through the first communication network. For example, the above communication process can be used to improve the communication efficiency between computing cards in scenarios such as distributed training of large-scale artificial intelligence models, high-performance computing, large-scale data analysis and graph computing.

[0055] See Figure 2 , Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a server. Figure 2 The illustrated electronic device includes at least one processor 410, a memory 450, and at least one network interface 420. The various components of the electronic device are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 2 The general labeled all buses as Bus System 440.

[0056] Processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0057] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0058] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0059] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0060] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, and driver layer, for implementing various basic business functions and handling hardware-based tasks.

[0061] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including Bluetooth, WiFi, and Universal Serial Bus (USB).

[0062] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A communication device 455 stored in memory 450 is shown. It may be software in the form of programs and plug-ins, including the following software modules: data allocation module 4551, core determination module 4552, first processing module 4553, second processing module 4554, and data broadcasting module 4555. These modules are logical and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0063] The communication method provided in this application will be described in conjunction with exemplary applications and implementations of the server device provided in the embodiments of this application.

[0064] The following describes the communication method provided in the embodiments of this application. For example, to facilitate understanding of the communication method provided in the embodiments of this application, the example described is the collective communication between computing cards.

[0065] As mentioned above, the electronic device implementing the communication method of the embodiments of this application can be a server. The communication method provided in the embodiments of this application will now be described using an electronic device as a server as an example. See also Figure 3 , Figure 3 This is a first flowchart illustrating the communication method provided in this application embodiment, which will be combined with... Figure 3 The steps shown are explained.

[0066] In step 301, in response to receiving the data to be processed, the data to be processed is distributed to multiple target computing cards.

[0067] Here, the data to be processed is a set of data that requires collective communication operations in a multi-node collaborative computing system, such as intermediate computational data like gradient tensors, weight parameters, or activation values ​​during neural network computation. The target computing card is a multi-core packaged graphics processing unit (GPU), with each target computing card comprising multiple computing cores. , ...} and at least one communication chip, which is a chip with inter-card communication network interface (I / O) and data routing interaction functions, and each computing chip is allocated a data fragment of the data to be processed.

[0068] When data to be processed is received, it is divided into multiple data chunks based on a preset data partitioning logic. These chunks can be randomly assigned to multiple target computing cards, or they can be evenly distributed among multiple target computing cards. Finally, the computing die within the target computing card is responsible for processing each data chunk.

[0069] In step 302, for each target computing card, the target chip corresponding to the target computing card is determined from at least one communication chip, and a first communication network is constructed based on each computing chip and the target chip.

[0070] Here, for each target computing card, based on the communication performance indicators between the communication core and the communication cores in other target computing cards, the target core corresponding to the target computing card is determined from at least one communication core. The target core represents the agent node corresponding to a target computing card.

[0071] In some embodiments, see Figure 4 , Figure 4 This is a second flowchart illustrating the communication method provided in an embodiment of this application. Figure 3 In step 302 shown, "determining the target chip corresponding to the target computing card from at least one communication chip" can be achieved through... Figure 4 Steps 3021 to 3023 are implemented, and will be explained in detail below.

[0072] In step 3021, for each communication chip, the communication performance indicators between the communication chip and the communication chips in other target computing cards are obtained.

[0073] Here, for each communication chip in a target computing card, a test pair is constructed between the communication chip and communication chips on other target computing cards. For example, there are target computing card A and target computing card B, where target computing card A contains communication chips. The target computing card B contains a communication chip. and The constructed test pair is represented as ( , )and( , ).

[0074] Communication performance metrics include communication latency and bandwidth. The test pair includes a source communication chip and a target communication chip. One or more small data packets are generated in the memory of the source communication chip and sent to the target communication chip through its physical communication interface. Upon receiving the data packet, the target communication chip immediately sends it back completely to the source communication chip. A high-precision timer is used on the source communication chip to record the total time from sending the data packet to receiving the returned data packet. The average of the total time is taken to obtain the communication latency between the source communication chip and the communication chips in other target computing cards. Continuing with the above example, the test pair ( , Zhongyuan Communication's core chip is The target communication chip is Source communication chip The total time from sending a data packet to receiving a return data packet is 10 seconds. Taking the average of the total time yields the communication core. Communication chips in other target computing cards The communication delay between them is 5 seconds.

[0075] One or more large data blocks are generated in the memory of the source communication chip, and the source communication chip continuously transmits these data blocks to the target communication chip. The timestamps for the start and end of data block transmission are recorded on the source communication chip side to obtain the total transmission time. The ratio of the total number of data blocks to the total transmission time is determined as the communication bandwidth. Continuing with the above example, the test is performed on (…). , Zhongyuan Communication's core chip is The target communication chip is Source communication chip To target communication chip Multiple data blocks were transmitted continuously over a total time of 20 seconds, with a total data block size of 100MB, resulting in a communication core. Communication chips in other target computing cards The communication bandwidth between them is 5MB / s.

[0076] For each test pair, the communication delay and communication bandwidth can be associated with the test pair to obtain a communication performance matrix. The element M(i,j) in the matrix stores the communication performance indicators (e.g., communication delay or communication bandwidth) from communication core i to communication core j.

[0077] Continue to refer to Figure 4 In step 3022, the performance evaluation data of the communication core is determined based on the communication performance indicators, and the communication cores are sorted in order from best to worst according to the performance evaluation data to obtain the communication core sequence.

[0078] Here, for each communication chip in a target computing card, multiple communication performance indicators related to that communication chip are extracted from the communication performance matrix. These indicators include the communication latency and bandwidth between the communication chip and communication chips in other target computing cards. Based on these indicators, the average communication latency and average bandwidth between the communication chip and other communication chips in the target computing cards are calculated. Using a pre-defined weighted scoring function, the average communication latency and average bandwidth are weighted and summed to obtain a numerical value characterizing communication capability—the performance evaluation data of the communication chip. A higher performance evaluation data indicates better performance. The communication chips are then arranged in descending order of performance evaluation data, with the chip with the best performance evaluation data at the top and the chip with the worst performance evaluation data at the bottom, resulting in a communication chip sequence.

[0079] For example, a target computing card contains four communication chips: communication chip 1, communication chip 2, communication chip 3, and communication chip 4. The performance evaluation data for communication chip 1 is Score1, for communication chip 2 it is Score2, for communication chip 3 it is Score3, and for communication chip 4 it is Score4. Score2 Score3 Score4. The performance evaluation data of each communication chip is ranked from best to worst. Communication chip 1 (with a performance evaluation score of Score1) is placed first, communication chip 2 (with a performance evaluation score of Score2) is placed second, communication chip 3 (with a performance evaluation score of Score3) is placed third, and communication chip 4 (with a performance evaluation score of Score4) is placed last, resulting in the communication chip sequence "Communication chip 1 → Communication chip 2 → Communication chip 3 → Communication chip 4".

[0080] Continue to refer to Figure 4In step 3023, the first K communication chips in the communication chip sequence are determined as the target chips corresponding to the target computing card.

[0081] Here, K is a positive integer. When K is 1, the first communication chip in the communication chip sequence is identified as a target chip corresponding to the target computing card, and one target computing card corresponds to one target chip. When K is greater than 1, the first K communication chips in the communication chip sequence are identified as K target chips corresponding to the target computing card, and one target computing card corresponds to multiple target chips.

[0082] Continuing with the example above, the communication chip sequence is "Communication Chip 1 → Communication Chip 2 → Communication Chip 3 → Communication Chip 4". When K is 1, the first communication chip 1 in the communication chip sequence is identified as one target chip corresponding to the target computing card. When K is 2, the first communication chip 1 to the second communication chip 2 in the communication chip sequence are identified as two target chips corresponding to the target computing card.

[0083] In this embodiment, for each communication chip, the communication performance indicators between the communication chip and the communication chips in other target computing cards are first obtained. Based on the communication performance indicators, the performance evaluation data of the communication chip is determined. Then, the communication chips are sorted according to the order of the performance evaluation data from best to worst to obtain a communication chip sequence. The first K communication chips in the communication chip sequence are determined as the target chips corresponding to the target computing cards. This realizes the sorting of the performance evaluation data of each communication chip and the selection of the first K communication chips with the best performance as target chips (proxy nodes), establishing a dynamic and adaptive proxy node election mechanism. This mechanism ensures that the target chips used to perform inter-card aggregate communication exchange data through the physical link with the best performance, thereby significantly improving the overall performance of cross-card aggregate communication.

[0084] In some embodiments, constructing a first communication network based on each computing chip and a target chip can be achieved through the following steps: obtaining the physical interconnect structure between each computing chip and the target chip; determining the link performance indicators between each computing chip and the target chip based on the physical interconnect structure; determining the target logical topology for connecting each computing chip and the target chip based on the link performance indicators; and generating the first communication network based on the target logical topology.

[0085] Here, the physical interconnect structure refers to the inherent, physical connections between the computing chips and the target chip within the target computing card. The physical interconnect structure between the computing chips and the target chip is obtained by querying a predefined hardware topology data structure (e.g., a static configuration table). The physical interconnect structure can be represented as a graph, where nodes represent individual computing chips and the target chip, and edges represent the physical links connecting these chips.

[0086] For each computing core, the physical path from the computing core to the target core is resolved from the physical interconnect structure. A physical path consists of a series of core nodes and intermediate routing nodes. For example, the physical path from computing core A to target core B is represented as A->Router1->Router2->B.

[0087] The link performance metrics between the computational core and the target core include hop count and theoretical bandwidth. Hop count refers to the number of routing nodes that data must traverse from the computational core to the target core. A lower hop count indicates lower transmission latency. Theoretical bandwidth refers to the bandwidth of the physical path with the lowest bandwidth, i.e., the bottleneck bandwidth of the path. The hop count of the routing nodes along the physical path from the computational core to the target core is obtained. The theoretical bandwidth of each physical link segment on the physical path is obtained by querying the hardware specifications, and the minimum of these is taken as the theoretical bandwidth of the entire physical path.

[0088] The target logical topology is a data structure that represents the logical communication relationships between each computational particle and the target particle. The target logical topology can be a tree topology, where the target particle is the root node and each computational particle is a leaf node or intermediate node. Based on the link performance metrics between each computational particle and the target particle, the computational particles are hierarchically divided. For example, all computational particles with a hop count of 1 are placed in the first layer, computational particles with a hop count of 2 are placed in the second layer, and so on. The computational particles in the first layer (with the best performance) are connected to the target particle (the root node), making them child nodes of the target particle. The computational particles in the second layer are connected to the computational particles in the first layer that are physically closest to them or have the best link performance, making them child nodes of the nodes in the first layer, until all computational particles are included in the tree topology.

[0089] Based on the target logical topology, routing tables are configured and communication resources are allocated for each computing chip and the target chip to obtain an in-card logical communication network connecting each computing chip and the target chip, which is also the first communication network.

[0090] In this embodiment, based on the physical interconnection structure between each computing chip and the target chip, the link performance index between each computing chip and the target chip is determined. Based on the link performance index, the target logical topology for connecting each computing chip and the target chip is determined, and a first communication network is generated based on the target logical topology. This achieves the goal of dynamically constructing the optimal data transmission path for in-card communication tasks by accurately quantifying the physical link performance within the card, thereby improving the efficiency and speed of in-card data exchange and thus improving in-card communication efficiency.

[0091] Continue to refer to Figure 3 The following will be an explanation following step 302.

[0092] In step 303, each computing chip is controlled to perform reduction processing on each data fragment through the first communication network to obtain the first processing result of the target computing card, and the first processing result is sent to the target chip corresponding to the target computing card through the first communication network.

[0093] Here, for each computing chip, the control unit performs reduction processing on the data fragments held by each computing chip through the first communication network, obtaining a reduction result corresponding to a computing chip. Reduction involves feature extraction or aggregation calculation on the data fragments held by each computing chip, such as summation, averaging, finding the maximum or minimum value, etc. The reduction results corresponding to each computing chip are combined to obtain the first processing result of the target computing card.

[0094] In some embodiments, a data fragment includes multiple data blocks, see [link to relevant documentation]. Figure 5 , Figure 5 This is a third flowchart illustrating the communication method provided in the embodiments of this application. Figure 3 In step 303 shown, "controlling each computing chip to perform reduction processing on each data fragment through the first communication network to obtain the first processing result of the target computing card" can be achieved through... Figure 5 Steps 3031 to 3033 are implemented, and will be explained in detail below.

[0095] In step 3031, for each computing chip, a target computing chip adjacent to the computing chip is determined based on the topology of the first communication network.

[0096] Here, the topology of the first communication network represents the logical connections between the various computing cores. Parsing the topology of the first communication network yields the target computing cores adjacent to each computing core. For example, refer to [reference needed]. Figure 7 , Figure 7This is a schematic diagram of the first structure of inter-card communication provided in an embodiment of this application. GPU0 card 701 is a target computing card. The topology of the first communication network in this card is a ring topology, represented as CD0<->CD1<->CD2<->CD0, where CD0, CD1, and CD2 represent each computing chip. Target computing chips adjacent to computing chip CD0 are identified as CD1 and CD2, target computing chips adjacent to computing chip CD1 are identified as CD2 and CD0, and target computing chips adjacent to computing chip CD2 are identified as CD1 and CD0.

[0097] In step 3032, based on the multiple data blocks corresponding to the computational kernel and the multiple data blocks corresponding to the target computational kernel, the target data block to be processed by the computational kernel is determined, and the computational kernel is controlled to perform aggregation operation on the target data block to obtain the first aggregation result of the computational kernel.

[0098] Here, when the first communication network contains N computing chips, the data fragments corresponding to the computing chips are divided into N data blocks of equal size to obtain multiple data blocks corresponding to the computing chips, and the data fragments corresponding to the target computing chips are divided into N data blocks of equal size to obtain multiple data blocks corresponding to the target computing chips.

[0099] This stage determines the first aggregation result of the computational kernel through N-1 iterations. When the topology of the first communication network is a ring topology, in the k-th iteration (k from 0 to N-2), the data block that computational kernel CD_i needs to send to the next adjacent target computational kernel is determined. This data block is the data block with index (ik) in the output buffer of computational kernel CD_i. The computational kernel CD_i is controlled to send this data block to the next adjacent target computational kernel through the first communication network. At the same time, computational kernel CD_i receives a data block from its previous adjacent target computational kernel and stores the data block in its input buffer. The data block with index (ik-1) in the output buffer of computational kernel CD_i and the received data block in the input buffer are determined as the target data block to be processed by computational kernel CD_i.

[0100] The computational unit controlling the computational kernel CD_i performs aggregation operations on the target data block to obtain the results. Aggregation operations include normalizing or reducing the characteristics of the data, such as summation, averaging, and obtaining the maximum / minimum value. The results are written back to the output buffer of computational kernel CD_i at index (ik-1) to overwrite the original data block. After N-1 iterations, the data block at index (i+1) in the output buffer of computational kernel CD_i represents the complete aggregation result of the data block at the corresponding index position on each computational kernel, which is also the first aggregation result of computational kernel CD_i.

[0101] For example, the first communication network contains three computing cores (N=3), with a ring topology: CD0->CD1->CD2->CD0. The first aggregation result for each computing core is determined through two iterations. Data fragments on each computing core are divided into three data blocks: {A0, B0, C0} for computing core CD0, {A1, B1, C1} for computing core CD1, and {A2, B2, C2} for computing core CD2. In the first iteration (k=0), the data block that computing core CD0 needs to send to the next adjacent target computing core CD1 is determined. This data block is data block A0 with index 0 in the output buffer of computing core CD0. Computing core CD0 is then controlled to send data block A0 to computing core CD1 through the first communication network. Simultaneously, computing core CD0 receives data block C2 from its adjacent previous target computing core CD2. Similarly, control computing kernel CD1 sends data block B1 to computing kernel CD2 via the first communication network, and computing kernel CD1 receives data block A0 from computing kernel CD0. Control computing kernel CD2 sends data block C2 to computing kernel CD0 via the first communication network, and computing kernel CD2 receives data block B1 from computing kernel CD1.

[0102] The target data blocks to be processed by computational kernel CD0 are determined to be data block C0 at index -1 in the output buffer and the received data block C2 in the input buffer. Computational kernel CD0 performs an aggregation operation on data blocks C0 and C2, setting the aggregation operation to summation. The result is C2 + C0, and the result is written back to the position at index -1 in the output buffer of computational kernel CD0 to overwrite the original data block C0. At this time, the multiple data blocks corresponding to computational kernel CD0 are updated to {A0, B0, C2 + C0}. Similarly, the multiple data blocks corresponding to computational kernel CD1 are updated to {A0 + A1, B1, C1}, and the multiple data blocks corresponding to computational kernel CD2 are updated to {A2, B1 + B2, C2}.

[0103] In the second iteration (k=1), the control computational kernel CD0 sends the data block C2+C0 with index -1 in its output buffer to computational kernel CD1. Simultaneously, computational kernel CD0 receives data block B1+B2 from computational kernel CD2. Similarly, the control computational kernel CD1 sends data block A0+A1 to computational kernel CD2, and computational kernel CD1 receives data block C2+C0 from computational kernel CD0. The control computational kernel CD2 sends data block B1+B2 to computational kernel CD0, and computational kernel CD2 receives data block A0+A1 from computational kernel CD1.

[0104] The control computational core CD0 performs aggregation operations on data blocks B0 and B1+B2 at index -2 in the output buffer, obtaining the result B0+B1+B2. This result is then written back to index -2 in the output buffer of computational core CD0 to overwrite the original data block B0. At this point, the multiple data blocks corresponding to computational core CD0 are updated to {A0, B0+B1+B2, C2+C0}. Data block B0+B1+B2 is the data block at index 2 in its output buffer, representing the complete aggregation result of the data blocks at index 2 on each computational core. In other words, the first aggregation result of computational core CD0 is B0+B1+B2. Similarly, the first aggregation result of computational core CD1 is C0+C1+C2, and the first aggregation result of computational core CD2 is A0+A1+A2.

[0105] Continue to refer to Figure 5 In step 3033, multiple first aggregation results are combined to obtain a first processing result.

[0106] Here, the first aggregation results of each computational core are logically combined at their corresponding index positions to obtain the first processing result. Continuing the example above, the first aggregation result of computational core CD0 is B0+B1+B2, the first aggregation result of computational core CD1 is C0+C1+C2, and the first aggregation result of computational core CD2 is A0+A1+A2. Combining multiple first aggregation results yields the first processing result {A0+A1+A2, B0+B1+B2, C0+C1+C2}.

[0107] In this embodiment, for each computing chip, firstly, based on the topology of the first communication network, target computing chips adjacent to the computing chip are determined. Then, based on multiple data blocks corresponding to the computing chip and multiple data blocks corresponding to the target computing chip, the target data block to be processed by the computing chip is determined, and the computing chip is controlled to perform aggregation operation on the target data block to obtain the first aggregation result of the computing chip. Finally, multiple first aggregation results are combined to obtain the first processing result of the target computing card. This establishes an efficient distributed intra-card data reduction mechanism, which completes the local reduction of all allocated data in the computing card without occupying inter-card bandwidth resources, maximizing the utilization of the high bandwidth resources in the card, and laying a solid foundation for subsequent low-load aggregated communication only between cards.

[0108] In some embodiments, sending the first processing result to the target chip corresponding to the target computing card through the first communication network can be achieved through the following steps: dividing the first processing result into K sub-processing results; sending each sub-processing result to each target chip, wherein the target chip corresponds one-to-one with the sub-processing result.

[0109] Here, as mentioned earlier, the number of target cores is K. When K is 1, meaning there is only one target core, the first processing result is directly determined as the sub-processing result, and the sub-processing result is sent to that target core. When K is greater than 1, meaning there are multiple target cores, the first processing result is divided into multiple sub-processing results, and each sub-processing result is sent to each target core.

[0110] Continuing with the example above, the first processing result is {A0+A1+A2, B0+B1+B2, C0+C1+C2}. When K is 1, meaning there is one target core, the first processing result {A0+A1+A2, B0+B1+B2, C0+C1+C2} is directly identified as a sub-processing result, and the sub-processing result {A0+A1+A2, B0+B1+B2, C0+C1+C2} is sent to that target core. When K is 3, meaning there are 3 target cores, the first processing result {A0+A1+A2, B0+B1+B2, C0+C1+C2} is divided into multiple sub-processing results A0+A1+A2, B0+B1+B2, and C0+C1+C2, and each sub-processing result is sent to each target core.

[0111] In this embodiment, the first processing result of the target computing card is divided into K sub-processing results, and each sub-processing result is sent to each target chip. The target chip corresponds one-to-one with the sub-processing result, thereby distributing the communication load evenly across multiple high-performance physical links and achieving effective bandwidth expansion. This makes each target chip a communication participant holding a portion of global data, providing a multi-path communication architecture for subsequent efficient cross-card aggregation communication.

[0112] Continue to refer to Figure 3 The following explanation will continue from step 303 above.

[0113] In step 304, a second communication network is constructed based on each target chip, and each target chip is controlled to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain the second processing result.

[0114] Here, a second communication network is constructed based on the target chips corresponding to each target computing card. This second communication network is an inter-card logical communication network. For each target chip corresponding to a target computing card, the target chip is controlled to perform global reduction processing on the first processing results of each target computing card through the second communication network, obtaining the global reduction result, which is the second processing result. Global reduction involves feature extraction or aggregation calculation on the first processing results of each target computing card, such as summation, averaging, finding the maximum or minimum value, etc.

[0115] In some embodiments, constructing a second communication network based on each target chip can be achieved through the following steps: obtaining the inter-card link bandwidth between each target chip; determining the global interconnection topology connecting each target chip based on the inter-card link bandwidth; and generating the second communication network based on the global interconnection topology.

[0116] Here, for each target compute card and its corresponding target chip, the target chip is controlled to send a probe data packet of a preset size to the target chips in other target compute cards. By measuring the time consumed in successfully transmitting the probe data packet, the inter-card link bandwidth between them is calculated. This operation is performed on the inter-card links between each target chip. This process can be repeated multiple times and the average value is taken to eliminate the impact of instantaneous network jitter, thereby obtaining the inter-card link bandwidth between each target chip.

[0117] Based on the inter-card link bandwidth between each target chip, the target inter-card link with the largest bandwidth is determined from the inter-card links between the target chips. Then, based on the preset logical topology type required for aggregated communication operations (e.g., ring, tree, fully connected mesh, etc.), multiple target inter-card links are combined into a global interconnection topology. Based on this global interconnection topology, routing rules are configured and communication endpoints are bound for each target chip, resulting in a logical inter-card communication network connecting all target chips, i.e., the second communication network.

[0118] For example, please continue to refer to [the example]. Figure 7 GPU0 card 701 and GPU1 card 702 are target computing cards, and the computing chips CD0, CD1, and CD2 in each target computing card correspond to one target chip (not shown in the figure). The links between target cards are GPU0-CD0<->GPU1-CD0, GPU0-CD1<->GPU1-CD1, and GPU0-CD2<->GPU1-CD2. Therefore, the above three links between target cards are combined into a global interconnect topology.

[0119] In this embodiment, based on the inter-card link bandwidth between each target chip, a global interconnection topology connecting each target chip is determined, and a second communication network is generated based on the global interconnection topology. This can actively avoid low-bandwidth, high-latency physical links and prioritize the optimal link set under the current network state, thereby constructing an inter-card logical communication network for high-performance communication, which significantly improves the efficiency of subsequent inter-card set communication.

[0120] In some embodiments, see Figure 6 , Figure 6 This is a fourth flowchart illustrating the communication method provided in the embodiments of this application. Figure 3In step 304 shown, "controlling each target chip to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain the second processing result" can be achieved through... Figure 6 Steps 3041 to 3042 are implemented, and will be explained in detail below.

[0121] In step 3041, the data interaction paths between each target chip are determined based on the global interconnection topology used to generate the second communication network.

[0122] Here, the global interconnect topology represents the logical connection relationship between the target chips corresponding to each target computing card. The global interconnect topology is analyzed, and for each target chip, the target chips in other target computing cards that interact with the target chip (hereinafter referred to as the first target chip) are identified. The communication link between the target chip and the first target chip is then determined as the data interaction path.

[0123] For example, please continue to refer to [the example]. Figure 7 The global interconnect topology is: GPU0-CD0<->GPU1-CD0, GPU0-CD1<->GPU1-CD1, GPU0-CD2<->GPU1-CD2. Calculations show that each of the computed particles CD0, CD1, and CD2 corresponds to a target particle (not shown in the diagram). Based on the global interconnect topology "GPU0-CD0<->GPU1-CD0", the target particle corresponding to GPU0-CD0 is determined as the first target particle for data interaction with the target particle corresponding to GPU1-CD0. Therefore, the communication link between the target particles corresponding to GPU0-CD0 and GPU1-CD0 is determined as the data interaction path.

[0124] In step 3042, each target chip is controlled to perform a global aggregation operation on the first processing result of each target computing card based on the data interaction path to obtain the second processing result.

[0125] Here, when the second communication network contains M target cores, for each target core, the sub-processing result received by the target core is divided into M data blocks of equal size, resulting in multiple data blocks corresponding to the target core. Based on the data interaction path and the multiple data blocks corresponding to the target core, the target data block to be processed by the target core is determined. The target core is controlled to perform aggregation operations on the target data blocks (since the target core is bound to the computing core, it has computing capabilities), obtaining the aggregation result of the target core. This stage determines the aggregation result of the target core through M-1 iterations. The aggregation results of each target core are combined to obtain the second processing result. That is, the implementation process in step 3042 is the same as the implementation process in step 3032, and will not be repeated here.

[0126] In this embodiment, based on the global interconnection topology used to generate the second communication network, the data interaction path between each target chip is determined, and each target chip is controlled to perform a global aggregation operation on the first processing result of each target computing card based on the data interaction path to obtain the second processing result. This ensures that the aggregated communication task is executed efficiently on the link with the best performance, significantly reduces the communication latency of the inter-card aggregated communication task, and maximizes the actual throughput of cross-card data exchange.

[0127] Continue to refer to Figure 3 The following will be an explanation following step 304.

[0128] In step 305, for each target chip, the target chip is controlled to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

[0129] Here, for each target chip, based on the topology of the first communication network, the target path for data broadcasting of the target chip is determined, and the target chip is controlled to broadcast the second processing result to each computing chip in the target computing card based on the target path.

[0130] Example, reference Figure 13 , Figure 13 This is a fifth structural diagram of inter-card communication provided in this application embodiment. The target chip in GPU0 card 701 is the target chip corresponding to GPU0-CD2. The target chip in GPU0 card 701 is controlled to broadcast the second processing result to computing chips CD0 and CD1 based on the target path.

[0131] In some embodiments, monitoring and adaptively adjusting the target core can be achieved through the following steps: for each target core, acquiring the performance data of the target core in the second communication network; if the performance data meets preset abnormal conditions, redetermining a new target core from the target computing card where the target core is located; and reconstructing the first and second communication networks based on the new target core.

[0132] Here, point-to-point communication probes are periodically performed between the target cores that constitute the logical links of the second communication network. For each target core, by sending probe data packets of a preset size and measuring the transmission time, the instantaneous effective bandwidth and delay of the link are calculated, thereby obtaining the performance data of the target core in the second communication network.

[0133] The preset anomaly conditions are a set of configurable logical rules, including but not limited to: the actual link bandwidth of a link used by the target core is lower than a preset minimum bandwidth threshold, or the actual link latency is higher than a preset maximum latency threshold. When the performance data of the target core in the second communication network meets the preset anomaly conditions, a communication core that can optimize the link performance indicators of the second communication network (such as minimizing link latency and maximizing link bandwidth) is selected from the target computing card where the target core is located, and this communication core is determined as the new target core.

[0134] In the target computing card where the new target chip resides, a target logical topology for connecting each computing chip and the new target chip is determined, and a first communication network is regenerated based on the target logical topology. A global interconnect topology for connecting the new target chip and target chips in other target computing cards is determined, and a second communication network is regenerated based on the global interconnect topology.

[0135] In this embodiment, for each target chip, performance data of the target chip in the second communication network is obtained. When the performance data meets preset abnormal conditions, a new target chip is re-determined from the target computing card where the target chip is located, and the first and second communication networks are reconstructed based on the new target chip. This establishes a continuous and quantitative performance monitoring and feedback mechanism, which can identify target chips whose performance has significantly decreased due to hardware failure, link congestion or other reasons in real time. Only the target chip needs to be replaced within the card, realizing the localization of fault handling. Furthermore, by dynamically rebuilding the intra-card logical communication network and the inter-card logical communication network, the performance stability of inter-card aggregated communication is guaranteed.

[0136] In some embodiments, the communication method provided in this application can be applied to scenarios such as distributed model training and high-performance computing. It distributes data to be processed to multiple target computing cards. Each target computing card includes multiple computing cores and at least one communication core. Each computing core is allocated a data slice from the data to be processed. Next, for each target computing card, a target core corresponding to the target computing card is determined from the at least one communication core. A first communication network is constructed based on each computing core and the target core. Each computing core is controlled to perform reduction processing on each data slice through the first communication network to obtain a first processing result from the target computing card. This decomposes the data aggregation task between computing cards into local aggregation subtasks executed independently within each computing card, achieving localized processing of communication tasks and significantly reducing the amount of data in cross-card communication. Then, a second communication network is constructed based on each target core, and each target core is controlled to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain a second processing result. Finally, each target core is controlled to broadcast the second processing result to each computing core in the target computing card corresponding to the target core through the first communication network. In this way, the data aggregation stage in the aggregated communication task can be limited to be completed in the high-speed intracard link, making full use of the high bandwidth resources of the intracard under the multi-core chip packaging architecture, so that the low-speed inter-card link only carries a small amount of aggregated data, thereby improving the efficiency of aggregated communication between computing cards.

[0137] The following will describe an exemplary application of the communication method provided in the embodiments of this application in the hierarchical topology construction and communication scenario of multi-core chip packaging.

[0138] With the rapid expansion of artificial intelligence models and high-performance computing applications, the demand for graphics processing units (GPUs) computing power is constantly increasing. The development of a single monolithic die faces challenges such as process limitations, declining yields, and rising costs. Therefore, the adoption of multi-chip module (MCM) technology, which integrates multiple smaller computing dies into a single GPU card using advanced packaging technologies such as silicon interposers, has become the mainstream development direction in the industry. However, this architecture brings new challenges to aggregate communication (such as global reduction, global collection, and broadcasting) in large-scale distributed training:

[0139] 1) Heterogeneous and Layered Communication Topology: In multi-core GPU card (MCM-GPU) systems, the communication path exists at two levels: intra-card level (multiple computing cores within the same GPU card communicate via ultra-high bandwidth on-chip networks or interconnect buses) and inter-card level (different GPU cards are interconnected via high-speed interconnect links (NVLink), InfiniBand, Ethernet, etc.). Traditional communication libraries (such as NCCL) treat each GPU card as an atomic communication node, failing to perceive and utilize the rich intra-card interconnect topology.

[0140] 2) Sub-optimization and performance loss: Due to the lack of a global topology view, the aggregation communication algorithm can only be planned on the inter-card topology (e.g., building a ring or tree across cards). When data needs to be aggregated or distributed among multiple computing chips within the card, additional intra-card communication overhead will be generated, and due to improper planning, traffic bottlenecks and congestion may form at the boundaries between intra-card and inter-card devices, failing to fully utilize the extremely high communication bandwidth within the card.

[0141] 3) Uneven resource utilization: The interconnect bandwidth between intra-card computing chips is usually much higher than the inter-card link bandwidth. Traditional communication modes lead to insufficient utilization of high-bandwidth intra-card links, while relatively low-bandwidth inter-card links become performance bottlenecks, resulting in low overall system communication efficiency.

[0142] When optimizing multi-GPU card processing, related technologies typically construct a single, planar communication topology (such as a single ring or binary tree) between all physically interconnected computing units (including all computing chips). The drawback of this approach is that it ignores the fundamental differences in physical bandwidth and latency between "intra-card" and "inter-card" communication, resulting in high-speed intra-card links and low-speed inter-card links being mixed in a single logical topology. The low-speed inter-card links become a necessary component of the global communication chain, bearing a data transmission volume that is disproportionate to their capabilities, thus becoming a bottleneck for the overall system communication performance.

[0143] While some related technical implementations take into account the differences between within-node and between-node architectures, their "nodes" typically refer to a server or a device containing multiple independent GPU cards. Their shortcoming lies in failing to penetrate into the "encapsulated card" itself, and not managing and utilizing the ultra-high bandwidth interconnects between multiple computing chips within the card as an independently optimizable sub-topology. The optimization granularity remains too coarse. In summary, the lack of a fine-grained communication topology construction method that accurately matches the hierarchical interconnect characteristics of this novel hardware architecture of "multi-chip encapsulation" prevents the aggregated communication performance from reaching the theoretical hardware limit.

[0144] This application proposes a communication method to address the problems existing in related technologies, which includes the following improvements compared to related technologies:

[0145] Two logically decoupled but collaborative communication layers are constructed: a primary topology based on intra-card physical interconnection (the first communication network in the above embodiments) and a secondary topology based on inter-card physical networks (the second communication network in the above embodiments), realizing the explicit construction and collaboration of the "intra-card-inter-card" two-level topology. A link performance (bandwidth, latency) matrix based on real-time detection is proposed to dynamically elect the optimal card proxy node (the target chip in the above embodiments) for each GPU card, seamlessly switching according to network status, enabling the communication topology to have adaptive optimization capabilities and realizing a performance-aware dynamic proxy election mechanism, overcoming the limitations of statically assigning card proxy nodes. The global collective communication algorithm is decomposed into a "local aggregation / distribution" stage executed in the intra-card primary topology and a "global exchange" stage executed in the inter-card proxy node secondary topology, realizing a two-level decomposition and scheduling strategy for communication tasks.

[0146] By implementing hierarchical topology construction and communication scheduling in this embodiment, the following significant benefits can be achieved:

[0147] 1) Significantly reduce communication latency: Most data aggregation / distribution operations are restricted to the card with nanosecond latency, and only the streamlined intermediate results are exchanged through the card-to-card network with microsecond latency, thus significantly reducing end-to-end communication latency.

[0148] 2) Adaptability and high robustness: Through the dynamic card agent node election mechanism, the system can automatically sense and adapt to the heterogeneity and state changes of network connections, always select the near-optimal communication exit, avoid performance degradation caused by statically bound performance-limited links or faulty links, and enhance the stability and performance consistency of the system in complex environments.

[0149] 3) Significantly improves effective bandwidth utilization: The high bandwidth resources within the card are fully utilized to handle the main data exchange tasks, and the inter-card links only carry a small amount of aggregated data, avoiding them from becoming a bottleneck, so that the effective communication bandwidth of the system approaches the bandwidth within the card.

[0150] 4) Improve system scalability: The increase in system communication overhead is mainly strongly correlated with the number of GPU cards (rather than the total number of computing chips). Increasing the number of chips per card has little impact on inter-card communication pressure, which is beneficial for building ultra-large-scale computing systems.

[0151] In this embodiment, the multi-core GPU card system is hierarchically abstracted (including card proxy nodes and internal computing nodes), enabling the decomposition of global aggregated communication operations into intra-card subtasks and inter-card subtasks, and scheduling is performed on the constructed two-level communication topology. Furthermore, based on real-time network performance metrics (such as point-to-point bandwidth and latency), card proxy nodes are dynamically elected and migrated. The above method can be implemented in a communication library, task scheduler, and computing device.

[0152] In this embodiment of the application, a hierarchical topology construction and communication method for achieving efficient inter-card communication in a multi-core chip co-packing scenario is described. The technical solution includes four steps: system modeling and dynamic performance detection, dynamic hierarchical topology construction, hierarchical communication scheduling and execution, and runtime monitoring and adaptive adjustment.

[0153] The following describes the process of system modeling and dynamic performance probing. This involves identifying all multi-chip GPU cards (denoted as cards) in the system. , =0, 1...). For each card Identify all computing cores within it { , ...} and all I / O chips with inter-card connectivity { , ...} (the communication core in the above embodiments), which constitute the set of candidate proxy nodes for the card. A dynamic performance matrix is ​​actively measured and maintained, which records the candidate proxy nodes for each card. To each agent candidate node on all other cards in the system Real-time communication performance metrics (e.g., bandwidth, latency). For each card... The dynamic model is a hierarchical group of nodes, which includes a dynamic card agent node (selected from the set of card agent candidate nodes) and multiple internal computing nodes.

[0154] The following explains the process of constructing a dynamic hierarchical topology. For each card... In all its internal compute nodes { Based on the physical interconnect structure within the card, a high-bandwidth logical communication subnet (e.g., ring, mesh, etc.) is constructed. Before each global aggregate communication task is executed or a dynamic election mechanism for card agent nodes is triggered periodically, each card... Run a card proxy node election algorithm. This algorithm aims to minimize the estimated completion time of the current set communication operation and, based on the current dynamic performance matrix, elects the optimal card proxy node from the set of candidate proxy nodes for this card at the current moment. (The target chip in the above embodiments). The election strategy may include minimizing the maximum communication latency, maximizing the critical path bandwidth, and achieving inter-card link load balancing, etc.

[0155] The following describes the hierarchical communication scheduling and execution process. For a given set of communication operations, various algorithms can be flexibly used, selecting different algorithms based on different interconnection forms. A hierarchical ring algorithm is used as an example for scheduling. In the local reduce-scatter phase within the card, communication tasks are scheduled to execute independently on the intra-card logical communication subnet of each card. Each internal computing node... The data it holds is fragmented and reduced within the card, and then distributed and sent to the card agent node. Ultimately, each card agent node This involves holding only a partial sum of the global data (the first processing result in the above embodiment). For example, a card contains 4 compute nodes and 2 card agent nodes, with data [1, 2], [3, 4], [5, 6], and [7, 8] respectively. The reduction (taking summation as an example) results in [1+3+5+7, 2+4+6+8], which is [16, 20]. The distributed result is that each of the two card agent nodes receives... The data, namely

[16] and

[20] . Communication at this stage is completed at high speed within the card and does not occupy the inter-card link.

[0156] During the AllReduce phase, each card's card agent node... Local sums are collected from the internal computing nodes of this card. Then, the aggregation communication task is scheduled to be executed on the inter-card logical communication network, involving only the card's agent nodes. They perform global reduction-distribution and global collection (All-Gather) (reducing the data from each card's agent nodes and broadcasting the reduction result to the agent nodes). At this stage, the data volume is significantly reduced, and communication between agent nodes is achieved through optimal paths. In the intra-card result global collection / broadcast stage, each card's agent node broadcasts the complete global reduction result (the second processing result in the above embodiment) to all internal computing nodes through the intra-card logical communication subnet.

[0157] The following describes the runtime monitoring and adaptive adjustment process. During communication, the performance of each link is continuously monitored. If a significant performance degradation is detected in a critical link of the current card proxy node, or if a better proxy node is found through periodic evaluation, dynamic migration of the proxy node can be triggered. The migration process must ensure the correct transfer of communication context and synchronization of the cluster view, while remaining transparent to upper-layer applications.

[0158] Next, an exemplary description of the communication method provided by the embodiments of the present application is given. For example, refer to Figure 7 , Figure 7 Figure 1 is a schematic diagram of the first structure of inter-card communication provided by the embodiments of the present application. In the first structure of inter-card communication, there are two interconnected GPU cards, namely GPU0 card 701 and GPU1 card 702. Three homogeneous computing dies (CD0-CD2) in each card are interconnected through a 2.5D silicon interposer in a full-mesh network, and the bidirectional link bandwidth between the computing dies within the card is X GB / s. The two GPU cards are fully connected through 3 high-speed interconnect links (NVLink), and the bandwidth of each link is Y GB / s (Y < X). The total amount of data for performing a global reduction operation across two GPU cards (a total of 6 computing dies) is M.

[0159] In the system modeling and dynamic performance detection phase, all multi-die-packaged GPU cards in the system are identified. It is identified that there are 3 computing dies and 3 proxy nodes (here, it can be understood that one I / O die corresponding to each computing die is a proxy node) in GPU0 card and GPU1 card respectively, and there are 3 NVLink links between the two GPU cards. For example, refer to Figure 8 , Figure 8 Figure 2 is a schematic diagram of the first link bandwidth provided by the embodiments of the present application. The available bandwidth of the link decreases as the load increases. Here, the empty load scenario is shown. For example, the link bandwidth between computing die CD0 in GPU0 card 701 and computing die CD0 in GPU1 card 702 is Y GB / s, the link bandwidth between computing die CD0 in GPU0 card 701 and computing die CD1 in GPU1 card 702 is X GB / s, and the link bandwidth between computing die CD0 in GPU0 card 701 and computing die CD1 in GPU1 card 702 is XY / (X + Y) GB / s, where Y > X > XY / (X + Y).

[0160] In the dynamic hierarchical topology construction phase, there is 1 bidirectional ring path in GPU0 card and GPU1 card respectively. Therefore, the intra-card topologies of the two are the same, and the first-level topologies constructed respectively are represented as "CD0 <-> CD1 <-> CD2 <-> CD0". Continuing to refer to Figure 7 , since there are three groups of interconnected nodes between GPU0 card 701 and GPU1 card 702, that is, three groups of same-numbered dies, based on Figure 8 the schematic diagram of the link bandwidth maintained in Figure 2, the second-level topology constructed is represented as three bidirectional ring paths: "Path A: GPU0-CD0 <-> GPU1-CD0", "Path B: GPU0-CD1 <-> GPU1-CD1", "Path C: GPU0-CD2 <-> GPU1-CD2".

[0161] In the hierarchical communication scheduling and execution stage, in the first-level topology of each card, the in-card specification data is evenly divided into three parts of data, and each of the three proxy nodes receives one part of the data. Then, global specification is performed in the second-level topology, and each path processes one part of the data. Finally, in the first-level topology, the global specification result is broadcast to each computing die through the proxy nodes. Exemplarily, the data flow is expressed as: "Path A: (GPU0-CD1 GPU0-CD2) GPU0-CD0<->GPU1-CD0 (GPU1-CD1 GPU1-CD2)", "Path B: (GPU0-CD2 GPU0-CD0) GPU0-CD1<->GPU1-CD1 (GPU1-CD2 GPU1-CD0)", "Path C: (GPU0-CD0 GPU0-CD1) GPU0-CD2<->GPU1-CD2 (GPU1-CD0 GPU1-CD1)". Among them, the proxy node of GPU0 card in Path A is the I / O die corresponding to GPU0-CD0, and the proxy node of GPU1 card is the I / O die corresponding to GPU1-CD0 (at this time, the computing die and the I / O die correspond one by one).

[0162] In the runtime monitoring and adaptive adjustment stage, if it is detected that the load of GPU0-CD1 in Path B is too high, then it is adjusted so that Path A and Path C each process 1 / 2 of the data.

[0163] Exemplarily, refer to Figure 9 , Figure 9 is the second structural schematic diagram of inter-card communication provided by the embodiment of the present application. In the second structure of inter-card communication, there are three interconnected GPU cards, namely GPU0 card 701, GPU1 card 702, and GPU2 card 703. Three homogeneous computing dies (CD0-CD2) in each card are interconnected through a 2.5D silicon interposer in a full-mesh network, and the bidirectional link bandwidth between the in-card computing dies is X GB / s. The three GPU cards are heterogeneously connected through 3 NVLink links, and the bandwidth of each link is Y GB / s (Y < X). The total amount of data for performing the global specification operation across three GPU cards (a total of 9 computing dies) is M.

[0164] In the system modeling and dynamic performance detection stage, identify the proxy nodes between all multi-die-packaged GPU cards in the system, as well as performance data such as link bandwidth. Exemplarily, refer to Figure 10 , Figure 10This is a schematic diagram of the second link bandwidth provided in an embodiment of this application. The link bandwidth between computing chip CD1 in GPU0 card 701 and computing chip CD0 in GPU1 card 702 is Y GB / s, the link bandwidth between computing chip CD0 in GPU0 card 701 and computing chip CD1 is X GB / s, and the link bandwidth between computing chip CD0 in GPU0 card 701 and computing chip CD0 in GPU1 card 702 is XY / (X+Y) GB / s, where Y>X>XY / (X+Y).

[0165] During the dynamic hierarchical topology construction phase, GPU0, GPU1, and GPU2 each contain one bidirectional ring path. Therefore, their in-card topologies are consistent, and the first-level topologies constructed are represented as "CD0<->CD1<->CD2<->CD0". (Continue to refer to...) Figure 9 Due to the heterogeneity of the interconnection, the proxy nodes will be inconsistent depending on the direction of the data flow, and based on... Figure 10 The link bandwidth diagram maintained in the diagram shows the constructed two-level topology as follows: "Path A: GPU0-CD1->GPU1-CD0->GPU1-CD2->GPU2-CD1->GPU2-CD0->GPU0-CD2->GPU0-CD1", and "Path B: GPU0-CD2->GPU2-CD0<->GPU2-CD1<->GPU1-CD2<->GPU1-CD0<->GPU0-CD1<->GPU0-CD2". In Path A, the proxy node for GPU0 is the I / O chip corresponding to GPU0-CD1, the proxy node for GPU1 is the I / O chip corresponding to GPU1-CD2, and the proxy node for GPU2 is the I / O chip corresponding to GPU2-CD0 (at this point, there is a one-to-one correspondence between the calculated chips and the I / O chips).

[0166] During the hierarchical communication scheduling and execution phase, in the primary topology of each card, the card-specific reduction data is aggregated to the proxy node. Then, the global reduction is executed in the secondary topology, and finally, in the primary topology, the global reduction result is broadcast to each computing chip through the proxy node.

[0167] Example, reference Figure 11 , Figure 11This is a schematic diagram of the third structure for inter-card communication provided in this application embodiment. The proxy node for GPU0 card 701 is the I / O chip corresponding to GPU0-CD2; the proxy node for GPU1 card 702 is the I / O chip corresponding to GPU1-CD0; and the proxy node for GPU2 card 703 is the I / O chip corresponding to GPU2-CD1 (at this time, there is a one-to-one correspondence between the computation chip and the I / O chip). In the first-level topology of GPU0 card, the protocol data within the card is aggregated into the I / O chip corresponding to GPU0-CD2. In the first-level topology of GPU1 card, the protocol data within the card is aggregated into the I / O chip corresponding to GPU1-CD0. In the first-level topology of GPU2 card, the protocol data within the card is aggregated into the I / O chip corresponding to GPU2-CD1.

[0168] Example, reference Figure 12 , Figure 12 This is a fourth structural diagram of inter-card communication provided in the embodiments of this application. The global protocol data flow between GPU0 card 701, GPU1 card 702 and GPU2 card 703 is represented as: "GPU0-CD1->GPU0-CD2->GPU2-CD0->GPU2-CD1->GPU1-CD2->GPU1-CD0".

[0169] Example, reference Figure 13 , Figure 13 This is a fifth structural diagram of inter-card communication provided in this application embodiment. In the primary topology of GPU0 card 701, the global reduction result is broadcast to each computing chip within GPU0 card through the I / O chip corresponding to GPU0-CD2. In the primary topology of GPU1 card 702, the global reduction result is broadcast to each computing chip within GPU1 card through the I / O chip corresponding to GPU1-CD0. In the primary topology of GPU2 card 703, the global reduction result is broadcast to each computing chip within GPU2 card through the I / O chip corresponding to GPU2-CD1.

[0170] In this embodiment, a software-defined network is used to intelligently sense and adapt to the hierarchical characteristics of hardware interconnection. Communication tasks are decoupled and coordinated in the two-level topology of high-speed "intra-card" and low-speed "inter-card", thereby minimizing the occupation of inter-card links and data transmission volume, significantly improving the overall efficiency of cross-card aggregate communication, reducing communication latency and improving system scalability. This enables the optimization of aggregate communication performance and management between multiple GPU cards or multiple computing nodes in a multi-core packaged hardware architecture.

[0171] The following description continues to illustrate the exemplary structure of the communication device 455 provided in the embodiments of this application as a software module. In some embodiments, such as... Figure 2As shown, the software modules stored in the communication device 455 of the memory 450 may include: a data allocation module 4551, configured to allocate the data to be processed to multiple target computing cards in response to receiving data to be processed, each target computing card including multiple computing chips and at least one communication chip, each computing chip being allocated a data fragment of the data to be processed, and the communication chip having inter-card communication functionality; a chip determination module 4552, configured to determine the target chip corresponding to each target computing card from at least one communication chip, and construct a first communication network based on each computing chip and the target chip; and a first processing module 4553, configured to control each A computing chip performs reduction processing on each data fragment through a first communication network to obtain a first processing result of the target computing card, and sends the first processing result to the target chip corresponding to the target computing card through the first communication network; a second processing module 4554 is used to construct a second communication network based on each target chip, and control each target chip to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain a second processing result; a data broadcasting module 4555 is used to control each target chip to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

[0172] In some embodiments, the chip determination module 4552 is further configured to, for each communication chip, obtain communication performance indicators between the communication chip and communication chips in other target computing cards; determine the performance evaluation data of the communication chip based on the communication performance indicators, and sort each communication chip in descending order of performance evaluation data to obtain a communication chip sequence; and determine the first K communication chips in the communication chip sequence as the target chips corresponding to the target computing card, where K is a positive integer.

[0173] In some embodiments, the chip determination module 4552 is further configured to obtain the physical interconnection structure between each computing chip and the target chip; determine the link performance index between each computing chip and the target chip based on the physical interconnection structure; determine the target logical topology for connecting each computing chip and the target chip based on the link performance index; and generate a first communication network based on the target logical topology.

[0174] In some embodiments, the data sharding includes multiple data blocks. The first processing module 4553 is further configured to, for each computing chip, determine a target computing chip adjacent to the computing chip based on the topology of the first communication network; determine the target data block to be processed by the computing chip based on the multiple data blocks corresponding to the computing chip and the multiple data blocks corresponding to the target computing chip; control the computing chip to perform aggregation operations on the target data block to obtain a first aggregation result of the computing chip; and combine multiple first aggregation results to obtain a first processing result.

[0175] In some embodiments, the first processing module 4553 is further configured to divide the first processing result into K sub-processing results; and send each sub-processing result to each target core, wherein the target core corresponds one-to-one with the sub-processing result.

[0176] In some embodiments, the second processing module 4554 is further configured to acquire the inter-card link bandwidth between each target chip; determine the global interconnection topology connecting each target chip based on the inter-card link bandwidth; and generate a second communication network based on the global interconnection topology.

[0177] In some embodiments, the second processing module 4554 is further configured to determine the data interaction path between each target chip based on the global interconnection topology used to generate the second communication network; and control each target chip to perform a global aggregation operation on the first processing result of each target computing card based on the data interaction path to obtain the second processing result.

[0178] In some embodiments, the second processing module 4554 is further configured to: acquire performance data of the target chip in the second communication network for each target chip; if the performance data meets preset abnormal conditions, redetermine a new target chip from the target computing card where the target chip is located; and reconstruct the first communication network and the second communication network based on the new target chip.

[0179] This application provides a computer program product, which includes computer-executable instructions or a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions or computer program from the computer-readable storage medium and executes the computer-executable instructions or computer program, causing the electronic device to perform the communication method described in this application.

[0180] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the communication method provided in this application, for example... Figure 3 The communication method is shown.

[0181] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0182] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0183] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0184] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0185] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A communication method, characterized in that, The method includes: In response to receiving data to be processed, the data to be processed is allocated to multiple target computing cards. Each target computing card includes multiple computing chips and at least one communication chip. Each computing chip is allocated a data fragment of the data to be processed. The communication chip has inter-card communication functionality. For each target computing card, a target chip corresponding to the target computing card is determined from the at least one communication chip, and a first communication network is constructed based on each computing chip and the target chip; Each computing chip is controlled to perform reduction processing on each data fragment through the first communication network to obtain the first processing result of the target computing card, and the first processing result is sent to the target chip corresponding to the target computing card through the first communication network. A second communication network is constructed based on each of the target chips, and each target chip is controlled to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain a second processing result; For each target chip, the target chip is controlled to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

2. The method according to claim 1, characterized in that, Determining the target chip corresponding to the target computing card from the at least one communication chip includes: For each of the aforementioned communication chips, obtain the communication performance indicators between the communication chip and the communication chips in other target computing cards; Based on the communication performance indicators, the performance evaluation data of the communication core is determined, and the communication cores are sorted in descending order of the performance evaluation data to obtain a communication core sequence. The first K communication chips in the communication chip sequence are determined as the target chips corresponding to the target computing card, where K is a positive integer.

3. The method according to claim 1, characterized in that, The construction of the first communication network based on each of the computing chips and the target chip includes: Obtain the physical interconnect structure between each computing chip and the target chip; Based on the physical interconnect structure, determine the link performance metrics between each computing core and the target core; Based on the link performance metrics, a target logical topology for connecting each computing core and the target core is determined; The first communication network is generated based on the target logical topology.

4. The method according to claim 1, characterized in that, The data shards comprise multiple data blocks. The control of each computing chip via the first communication network performs reduction processing on each data shard to obtain a first processing result from the target computing card, including: For each of the computing cores, based on the topology of the first communication network, target computing cores adjacent to the computing cores are determined; Based on the multiple data blocks corresponding to the computing kernel and the multiple data blocks corresponding to the target computing kernel, the target data block to be processed by the computing kernel is determined, and the computing kernel is controlled to perform aggregation operation on the target data block to obtain the first aggregation result of the computing kernel. The first processing result is obtained by combining multiple first aggregation results.

5. The method according to claim 2, characterized in that, The step of sending the first processing result to the target chip corresponding to the target computing card through the first communication network includes: The first processing result is divided into K sub-processing results; Each of the sub-processing results is sent to each of the target cores, wherein each target core corresponds one-to-one with the sub-processing result.

6. The method according to claim 1, characterized in that, The construction of the second communication network based on each of the target chips includes: Obtain the inter-card link bandwidth between each of the target chips; Based on the inter-card link bandwidth, determine the global interconnect topology connecting each of the target cores; The second communication network is generated based on the global interconnection topology.

7. The method according to claim 1, characterized in that, The control mechanism for each target chip to perform global reduction processing on the first processing result of each target computing card through the second communication network to obtain a second processing result includes: Based on the global interconnection topology used to generate the second communication network, the data interaction paths between each of the target core particles are determined; Each target chip is controlled to perform a global aggregation operation on the first processing result of each target computing card based on the data interaction path to obtain the second processing result.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: For each target chip, acquire the performance data of the target chip in the second communication network; If the performance data meets the preset abnormal conditions, a new target core is re-determined from the target computing card where the target core is located; The first communication network and the second communication network are reconstructed based on the new target core.

9. A communication device, characterized in that, The device includes: A data allocation module is used to allocate the data to be processed to multiple target computing cards in response to receiving data to be processed. Each target computing card includes multiple computing cores and at least one communication core. Each computing core is allocated a data fragment of the data to be processed, and the communication core has inter-card communication function. A chip determination module is used to determine, for each target computing card, the target chip corresponding to the target computing card from the at least one communication chip, and to construct a first communication network based on each computing chip and the target chip; The first processing module is used to control each of the computing chips to perform reduction processing on each of the data fragments through the first communication network to obtain the first processing result of the target computing card, and send the first processing result to the target chip corresponding to the target computing card through the first communication network. The second processing module is used to construct a second communication network based on each of the target chips, and control each of the target chips to perform global reduction processing on the first processing result of each of the target computing cards through the second communication network to obtain a second processing result; The data broadcasting module is used to control each target chip to broadcast the second processing result to each computing chip in the target computing card corresponding to the target chip through the first communication network.

10. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the communication method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the communication method according to any one of claims 1 to 8.

12. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the communication method according to any one of claims 1 to 8.

Images