A communication and computation collaborative optimization method and device for a heterogeneous intelligent computing network

Abstract

The application provides a communication and calculation cooperative optimization method and device for a heterogeneous intelligent computing network, wherein the method comprises the following steps: generating a task access matrix according to a global sparse matrix; determining a node in which a data block resides and a processing unit in the node according to the task access matrix and node topology information; determining a communication transmission mode of the data block according to a sparse density and column dimension of the data block; and performing multi-flow concurrency to complete a calculation task of a current neural network layer. The application alleviates the parallel expansion bottleneck of a large model training of a computing network, and effectively slows down the growth trend of communication cost.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence technology, specifically to a collaborative optimization method for communication, computation, and load distribution of the underlying core operators of a large model in a multi-GPU cluster and heterogeneous network environment. Background Technology

[0002] In current neural network training, sparse-dense matrix multiplication is a core, fundamental operator that consumes significant computing power and time. With the surge in the number of large model parameters and the scale of distributed computing, the growth in computing power far exceeds the growth in communication bandwidth, making communication the main performance bottleneck for training large models in computing networks.

[0003] Existing technologies face three significant challenges in addressing this problem: First, current data partitioning methods largely rely on complex graph partitioning tools (such as METIS), whose preprocessing is extremely time-consuming and cannot meet the needs of dynamic computing networks or short-term iterative tasks. Second, modern intelligent computing networks exhibit significant heterogeneity, with inter-node InfiniBand network bandwidth being far lower than intra-node NVLink network bandwidth. Existing distributed algorithms lack awareness of this hardware topology, leading to a large amount of expensive and redundant cross-node data fetching. Finally, regarding the choice of communication granularity, traditional block transmission transmits a large amount of redundant zero-data, while fine-grained transmission, although reducing data volume, introduces extremely high software protocol overhead and network latency. Existing systems lack quantifiable dynamic adaptive decision-making models. These shortcomings directly limit the parallel scalability of mega-card-scale computing in heterogeneous intelligent computing networks. Summary of the Invention

[0004] This invention describes a method and apparatus for collaborative optimization of communication and computing in heterogeneous intelligent computing networks, which can solve the above-mentioned technical problems.

[0005] According to the first aspect, a method and apparatus for collaborative optimization of communication and computing in heterogeneous intelligent computing networks are provided, comprising:

[0006] Based on the global sparse matrix, a task access matrix is ​​generated. The task access matrix is ​​used to characterize the access requirements of the processing unit to data blocks of the dense matrix. The dense matrix refers to the weight matrix that is multiplied with the global sparse matrix and is divided into multiple data blocks by columns.

[0007] Based on the task access matrix and node topology information, the node where the data block resides and the processing unit within the node are determined, wherein the node topology information represents the affiliation relationship between the processing unit and the node it resides in;

[0008] Based on the sparse density and column dimension of the data block, the communication transmission mode of the data block is determined, and the communication transmission mode includes whole block transmission and fine-grained sparse transmission.

[0009] Multi-stream concurrency is executed to complete the computational task of the current neural network layer. The asynchronous communication stream aggregates the access requirements of remote data blocks within the same node according to the communication transmission mode of the data blocks, and initiates cross-node asynchronous communication to cache the remote data blocks in the node-level buffer; the high-speed exchange stream within the node distributes the data blocks to other processing units within the node; and the foreground computation stream performs local computation after the required data is ready.

[0010] In some embodiments, generating the task access matrix based on the global sparse matrix includes:

[0011] The global sparse matrix is ​​divided into multiple row slices, and each row slice is assigned to a processing unit.

[0012] Each processing unit divides the row slice into multiple data blocks according to the column index range of the row slice, counts the number of columns in the data blocks that have at least one non-zero element, and obtains a local non-zero column count vector.

[0013] All processing units exchange their respective local non-zero column count vectors via full collection communication and aggregate them to construct the task access matrix;

[0014] In this context, the row index of the task access matrix corresponds to the processing unit number, the column index corresponds to the data block number, and the value of each element in the task access matrix represents the number of times the corresponding processing unit needs to access the non-zero column in the corresponding data block.

[0015] In some embodiments, determining the node where each data block resides and the processing unit within that node based on the task access matrix and node topology information includes:

[0016] A two-stage divide-and-conquer strategy is adopted:

[0017] In the first stage, based on the node-level aggregation results of the task access matrix, the node where each data block resides is determined;

[0018] In the second stage, for each node, based on the access requirements of each processing unit within the node for data blocks, the processing unit where each data block resides is determined.

[0019] In some embodiments, determining the node where each data block resides based on the node-level aggregation result of the task access matrix includes:

[0020] Aggregate the task access matrix by node to obtain the access requirement matrix of each node for all data blocks;

[0021] An initial resident node is randomly assigned to each data block, and the initial total cross-node communication cost is calculated. The total cross-node communication cost is a value calculated based on the access requirements of all nodes to their non-local data blocks and the cross-node unit cost.

[0022] Examine each pair of nodes in turn, and for each data block currently assigned to these two nodes, evaluate the change in cross-node communication cost that would result from moving the data block from its current node to the other node.

[0023] Select the data block with the largest positive change in cross-node communication cost. If the largest change in cross-node communication cost is greater than the current threshold, move the data block to another node and update the total cross-node communication cost.

[0024] When the maximum change in cross-node communication cost is less than the first threshold or the number of iterations reaches the first preset number of iterations, the iteration is terminated, and the resident node of each data block is output.

[0025] In some embodiments, determining the processing unit where each data block resides, based on the access requirements of each processing unit within the node, for each node, includes:

[0026] For each node, extract the set of data blocks belonging to that node, as well as the access requirements of each processing unit within that node for those data blocks;

[0027] An initial resident processing unit is randomly assigned to each data block, and the initial total intra-node communication cost is calculated. The total intra-node communication cost is determined based on the communication cost of the high-speed link within the node.

[0028] Examine each pair of processing units in turn, and for each data block currently assigned to these two processing units, evaluate the change in intra-node communication cost that would result from moving the data block from its current processing unit to the other processing unit.

[0029] Select the data block with the largest positive change in intra-node communication cost. If the largest change in intra-node communication cost is greater than the second threshold, move the data block to another processing unit and update the total intra-node communication cost.

[0030] When the change in the largest intra-node communication cost is less than the second threshold or the number of iterations reaches the second preset number of iterations, the iteration is terminated, and the processing unit where each data block resides in the current node is output.

[0031] In some embodiments, determining the communication transmission mode of the data block based on its sparse density and column dimension includes:

[0032] Based on the hardware network model, a critical sparse column density function is constructed, and the transmission task of each data block is mapped to a two-dimensional decision space with its column dimension and sparse density as coordinates to obtain the mapping point.

[0033] The communication transmission mode is determined based on the position of the mapping point in the two-dimensional decision space:

[0034] If the mapping point is located in a high-density region or a low-dimensional region, the block communication mode is adopted.

[0035] If the mapping point is located in a low-density and high-dimensional region, a fine-grained sparse communication mode is adopted.

[0036] In some embodiments, performing multi-stream concurrency to complete the computational task of the current neural network layer includes:

[0037] During pipeline operation, the background asynchronous communication stream continues to run, aggregating the access requests of each processing unit within the same node to remote data blocks according to the communication mode label, and initiating cross-node asynchronous communication to read the remote data blocks into the node-level buffer.

[0038] Once the node-level synchronization conditions are met and the remote data block transfer is completed, the high-speed switching stream within the node distributes the remote data block to other processing units within the node that require it through the high-speed interconnection network within the node.

[0039] After confirming that all the required data is ready, the foreground computing stream independently performs local sparse-dense matrix multiplication calculations.

[0040] The asynchronous communication stream, the high-speed exchange stream within the node, and the foreground computing stream are synchronized and woken up by a fine-grained event lock mechanism until the computing task of the current neural network layer is completed.

[0041] According to the second aspect, a communication and computing collaborative optimization device for heterogeneous intelligent computing networks is provided, comprising:

[0042] The first processing module is used to generate a task access matrix based on the global sparse matrix. The task access matrix is ​​used to characterize the access requirements of each processing unit to the data blocks of the dense matrix. The dense matrix refers to the weight matrix that is multiplied by the global sparse matrix and is divided into multiple data blocks by columns.

[0043] The second processing module is used to determine the node where the data block resides and the processing unit within the node based on the task access matrix and node topology information, wherein the node topology information represents the affiliation relationship between the processing unit and the node it resides in.

[0044] The third processing module is used to determine the communication transmission mode of the data block based on the sparse density and column dimension of the data block. The communication transmission mode includes whole block transmission and fine-grained sparse transmission.

[0045] The fourth processing module is used to perform multi-stream concurrency to complete the computation task of the current neural network layer. The asynchronous communication stream aggregates the access requirements of remote data blocks within the same node according to the communication transmission mode of the data blocks, and initiates cross-node asynchronous communication to cache the remote data blocks in the node-level buffer; the high-speed exchange stream within the node distributes the data blocks to other processing units within the node; and the foreground computation stream performs local computation after the required data is ready.

[0046] In some embodiments, the first processing module is configured to divide the global sparse matrix into multiple row slices, with each row slice assigned to a processing unit.

[0047] Each processing unit divides the row slice into multiple data blocks according to the column index range of the row slice, counts the number of columns in the data blocks that have at least one non-zero element, and obtains a local non-zero column count vector.

[0048] All processing units exchange their respective local non-zero column count vectors via full collection communication and aggregate them to construct the task access matrix;

[0049] In this context, the row index of the task access matrix corresponds to the processing unit number, the column index corresponds to the data block number, and the value of each element in the task access matrix represents the number of times the corresponding processing unit needs to access the non-zero column in the corresponding data block.

[0050] In some embodiments, the second processing module is configured to employ a two-stage divide-and-conquer strategy:

[0051] In the first stage, based on the node-level aggregation results of the task access matrix, the node where each data block resides is determined;

[0052] In the second stage, for each node, based on the access requirements of each processing unit within the node for data blocks, the processing unit where each data block resides is determined.

[0053] In some embodiments, the second processing module is used to aggregate the task access matrix by node to obtain an access requirement matrix for each node to all data blocks.

[0054] An initial resident node is randomly assigned to each data block, and the initial total cross-node communication cost is calculated. The total cross-node communication cost is a value calculated based on the access requirements of all nodes to their non-local data blocks and the cross-node unit cost.

[0055] Examine each pair of nodes in turn, and for each data block currently assigned to these two nodes, evaluate the change in cross-node communication cost that would result from moving the data block from its current node to the other node.

[0056] Select the data block with the largest positive change in cross-node communication cost. If the largest change in cross-node communication cost is greater than the current threshold, move the data block to another node and update the total cross-node communication cost.

[0057] When the maximum change in cross-node communication cost is less than the first threshold or the number of iterations reaches the first preset number of iterations, the iteration is terminated, and the resident node of each data block is output.

[0058] In some embodiments, the second processing module is configured to extract, for each node, a set of data blocks belonging to the node, and the access requirements of each processing unit within the node for the data blocks;

[0059] An initial resident processing unit is randomly assigned to each data block, and the initial total intra-node communication cost is calculated. The total intra-node communication cost is determined based on the communication cost of the high-speed link within the node.

[0060] Examine each pair of processing units in turn, and for each data block currently assigned to these two processing units, evaluate the change in intra-node communication cost caused by moving the data block from its current processing unit to the other processing unit.

[0061] Select the data block with the largest positive change in intra-node communication cost. If the largest change in intra-node communication cost is greater than the second threshold, move the data block to another processing unit and update the total intra-node communication cost.

[0062] When the change in the largest intra-node communication cost is less than the second threshold or the number of iterations reaches the second preset number of iterations, the iteration is terminated, and the processing unit where each data block resides in the current node is output.

[0063] In some embodiments, the third processing module is used to construct a critical sparse column density function based on the hardware network model, and map the transmission task of each data block to a two-dimensional decision space with its column dimension and sparse density as coordinates to obtain the mapping point.

[0064] The communication transmission mode is determined based on the position of the mapping point in the two-dimensional decision space:

[0065] If the mapping point is located in a high-density region or a low-dimensional region, the block communication mode is adopted.

[0066] If the mapping point is located in a low-density and high-dimensional region, a fine-grained sparse communication mode is adopted.

[0067] In some embodiments, the fourth processing module is configured to, during pipeline operation, continuously run the background asynchronous communication stream, aggregate the access requests of each processing unit within the same node to remote data blocks according to the communication mode label, and initiate cross-node asynchronous communication to read the remote data blocks into the node-level buffer.

[0068] Once the node-level synchronization conditions are met and the remote data block transfer is completed, the high-speed switching stream within the node distributes the remote data block to other processing units within the node that require it through the high-speed interconnection network within the node.

[0069] After confirming that all the required data is ready, the foreground computing stream independently performs local sparse-dense matrix multiplication calculations.

[0070] The asynchronous communication stream, the high-speed exchange stream within the node, and the foreground computing stream are synchronized and woken up by a fine-grained event lock mechanism until the computing task of the current neural network layer is completed.

[0071] According to a third aspect, a computer storage medium is provided, on which a computer program is stored, which, when executed by one or more processors, implements the communication and computing collaborative optimization method for heterogeneous intelligent computing networks as described in any of the above embodiments.

[0072] According to a fourth aspect, an electronic device is provided, including a memory and one or more processors, wherein a computer program is stored on the memory, and the computer program, when executed by the one or more processors, implements the communication and computing collaborative optimization method for heterogeneous intelligent computing networks as described in any of the above embodiments.

[0073] The methods and apparatuses described in the embodiments of this specification alleviate the parallel scalability bottleneck of large-scale model training in computing networks. In real gigabit-level heterogeneous network interconnected multi-node, multi-card clusters, compared with existing baseline algorithms, an average end-to-end acceleration of approximately 4.0 to 5.1 times is achieved in a multi-node environment. Secondly, this invention aligns with the characteristics of heterogeneous intelligent computing. Addressing the pain points of increased data sharing and cross-node communication overhead, its topology-aware two-stage pipelined distribution mechanism effectively mitigates the growth trend of communication costs when computing nodes are horizontally expanded, demonstrating good strong and weak scalability. Finally, this invention improves task throughput through lightweight scheduling. Due to the extremely low algorithm preprocessing time, the system significantly reduces preprocessing startup costs, enabling better support for rapid response to short-term, dynamic large-scale model inference or function computation tasks, thus contributing to improved overall utilization of computing network resources. Attached Figure Description

[0074] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0075] Figure 1 This document illustrates a flowchart of a communication and computing collaborative optimization method for heterogeneous intelligent computing networks, as provided in an embodiment of this specification.

[0076] Figure 2 This document illustrates a flowchart of a communication and computing collaborative optimization method for heterogeneous intelligent computing networks, as provided in an embodiment of this specification.

[0077] Figure 3 This diagram illustrates a comparison of dense matrix rearrangement based on hierarchical awareness, as provided in the embodiments of this specification.

[0078] Figure 4 A flowchart illustrating the topology-aware two-stage communication aggregation and switching mechanism provided in the embodiments of this specification is shown.

[0079] Figure 5 This diagram illustrates the adaptive communication granularity decision space partitioning based on a performance model, as provided in the embodiments of this specification.

[0080] Figure 6 This diagram illustrates a module schematic of a communication and computing collaborative optimization device for heterogeneous intelligent computing networks, as provided in an embodiment of this specification. Detailed Implementation

[0081] The solution provided in this specification will now be described with reference to the accompanying drawings.

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be described below with reference to the accompanying drawings.

[0083] In the description of the embodiments of the present invention, the words "exemplary," "for example," or "for instance" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary," "for example," or "for instance" in the embodiments of the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Rather, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0084] In the description of the embodiments of the present invention, the term "and / or" is merely a description of the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, B existing alone, and A and B existing simultaneously. Furthermore, unless otherwise stated, the term "multiple" means two or more.

[0085] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and their variations all mean "including but not limited to," unless otherwise specifically emphasized.

[0086] To address the problems of existing technologies, this invention proposes a collaborative optimization method for communication and computation in heterogeneous intelligent computing networks. First, the global sparse matrix is ​​uniformly divided by row, and each processing unit statistically analyzes the distribution information of its local non-zero columns to construct a task access matrix reflecting the access requirements between the GPU and data blocks. Based on this, a hierarchical, multi-granularity load balancing allocation strategy is used. Through a two-stage gain-based iterative exchange algorithm (inter-node and intra-node), data blocks are rearranged and mapped to the optimal resident nodes and GPUs, thereby transforming a large number of cross-node remote accesses into high-speed local or intra-node NVLink accesses, effectively reducing the overhead of slow InfiniBand communication. Subsequently, a critical sparse column density function is derived based on the LogGP hardware model, and the column dimension and sparse column density of each data block are mapped online to a two-dimensional decision space. This adaptively selects between whole-block communication or fine-grained sparse communication modes, achieving a dynamic balance between protocol overhead and the amount of data transmitted. Furthermore, a multi-stream asynchronous collaborative execution mechanism is introduced, creating communication, exchange, and computation flows within nodes. Through demand aggregation, cross-node asynchronous RDMA prefetching, intra-node NVLink secondary distribution, and fine-grained event synchronization, a deep overlap between cross-node data acquisition, intra-node data exchange, and local matrix computation is achieved. Ultimately, distributed and efficient computation of sparse-dense matrix multiplication is accomplished. The entire method significantly reduces preprocessing overhead and cross-node redundant communication through lightweight data rearrangement, topology-aware pipelined parallelism, and adaptive communication granularity selection, achieving near-linear parallel scalability in heterogeneous intelligent computing networks.

[0087] Figure 1 This is a flowchart illustrating a communication and computational collaborative optimization method for heterogeneous intelligent computing networks, as shown in an embodiment of the present invention. It demonstrates the complete process from loading sparse matrix features, inputting them separately into a hierarchical sensing multi-granularity data load balancing allocation method module and an adaptive communication selector, and finally converging into a topology-aware pipeline to execute computation.

[0088] like Figure 1 As shown, the flowchart starts from the beginning node and then proceeds to the following sequential steps:

[0089] In step 1, the sparse matrix is ​​divided into blocks. Loaded into each processing unit (PE).

[0090] In step 2, the hierarchical sensing data allocation module processes the data to generate layout information, i.e., the physical distribution strategy, and passes it to subsequent steps. Based on the layout information, the dense matrix is ​​sliced. Loaded to each processing unit.

[0091] In step 3, the adaptive communication granularity selector outputs adaptive communication tags based on the input sparsity information, including two modes: block communication and fine-grained communication.

[0092] In step 5, the topology-aware two-stage pipeline receives layout information and adaptive communication tags, and performs the final parallel computation.

[0093] In the information flow diagram, layout information flows from step 2 to step 5, adaptive communication tags flow from step 3 to step 5, and sparsity information is input into step 3. The entire process achieves end-to-end collaborative optimization from data loading, layout allocation, strategy selection to pipeline execution.

[0094] Figure 2 This is a flowchart illustrating a communication and computing collaborative optimization method for heterogeneous intelligent computing networks, as shown in an embodiment of the present invention. Figure 2 As shown, it includes the following steps:

[0095] In step 210, system initialization and network resource feature extraction are performed.

[0096] In this embodiment of the invention, the global sparse matrix is ​​first uniformly divided by rows and allocated to each GPU processing unit.

[0097] It should be understood that a global sparse matrix refers to a sparse matrix representing the graph structure or node features during the training of a distributed graph neural network. In the multi-node, multi-GPU environment of heterogeneous intelligent computing networks, this matrix cannot be completely stored in a single processing unit. Therefore, it needs to be evenly divided by rows, with each GPU responsible for storing and processing only one continuous row interval, i.e., a row slice, which is one of the processing objects of the entire communication optimization method.

[0098] A dense matrix is ​​another matrix multiplied by a global sparse matrix, typically the learnable weight matrix or feature transformation matrix in a graph neural network. In the sparse-dense matrix multiplication operation C = A × B, A is the global sparse matrix, and B is the dense matrix, typically with dimensions N × K, where N is the same as the number of columns in the sparse matrix A, and K is the dimension of the output features. The vast majority of elements in the dense matrix B are non-zero, exhibiting a dense distribution. To facilitate distributed processing, it can be divided into several fixed-size data blocks, also called data blocks or sparse sub-blocks, each containing multiple consecutive columns.

[0099] In this embodiment of the invention, each non-zero element of the global sparse matrix A indicates the corresponding column of the dense matrix B that needs to be read. Therefore, the distribution of non-zero columns in the global sparse matrix A determines which data blocks in the dense matrix B each GPU needs to access and the frequency of access. The global sparse matrix A is divided into row slices distributed across the GPUs, while the dense matrix B is divided into fixed-size data blocks, which are distributed across the nodes and GPUs according to the load balancing results.

[0100] Therefore, in this embodiment of the invention, the access requirements of each GPU to each data block in the dense matrix B, i.e., the task access matrix, can be calculated based on the local row slices of the global sparse matrix A. Subsequently, the data blocks in the dense matrix B can be rearranged and allocated accordingly, and adaptive granularity and pipeline optimization can be employed when transmitting the data blocks of the dense matrix B. In summary, the global sparse matrix A is used to characterize the column access pattern of the dense matrix B, and the dense matrix B is the target data object being accessed.

[0101] Here, each GPU obtains a continuous row interval from the global sparse matrix and reads the corresponding sparse matrix row slice. Next, each GPU divides the column index range of this row slice into several fixed-size continuous sparse sub-blocks, for example, each sub-block contains 64, 128, or 256 columns. For each sparse sub-block, the GPU scans all non-zero elements in its local row slice, determines whether the column index of each non-zero element falls within the column range of that sub-block, and records all column indices of non-zero elements that appear at least once in that sub-block. Finally, the total number of columns with non-zero elements appearing at least once in each sparse sub-block is counted, forming a local non-zero column count vector.

[0102] Subsequently, all GPUs exchange their computed local non-zero column count vectors via a full collection communication operation. These collected local non-zero column count vectors are then aggregated to construct a task access matrix. In this matrix, row indices correspond to GPU numbers, column indices correspond to sparse sub-block numbers, and the values ​​of the elements represent the number of times the corresponding GPU needs to access the non-zero columns of the corresponding sparse sub-block.

[0103] Figure 3 This is a schematic diagram illustrating the comparison of dense matrix rearrangement based on hierarchical perception, as shown in an embodiment of the present invention. Figure 3 It consists of two subgraphs, labeled (a) traditional sequential allocation and (b) the rearrangement based on non-zero column layout of this invention. Each subgraph contains the following common elements:

[0104] Four GPU processing units, labeled GPU0, GPU1, GPU2, and GPU3, are represented by a rectangle. Eight dense matrix column blocks (data blocks), labeled Block0 to Block7, are represented by a small rectangle. Dashed boxes divide the GPUs and column blocks into two computing nodes: the left dashed box contains GPU0, GPU1, and part of the column blocks, labeled "Node 0"; the right dashed box contains GPU2, GPU3, and the remaining column blocks, labeled "Node 1". Access relationships between GPUs and column blocks are indicated by arrows: the arrow starts at the GPU and ends at the column block. Long arrows (across dashed boxes) indicate cross-node InfiniBand communication, and short arrows (within the same dashed box) indicate intra-node NVLink communication.

[0105] Figure 3 (a) In the traditional sequential allocation, the column blocks are arranged from left to right according to the original column index order: Block0, Block1, Block2, Block3, Block4, Block5, Block6, Block7. The initial allocation of column blocks adopts a simple round-robin method: Block0 and Block1 are located at node 0, Block2 and Block3 are located at node 1, Block4 and Block5 are again located at node 0, and Block6 and Block7 are located at node 1.

[0106] The access requirements of each GPU originate from the row slice distribution of the sparse matrix. In this example: GPU0 needs to access Block0, Block2, Block4, and Block6; GPU1 needs to access Block1, Block3, Block5, and Block7; GPU2 and GPU3 have similar cross-node access patterns.

[0107] Ultimately, the arrows intertwine densely. Long arrows, represented by gray dashed lines, point from GPU0 to Block 2 and Block 6, from GPU1 to Block 3 and Block 7, from GPU2 to Block 0 and Block 4, and from GPU3 to Block 1 and Block 5, across nodes. Statistics show 12 arrows crossing nodes, and only 4 short arrows within a single node.

[0108] Figure 3(b) In the reordering allocation of this invention, the column block order is rearranged according to the task access matrix constructed in the previous step. The rearranged order is: Block0, Block4, Block1, Block5, Block2, Block6, Block3, Block7. The reordered column block allocation strategy is determined by a two-stage heuristic algorithm: columns with high access demand are concentrated in the corresponding nodes. Specifically, Block0, Block4, Block1, and Block5, which are frequently accessed by GPU0 and GPU1, are clustered in node 0; Block2, Block6, Block3, and Block7, which are frequently accessed by GPU2 and GPU3, are clustered in node 1.

[0109] Ultimately, the vast majority of arrows become short arrows within nodes, represented by solid black lines, with only two long arrows spanning nodes remaining, such as the special case where GPU0 occasionally needs to access Block 7. The clustered column blocks are highlighted in the diagram with bold solid-line rectangles. It is clear that the reduction in communication requirements is evident when comparing the "traditional sequential allocation" with the "allocation rearranged based on non-zero column access requirements."

[0110] In step 220, multi-granularity load balancing allocation is generated.

[0111] In this embodiment of the invention, the input tasks include a task access matrix T and node topology information, i.e., the node number to which each GPU belongs. A two-stage divide-and-conquer strategy is adopted, first performing inter-node allocation, and then performing intra-node allocation.

[0112] It should be understood that a node refers to a physical computing server, also known as a computing node. In heterogeneous intelligent computing networks, multiple nodes are interconnected through the InfiniBand network, resulting in low communication bandwidth and high latency between nodes.

[0113] GPUs refer to multiple graphics processing units integrated within a node. A node contains multiple GPUs, which are interconnected at high speed via NVLink or NVSwitch, offering significantly higher bandwidth than cross-node communication.

[0114] A node is the basic unit of cross-node communication, while a GPU is the basic unit of intra-node computation and intra-node communication. The two-stage load balancing distribution in this embodiment of the invention is based on this hierarchy: the first stage determines which node the data block resides on, and the second stage determines which GPU within the node the data block resides on.

[0115] Specifically, the data blocks here refer to the data blocks obtained by dividing the dense matrix by columns, which are the sparse sub-blocks mentioned in the previous steps. Specifically, the dense matrix is ​​divided into several fixed-size continuous intervals based on the column index range, and all columns within each interval constitute a data block. The column indices in the task access matrix from the previous steps correspond to the data blocks. In this step, it determines which GPU on which node each data block should reside.

[0116] In step 221, node allocation is performed.

[0117] First, the task access matrix is ​​aggregated by node to obtain the access demand matrix of each node for all data blocks, i.e., sparse sub-blocks. The matrix element represents the sum of the access demands of all GPUs in that node for a certain data block.

[0118] A resident node is randomly assigned to each data block. The total cross-node communication cost under the current scheme is calculated as the sum of the access requirements multiplied by the cross-node unit cost for all nodes and their non-local data blocks.

[0119] Subsequently, multiple iterations were performed, with each iteration performing the following operations:

[0120] Examine each pair of nodes sequentially, such as node u and node v. For all data blocks currently assigned to these two nodes, evaluate the change in cross-node communication cost, i.e., the gain, if the data block were moved from its current node to another node. Find the data block with the largest positive gain. If the maximum gain exceeds a current threshold, perform a swap, move the data block to another node, and update the total cost. Each pair of nodes swaps at most one data block per iteration. The iteration terminates when the maximum gain is less than or equal to a preset value or when a preset number of iterations is reached, and the residing node for each data block is output.

[0121] In step 222, intra-node allocation is performed.

[0122] For each node, extract the set of data blocks belonging to that node, and the corresponding submatrix of the task access matrix T, which represents the access requirements of each GPU within that node for these data blocks. Within each node, run a gain-based iterative swapping algorithm similar to step 121 again. The difference is that the unit cost is replaced with the intra-node NVLink communication cost. The algorithm maps each data block to a specific GPU within that node.

[0123] Output the target GPU for each data block within this node.

[0124] Finally, the complete layout information for each data block is output, including the node and GPU where data block b should reside. This layout is broadcast to all GPUs, and each GPU loads the dense matrix slices according to the layout strategy.

[0125] Figure 4 This is a schematic diagram of a topology-aware two-stage communication aggregation and exchange mechanism as shown in an embodiment of the present invention.

[0126] Figure 4 The diagram consists of two sub-graphs, labeled "(a) Topology-aware communication in TileSPMM" and "(b) Topology-aware aggregation and switching communication," respectively. Each sub-graph shows two compute nodes, node 0 and node 1, each containing two GPUs, GPU0 and GPU1. Nodes are interconnected via an InfiniBand network (IB communication) and interconnected within nodes via NVLink high-speed interconnection.

[0127] Figure 4 In (a) Topology-aware communication (traditional method), both GPU0 and GPU1 in node 0 need to access the same dense matrix column block stored on GPU0 in node 1. Each GPU independently initiates a cross-node InfiniBand read request to obtain data from GPU0 in node 1.

[0128] The diagram shows two independent IB communication arrows pointing from GPU0 and GPU1 of node 0 to GPU0 of node 1, labeled "IB communication". This method results in the same data being transmitted repeatedly across nodes, causing redundant communication traffic and wasted bandwidth.

[0129] Figure 4 (b) In the topology-aware aggregation exchange communication of the present invention, firstly, a demand aggregation operation (marked "① aggregation") is performed within node 0. GPU0 and GPU1 within node 0 merge their respective access requests for the remote data block, and only one GPU (e.g., GPU0) initiates a cross-node request on behalf of the entire node. Subsequently, a cross-node InfiniBand communication (marked "IB communication") is performed: GPU0 of node 0 reads the required data block from GPU0 of node 1, and only one copy of the data is transmitted.

[0130] Finally, the intra-node NVLink exchange communication is performed (marked "②NVLink exchange communication"): GPU0 of node 0 distributes the received data blocks to GPU1 within node 0 through high-speed NVLink, completing data sharing.

[0131] The diagram clearly illustrates this two-stage process using a single IB communication arrow and an intra-node exchange arrow. Figure 4Comparison: (a) involves two cross-node communications with redundant data transmission; (b) reduces the number of cross-node communications to one, with intra-node switching utilizing high-speed NVLink instead of slow InfiniBand, significantly reducing cross-node traffic and communication latency. This intuitively illustrates the advantages of the "aggregation-switching" two-stage mechanism in this invention compared to traditional non-topology-aware methods: by aggregating intra-node demands, multiple redundant cross-node communications are merged into one, and then distributed through high-speed intra-node links, effectively utilizing hardware topology characteristics and reducing InfiniBand bandwidth usage.

[0132] In step 230, adaptive communication strategy tagging is performed.

[0133] In this embodiment of the invention, the communication mode of each sparse sub-block is determined online by using the sparse column density of each data block, i.e., the sparse sub-block, and the column dimension of the dense matrix obtained in step 110.

[0134] It should be understood that sparse column density refers to the ratio of the number of columns containing non-zero elements in a sparse sub-block to the total number of columns in the sub-block. The column dimension of a dense matrix... This refers to the number of columns contained in each sparse sub-block. The communication efficiency of whole-block transmission and fine-grained sparse transmission differs significantly under different sparsity rates and column dimensions. To achieve a dynamic balance between protocol fixed overhead and transmitted data volume, a more optimal communication mode needs to be adaptively selected for each sparse sub-block.

[0135] In this embodiment of the invention, sparsity information and column dimension information of the dense matrix can be input in parallel into an adaptive communication selector. This selector incorporates a critical sparse column density function derived from the LogGP hardware network model, mapping each specific transmission task online to a two-dimensional decision space composed of the matrix column dimension and sparse column density.

[0136] Based on the region where the mapping point is located, the selector marks the communication strategy for each network transmission online. If the mapping point is located in a high-density or low-dimensional region, it is marked as using bulk communication with lower protocol overhead; if the mapping point is located in both low-density and high-dimensional regions, it is marked as using fine-grained sparse communication with more efficient data transfer.

[0137] Specifically, the selector incorporates a critical sparse column density function derived from the LogGP hardware model. Output density threshold .in To obtain the column dimensions of a dense matrix, retrieve the column dimensions of the data block. and sparse column density Calculate the threshold Based on the threshold, a block-wide communication mode is selected, which transmits all columns of the entire data block at once. Alternatively, a fine-grained sparse communication mode is selected, which transmits only the columns that actually contain non-zero columns. After the decision is made, each data block is labeled with a communication mode label, which includes either the block-wide communication mode or the fine-grained sparse communication mode.

[0138] Figure 5 This is an embodiment of the present invention illustrating an adaptive communication granularity decision space partitioning diagram based on a performance model. A two-dimensional decision space is constructed with column dimension K as the horizontal axis and log2 scaling and sparse column density ρ as the vertical axis. Critical density curves for intra-node NVLink links and inter-node InfiniBand links are given respectively to guide the adaptive selection of communication modes.

[0139] The two columns of data correspond to the critical NVLink density within the node: when the sparse column density is higher than this curve, block communication is used; when it is lower, fine-grained sparse communication is used. Its value decreases rapidly as K increases, for example, it is about 0.00 when K=0, 0.02 when K=2, 0.12 when K=4, 0.80 when K=6, and then tends to 0.90.

[0140] Inter-node IB critical density: Also used for decision-making, but the value is generally higher, for example 0.08 when K=2, 0.35 when K=4, 0.85 when K=6, and then stabilizes at around 0.90, reflecting that the fine-grained advantage of cross-node communication is more easily manifested.

[0141] The whole-block communication region is located in the lower right of the curve, in an area with small column dimensions or high sparsity. Transmitting the entire block in this region results in lower protocol overhead and better performance. The fine-grained sparse communication region is located in the upper left of the curve, in an area with large column dimensions and low sparsity. Transmitting only non-zero columns in this region significantly reduces data volume, outperforming packet splitting overhead. The two curves represent intra-node and inter-node communication scenarios, respectively. The system selects the corresponding critical density based on the actual communication link, illustrating the theoretical basis of the adaptive communication granularity selector in this invention: by obtaining the critical density function through offline fitting, the column dimensions and sparsity density are mapped online to the decision space, automatically selecting the optimal transmission mode and achieving a quantitative balance of communication overhead.

[0142] In step 240, multi-stream asynchronous collaborative execution is performed.

[0143] In this embodiment of the invention, a dual buffering mechanism can be used, with independent cross-node buffers and intra-node buffers combined with device multi-stream concurrency technology, to start a topology-aware two-stage communication pipeline to efficiently execute underlying operators.

[0144] During pipeline operation, the background asynchronous communication stream continues to work, aggregating access requests for remote data within the same node according to the strategy of the previous step, and initiating cross-node asynchronous communication to fetch data to the buffer area. Once node-level synchronization is triggered and data transfer is completed, the high-speed exchange stream within the node further distributes the data to other GPUs within the node that need the data through a high-speed network.

[0145] Meanwhile, after confirming that the required data is ready, the foreground computation stream independently performs local sparse-dense matrix multiplication calculations. The above three data streams are used for dependency wake-up and synchronization through a fine-grained event lock (Require / Release) mechanism, thereby achieving deep overlap between cross-node data acquisition, local data exchange, and local operator computation until the computation task of this layer of the neural network is successfully completed.

[0146] It should be understood that in traditional distributed SpMM execution, each GPU directly initiates cross-node access, resulting in a large amount of redundant cross-node communication, and computation and communication are executed serially, leading to low efficiency. This invention significantly reduces cross-node traffic and hides communication latency through intra-node demand aggregation, cross-node asynchronous RDMA prefetching, intra-node NVLink secondary distribution, and a three-stream parallel mechanism.

[0147] In summary, this invention aims to address the problems of high communication overhead and limited scalability of basic operators in large models under heterogeneous intelligent computing networks. It proposes a basic operator adaptation and optimization framework that combines the characteristics of heterogeneous computing devices, including multi-granularity load balancing partitioning, topology-aware pipeline, and communication adaptive strategy.

[0148] This invention provides a hierarchical, multi-granularity data load balancing allocation method that alleviates the problem of high preprocessing overhead in existing large-scale model basic operators. It eliminates the need for expensive global graph preprocessing by lightweight rearranging the physical storage distribution of dense matrices, converting some remote communication into local hits. The system first divides the global sparse matrix by rows and assigns it to each GPU processor, then statistically analyzes the distribution of non-zero columns to construct a task access matrix reflecting the "processor-data block" access requirements. Subsequently, a multi-level divide-and-conquer strategy is employed to decompose the complex allocation problem into two levels: the first stage utilizes a gain-based iterative exchange heuristic algorithm to minimize cross-node InfiniBand communication costs and determine the allocation of data blocks among nodes; the second stage runs the algorithm independently within each node, mapping data blocks to a single GPU to reduce NVLink communication costs. This heuristic algorithm has a short execution time (averaging approximately 0.191 milliseconds, with a maximum of no more than 0.5 milliseconds), significantly reducing startup costs to accommodate large static models while also effectively supporting low-latency tasks such as dynamic graphs and real-time inference, thus meeting the requirements for multi-granularity task load balancing.

[0149] This invention provides a topology-aware two-stage communication pipeline method tailored to the characteristics of heterogeneous computing devices. Addressing the bandwidth differences between slow InfiniBand between nodes and fast NVLink within nodes, it designs a multi-stream asynchronous collaborative mechanism of "aggregation-exchange" to more fully overlap computation and communication. During execution, the system first aggregates access requests from multiple GPUs within the same node for the same remote data block. A single GPU initiates cross-node asynchronous prefetching in the background using CUDA's multi-stream execution mechanism, thereby reducing redundant cross-node fetching. Subsequently, the acquired data, under multi-stream control, is redistributed among the GPUs within the local node via high-speed NVLink; simultaneously, the computation stream handles independent local sparse-dense multiplication calculations in the foreground. Combining device-to-device (D2D) data copying and a fine-grained event synchronization mechanism (Require / Release), this method achieves deep overlap between cross-node data acquisition, intra-node data exchange, and local matrix computation. Theoretically, on an 8-GPU node, it can reduce cross-node communication time by approximately 87.5%, thus more effectively adapting to the optimization of basic operators under the characteristics of heterogeneous computing devices.

[0150] This invention provides an adaptive communication granularity selection method based on performance evaluation, which can improve the problem of fixed communication transmission modes in distributed training of large models. By constructing a quantized decision mechanism based on the LogGP hardware network model, it dynamically selects the optimal communication mode for different network levels. During system operation, this method analyzes the latency, bandwidth, and message size characteristics of the underlying GPU-Direct RDMA and defines a "critical sparse column density function" based on the column dimension of the dense matrix. The system maps the dense matrix dimension and the non-zero column density of the sparse sub-blocks of the current task online to a two-dimensional decision space for evaluation: if the mapping point is in a high-density or low-dimensional region, the system will choose to transmit the entire block to maintain low software protocol overhead; if the mapping point is in both low-density and high-dimensional regions, the system switches to fine-grained communication to reduce the actual transmission volume. This method achieves a good balance between protocol overhead and transmission volume, which helps to further improve the parallel scalability of large-scale model training in computing power networks.

[0151] In summary, this invention alleviates the parallel scalability bottleneck of large-scale model training in computing networks. In real-world gigabit-level heterogeneous network interconnected multi-node, multi-GPU clusters, it achieves an average end-to-end speedup of approximately 4.0 to 5.1 times compared to existing baseline algorithms in a multi-node environment. Secondly, this invention aligns with the characteristics of heterogeneous intelligent computing, addressing the pain points of increased data sharing and cross-node communication overhead. Its topology-aware two-stage pipelined distribution mechanism effectively mitigates the growth trend of communication costs during horizontal scaling of computing nodes, demonstrating excellent strong and weak scalability. Finally, this invention improves task throughput through lightweight scheduling. Due to the extremely low algorithm preprocessing time (maximum less than 0.5 milliseconds), the system significantly reduces preprocessing startup costs, enabling better support for rapid response to short-term, dynamic large-scale model inference or function computation tasks, thus contributing to improved overall utilization of computing network resources.

[0152] Figure 6 This is a schematic diagram of a communication and computing collaborative optimization device for heterogeneous intelligent computing networks, as shown in an embodiment of the present invention. Figure 6 As shown, it includes:

[0153] The first processing module is used to generate a task access matrix based on the global sparse matrix. The task access matrix is ​​used to characterize the access requirements of each processing unit to the data blocks of the dense matrix. The dense matrix refers to the weight matrix that is multiplied by the global sparse matrix and is divided into multiple data blocks by columns.

[0154] The second processing module is used to determine the node where the data block resides and the processing unit within the node based on the task access matrix and node topology information, wherein the node topology information represents the affiliation relationship between the processing unit and the node it resides in.

[0155] The third processing module is used to determine the communication transmission mode of the data block based on the sparse density and column dimension of the data block. The communication transmission mode includes whole block transmission and fine-grained sparse transmission.

[0156] The fourth processing module is used to perform multi-stream concurrency to complete the computation task of the current neural network layer. The asynchronous communication stream aggregates the access requirements of remote data blocks within the same node according to the communication transmission mode of the data blocks, and initiates cross-node asynchronous communication to cache the remote data blocks in the node-level buffer; the high-speed exchange stream within the node distributes the data blocks to other processing units within the node; and the foreground computation stream performs local computation after the required data is ready.

[0157] In some embodiments, the first processing module is configured to divide the global sparse matrix into multiple row slices, with each row slice assigned to a processing unit.

[0158] Each processing unit divides the row slice into multiple data blocks according to the column index range of the row slice, counts the number of columns in the data blocks that have at least one non-zero element, and obtains a local non-zero column count vector.

[0159] All processing units exchange their respective local non-zero column count vectors via full collection communication and aggregate them to construct the task access matrix;

[0160] In this context, the row index of the task access matrix corresponds to the processing unit number, the column index corresponds to the data block number, and the value of each element in the task access matrix represents the number of times the corresponding processing unit needs to access the non-zero column in the corresponding data block.

[0161] In some embodiments, the second processing module is configured to employ a two-stage divide-and-conquer strategy:

[0162] In the first stage, based on the node-level aggregation results of the task access matrix, the node where each data block resides is determined;

[0163] In the second stage, for each node, based on the access requirements of each processing unit within the node for data blocks, the processing unit where each data block resides is determined.

[0164] In some embodiments, the second processing module is used to aggregate the task access matrix by node to obtain an access requirement matrix for each node to all data blocks.

[0165] An initial resident node is randomly assigned to each data block, and the initial total cross-node communication cost is calculated. The total cross-node communication cost is a value calculated based on the access requirements of all nodes to their non-local data blocks and the cross-node unit cost.

[0166] Examine each pair of nodes in turn, and for each data block currently assigned to these two nodes, evaluate the change in cross-node communication cost that would result from moving the data block from its current node to the other node.

[0167] Select the data block with the largest positive change in cross-node communication cost. If the largest change in cross-node communication cost is greater than the current threshold, move the data block to another node and update the total cross-node communication cost.

[0168] When the maximum change in cross-node communication cost is less than the first threshold or the number of iterations reaches the first preset number of iterations, the iteration is terminated, and the resident node of each data block is output.

[0169] In some embodiments, the second processing module is configured to extract, for each node, a set of data blocks belonging to the node, and the access requirements of each processing unit within the node for the data blocks;

[0170] An initial resident processing unit is randomly assigned to each data block, and the initial total intra-node communication cost is calculated. The total intra-node communication cost is determined based on the communication cost of the high-speed link within the node.

[0171] Examine each pair of processing units in turn, and for each data block currently assigned to these two processing units, evaluate the change in intra-node communication cost that would result from moving the data block from its current processing unit to the other processing unit.

[0172] Select the data block with the largest positive change in intra-node communication cost. If the largest change in intra-node communication cost is greater than the second threshold, move the data block to another processing unit and update the total intra-node communication cost.

[0173] When the change in the largest intra-node communication cost is less than the second threshold or the number of iterations reaches the second preset number of iterations, the iteration is terminated, and the processing unit where each data block resides in the current node is output.

[0174] In some embodiments, the third processing module is used to construct a critical sparse column density function based on the hardware network model, and to map the transmission task of each data block to a two-dimensional decision space with its column dimension and sparse density as coordinates to obtain the mapping point.

[0175] The communication transmission mode is determined based on the position of the mapping point in the two-dimensional decision space:

[0176] If the mapping point is located in a high-density region or a low-dimensional region, the block communication mode is adopted.

[0177] If the mapping point is located in a low-density and high-dimensional region, a fine-grained sparse communication mode is adopted.

[0178] In some embodiments, the fourth processing module is configured to, during pipeline operation, continuously run the background asynchronous communication stream, aggregate the access requests of each processing unit within the same node to remote data blocks according to the communication mode label, and initiate cross-node asynchronous communication to read the remote data blocks into the node-level buffer.

[0179] Once the node-level synchronization conditions are met and the remote data block transfer is completed, the high-speed switching stream within the node distributes the remote data block to other processing units within the node that require it through the high-speed interconnection network within the node.

[0180] After confirming that all the required data is ready, the foreground computing stream independently performs local sparse-dense matrix multiplication calculations.

[0181] The asynchronous communication stream, the high-speed exchange stream within the node, and the foreground computing stream are synchronized and woken up by a fine-grained event lock mechanism until the computing task of the current neural network layer is completed.

[0182] According to another embodiment, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, it implements a communication and computing collaborative optimization device for heterogeneous intelligent computing networks as described above.

[0183] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a communication and computing collaborative optimization device for heterogeneous intelligent computing networks.

[0184] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.

[0185] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for collaborative optimization of communication and computing in heterogeneous intelligent computing networks, characterized in that, include: Based on the global sparse matrix, a task access matrix is ​​generated. The task access matrix is ​​used to characterize the access requirements of the processing unit to data blocks of the dense matrix. The dense matrix refers to the weight matrix that is multiplied with the global sparse matrix and is divided into multiple data blocks by columns. Based on the task access matrix and node topology information, the node where the data block resides and the processing unit within the node are determined, wherein the node topology information represents the affiliation relationship between the processing unit and the node it resides in; Based on the sparse density and column dimension of the data block, the communication transmission mode of the data block is determined, and the communication transmission mode includes whole block transmission and fine-grained sparse transmission. Multi-stream concurrency is executed to complete the computational task of the current neural network layer. The asynchronous communication stream aggregates the access requirements of remote data blocks within the same node according to the communication transmission mode of the data blocks, and initiates cross-node asynchronous communication to cache the remote data blocks in the node-level buffer; the high-speed exchange stream within the node distributes the data blocks to other processing units within the node; and the foreground computation stream performs local computation after the required data is ready.

2. The method according to claim 1, characterized in that, The step of generating the task access matrix based on the global sparse matrix includes: The global sparse matrix is ​​divided into multiple row slices, and each row slice is assigned to a processing unit. Each processing unit divides the row slice into multiple data blocks according to the column index range of the row slice, counts the number of columns in the data blocks that have at least one non-zero element, and obtains a local non-zero column count vector. All processing units exchange their respective local non-zero column count vectors via full collection communication and aggregate them to construct the task access matrix; In this context, the row index of the task access matrix corresponds to the processing unit number, the column index corresponds to the data block number, and the value of each element in the task access matrix represents the number of times the corresponding processing unit needs to access the non-zero column in the corresponding data block.

3. The method according to claim 1, characterized in that, The step of determining the node where each data block resides and the processing unit within that node based on the task access matrix and node topology information includes: A two-stage divide-and-conquer strategy is adopted: In the first stage, based on the node-level aggregation results of the task access matrix, the node where each data block resides is determined; In the second stage, for each node, based on the access requirements of each processing unit within the node for data blocks, the processing unit where each data block resides is determined.

4. The method according to claim 3, characterized in that, The node-level aggregation result based on the task access matrix determines the node where each data block resides, including: Aggregate the task access matrix by node to obtain the access requirement matrix of each node for all data blocks; An initial resident node is randomly assigned to each data block, and the initial total cross-node communication cost is calculated. The total cross-node communication cost is a value calculated based on the access requirements of all nodes to their non-local data blocks and the cross-node unit cost. Examine each pair of nodes in turn, and for each data block currently assigned to these two nodes, evaluate the change in cross-node communication cost that would result from moving the data block from its current node to the other node. Select the data block with the largest positive change in cross-node communication cost. If the largest change in cross-node communication cost is greater than the current threshold, move the data block to another node and update the total cross-node communication cost. When the maximum change in cross-node communication cost is less than the first threshold or the number of iterations reaches the first preset number of iterations, the iteration is terminated, and the resident node of each data block is output.

5. The method according to claim 3, characterized in that, For each node, based on the access requirements of each processing unit within the node for data blocks, the process unit residing for each data block is determined, including: For each node, extract the set of data blocks belonging to that node, as well as the access requirements of each processing unit within that node for those data blocks; An initial resident processing unit is randomly assigned to each data block, and the initial total intra-node communication cost is calculated. The total intra-node communication cost is determined based on the communication cost of the high-speed link within the node. Examine each pair of processing units in turn, and for each data block currently assigned to these two processing units, evaluate the change in intra-node communication cost that would result from moving the data block from its current processing unit to the other processing unit. Select the data block with the largest positive change in intra-node communication cost. If the largest change in intra-node communication cost is greater than the second threshold, move the data block to another processing unit and update the total intra-node communication cost. When the change in the largest intra-node communication cost is less than the second threshold or the number of iterations reaches the second preset number of iterations, the iteration is terminated, and the processing unit where each data block resides in the current node is output.

6. The method according to claim 1, characterized in that, Determining the communication transmission mode of the data block based on its sparse density and column dimension includes: Based on the hardware network model, a critical sparse column density function is constructed, and the transmission task of each data block is mapped to a two-dimensional decision space with its column dimension and sparse density as coordinates to obtain the mapping point. The communication transmission mode is determined based on the position of the mapping point in the two-dimensional decision space: If the mapping point is located in a high-density region or a low-dimensional region, the block communication mode is adopted. If the mapping point is located in a low-density and high-dimensional region, a fine-grained sparse communication mode is adopted.

7. The method according to claim 1, characterized in that, The execution of multi-stream concurrency to complete the computational task of the current neural network layer includes: During pipeline operation, the background asynchronous communication stream continues to run, aggregating the access requests of each processing unit within the same node to remote data blocks according to the communication mode label, and initiating cross-node asynchronous communication to read the remote data blocks into the node-level buffer. Once the node-level synchronization conditions are met and the remote data block transfer is completed, the high-speed switching stream within the node distributes the remote data block to other processing units within the node that require it through the high-speed interconnection network within the node. After confirming that all the required data is ready, the foreground computing stream independently performs local sparse-dense matrix multiplication calculations. The asynchronous communication stream, the high-speed exchange stream within the node, and the foreground computing stream are synchronized and woken up by a fine-grained event lock mechanism until the computing task of the current neural network layer is completed.

8. A communication and computing collaborative optimization device for heterogeneous intelligent computing networks, characterized in that, include: The first processing module is used to generate a task access matrix based on the global sparse matrix. The task access matrix is ​​used to characterize the access requirements of each processing unit to the data blocks of the dense matrix. The dense matrix refers to the weight matrix that is multiplied by the global sparse matrix and is divided into multiple data blocks by columns. The second processing module is used to determine the node where the data block resides and the processing unit within the node based on the task access matrix and node topology information, wherein the node topology information represents the affiliation relationship between the processing unit and the node it resides in. The third processing module is used to determine the communication transmission mode of the data block based on the sparse density and column dimension of the data block. The communication transmission mode includes whole block transmission and fine-grained sparse transmission. The fourth processing module is used to perform multi-stream concurrency to complete the computation task of the current neural network layer. The asynchronous communication stream aggregates the access requirements of remote data blocks within the same node according to the communication transmission mode of the data blocks, and initiates cross-node asynchronous communication to cache the remote data blocks in the node-level buffer; the high-speed exchange stream within the node distributes the data blocks to other processing units within the node; and the foreground computation stream performs local computation after the required data is ready.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the communication and computing collaborative optimization method for heterogeneous intelligent computing networks as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the communication and computing collaborative optimization method for heterogeneous intelligent computing networks as described in any one of claims 1 to 7.

Images