Sparse matrix adaptive storage and parallel computing method based on vertex cut and two-stage compression

Abstract

This invention belongs to the field of sparse matrix storage and distributed parallel computing technology. It discloses an adaptive storage and parallel computing method for sparse matrices based on vertex cutting and two-stage compression. The method includes: initializing a cross-linked list in memory for dynamic assembly of sparse matrix elements; reducing the dimensionality of the dynamically assembled cross-linked list data structure to a one-dimensional continuous compressed sparse row format using a two-stage traversal algorithm; constructing a heuristic diagnostic model based on row non-zero variance and diagonal occupancy to achieve adaptive matrix storage; extracting the pure graph topology after removing self-loop elements on the main diagonal of the sparse matrix, and performing multi-way recursive graph segmentation based on vertex cutting on the pure graph topology to generate a distributed mapping array; and constructing a communication scheduling table based on the distributed mapping array to achieve deep overlap between non-blocking data exchange and local matrix computation. This invention significantly improves the storage compression rate and parallel computing scalability of large sparse matrices.

Description

Technical Field

[0001] This invention belongs to the field of sparse matrix storage and distributed parallel computing technology, and more specifically, relates to a sparse matrix adaptive storage and parallel computing method based on vertex cutting and two-stage compression. Background Technology

[0002] In the fields of scientific computing and engineering simulation (such as nonlinear finite element analysis, computational fluid dynamics, and state estimation of large power systems), the core computational process usually boils down to solving ultra-large-scale sparse linear algebraic equations. To reduce storage overhead and improve the computational efficiency of sparse matrix-vector multiplication (SpMV), it is usually necessary to compress the storage of sparse matrices and parallelize the task allocation.

[0003] Chinese invention patent application publication CN118484135A discloses a "sparse matrix storage method and vector multiplication method." This method obtains multiple matrix blocks from a sparse matrix based on a predetermined length and width, classifies them according to the number of non-zero elements in each block, and ultimately uses Compressed Sparse Row (CSR) format to store the data contiguously in memory. Furthermore, Chinese invention patent application publication CN120744294A also discloses a "sparse matrix processing method based on a block-based strategy." This method reads a static sparse matrix file, obtains matrices stored in CSC and CSR formats based on the file, and performs block processing and multiplication operations on the matrices according to a pre-set size. In distributed parallel computing, existing known technologies (such as conventional MPI parallel programs) typically map sparse matrices directly to a computational topology graph, then call standard graph partitioning algorithms (such as the conventional METIS algorithm) to divide the computational task among various computing nodes, and exchange boundary data between nodes through a synchronous blocking mechanism.

[0004] The existing sparse matrix compression storage and parallel computing techniques mentioned above have the following significant drawbacks in practical engineering applications: First, memory overhead is extremely high in dynamic assembly scenarios. In the early assembly stages of solving nonlinear problems, matrix elements typically arrive out of order and accumulate at high frequencies. The aforementioned patents, such as CN118484135A and CN120744294A, all rely on post-compression or block partitioning of already fully generated and statically existing sparse matrices. If this strategy of directly constructing CSR / CSC formats or pre-blocking is applied to dynamic assembly scenarios, when faced with the dynamic insertion and accumulation of new elements, the CSR format's strict requirement for contiguous memory addresses forces the system to perform extremely frequent memory reallocations and massive data migrations. This not only severely consumes system resources but also becomes a core performance bottleneck in the early stages of computation.

[0005] Second, graph segmentation evaluation distortion leads to uneven communication load. In known techniques for mapping sparse matrices to topological graphs and partitioning tasks, traditional methods typically map all non-zero elements to graph edges indiscriminately. However, this mapping ignores the essential physical difference between diagonal and off-diagonal elements: diagonal elements only represent the self-loop weights of computation nodes and never generate cross-node communication overhead. Indiscriminately retaining diagonal elements causes severe data pollution and bias in the graph segmentation engine when evaluating "edge-cut," resulting in extremely uneven communication load across nodes.

[0006] Third, the synchronous blocking mode makes it difficult to mask network latency. In existing distributed SpMV computing, due to the irregularity of non-zero element distribution, cross-node access to non-local dependent data is unavoidable. Existing solutions generally adopt a synchronous blocking mode of "communication first, computation later". In this mode, the computing core must be completely suspended after initiating a data request, remaining idle while waiting for network data transmission to complete. This mechanism exposes the underlying network's data exchange latency entirely to the computation timeline, making it impossible to effectively mask, thus significantly limiting the overall system's computational throughput and parallel scalability. Summary of the Invention

[0007] To address the aforementioned deficiencies or improvement needs of existing technologies, this invention provides a sparse matrix adaptive storage and parallel computing method based on vertex cutting and two-stage compression, aiming to reduce memory overhead when facing dynamic assembly scenarios.

[0008] To achieve the above objectives, this invention provides a sparse matrix adaptive storage method based on two-stage compression, comprising: Initialize a cross-linked list in memory to dynamically assemble the currently generated sparse matrix; Assign a length of to each row of the dynamically assembled cross-linked list. A temporary statistics array is initialized to zero; each row of the cross-linked list is traversed sequentially, and for each valid node encountered, the non-zero element value represented by the valid node is added to the current row number, and this sum is used as the value at the corresponding position in the temporary statistics array of the current row; the count of non-zero elements in each row is calculated based on the temporary statistics array of each row. This generates an array of row pointers in compressed sparse line (CSR) format; where, The number of columns in the sparse matrix is ​​given by the number of columns, and the effective nodes are those that represent the non-zero elements in the sparse matrix. The row pointer array is used as the dynamic insertion offset pointer for each row to determine the initial insertion offset for each row. Then, each row of the cross-linked list is traversed sequentially. When the current valid node of the current row is reached, the column index and value of the current valid node are written into the corresponding positions of the column index array and the value array according to the insertion offset of the current row. Then, the insertion offset of the current row is incremented by one and used as the insertion offset of the next valid node. After the traversal is completed, a matrix in CSR format is obtained. Calculate the row nonzero variance of the matrix in the CSR format. and the diagonal occupancy of the matrix in CSR format. ;judge or If the value exceeds a preset threshold, the CSR format matrix is ​​converted to a DIA format matrix so that the sparse matrix is ​​stored in the DIA format; otherwise, the sparse matrix is ​​stored in the CSR format.

[0009] Furthermore, during the process of initializing the cross-linked list in memory to dynamically assemble the currently generated sparse matrix, when inserting or updating the element (i, j, value) with the value 'value' at the i-th row and j-th column of the sparse matrix in the current node of the cross-linked list, the following operation is performed: In the row list of row i, search for a node with column index j in column index order. If found, it means that a non-zero element (i, j, old_val) already exists at that position, so perform an in-place update: update the value of the current node to the sum of value and old_val. If not found, it means that the current position is zero, so dynamically allocate a new node newNode with row and column index (i, j) and initial value value. Insert newNode into the row list of row i after the largest column index node with column index less than j, and into the column list of column j after the largest row index node with row index less than i. The first and second pointers of newNode point to its successor in the row and column lists, respectively. The first pointer is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same row as the current node during dynamic assembly. The second pointer is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same column as the current node during dynamic assembly.

[0010] Furthermore, the row non-zero variance for:

[0011] in, Let be the number of rows in the sparse matrix. This represents the average number of non-zero elements in each row of the cross-linked list.

[0012] Furthermore, diagonal occupancy rate The calculation method is as follows: The diagonal offset is determined based on the difference between the column index and the row index in the CSR format matrix; based on the diagonal offset, the CSR format matrix is ​​scanned to identify all diagonal sets containing non-zero elements; For each diagonal in the set of diagonals, calculate the total number of theoretically valid positions for it within the matrix dimension. And count the total number of non-zero elements actually filled on all diagonals. Generate diagonal occupancy rate : .

[0013] This invention also provides a parallel computation method for sparse matrices based on vertex cutting, comprising: S1. Based on the column pointer array in the compressed matrix column format, extract the interval from the starting memory offset corresponding to the current column to the ending memory offset corresponding to the next column as the loop traversal interval for the non-zero elements of the current column; wherein, the compressed matrix is ​​a CSR format matrix or a DIA format matrix stored using any of the above-described sparse matrix adaptive storage methods. S2. During the traversal of the current non-zero element, based on the current iteration step size, read the row index to which the current non-zero element belongs from the corresponding row index array; determine whether the row index is equal to the current column index. If so, determine that the current non-zero element is located on the main diagonal of the compressed matrix and terminate the participation of the current non-zero element in subsequent parallel computation; if not, use the row index as the memory addressing subscript and perform an incrementing accumulation operation on the non-zero element count value corresponding to the current row in the pre-initialized row feature statistics array; wherein, the memory addressing subscript is used to determine the non-zero element count value corresponding to the current row in the row feature statistics array. S3. Take the next non-zero element as the current non-zero element, repeat S2, until all non-zero elements in the compression matrix have been traversed, and obtain the filtered and purified graph topology after removing the non-zero elements located on the main diagonal of the compression matrix, which is determined by the count value of the remaining non-zero elements in the compression matrix. S4. Input the filtered and purified graph topology into the multi-path recursive graph partitioning engine, optimize load balancing by minimizing the amount of communication edge cutting across partitions, and output a distributed mapping array with a length equal to the number of global vertices of the filtered and purified graph topology; wherein, the multi-path recursive graph partitioning engine adopts a heuristic strategy based on vertex cutting. S5. Perform parallel sparse matrix-vector multiplication operations based on the distributed mapping array.

[0014] Furthermore, performing parallel sparse matrix-vector multiplication operations based on the distributed mapping array includes: Each element in the distributed mapping array is taken as a node, and each node obtains its own node identifier in the distributed mapping array. Each node traverses the global vertices of the filtered and purified graph topology to obtain a local vertex set belonging to the current node; for each local vertex in the local vertex set, it searches for all its neighboring vertices in the adjacency array of the filtered and purified graph topology, and queries the self-node identifier of the neighboring vertex in the distributed mapping array. Determine whether the self-node identifier of the adjacent vertex is equal to the self-node identifier of the current node. If they are not equal, add the adjacent vertex to the set of non-local dependent vertices to establish a communication scheduling table representing the mapping relationship between the current node and its corresponding set of non-local dependent vertices, and execute the following communication branch pipeline; if they are not equal, directly execute the following computation branch pipeline. Communication branch pipeline: Based on the communication scheduling table, the underlying asynchronous receiving interface is called to obtain the non-local dependent vertex data sent by the non-local dependent vertex of the current node, and the asynchronous sending interface is called to distribute the non-local dependent vertex data outward, so as to realize asynchronous sending and receiving; Calculate the branch pipeline: The current node and its corresponding adjacent vertices directly perform sparse matrix-vector multiplication. After the operation is completed, the synchronous blocking interface is called. When the asynchronous sending and receiving tasks of the communication branch pipeline have been transmitted in the network layer, the received non-local dependent vertex data is used to perform sparse matrix-vector multiplication with the current node.

[0015] The present invention also provides a sparse matrix adaptive storage system based on two-stage compression, including a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the sparse matrix adaptive storage method described above.

[0016] The present invention also provides a sparse matrix parallel computing system based on vertex cutting, including a computer-readable storage medium and a processor; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the sparse matrix parallel computing method described above.

[0017] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the sparse matrix adaptive storage method as described above, or / and implements the sparse matrix parallel computation method as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when the computer program is run on a computer, causes the computer to execute the sparse matrix adaptive storage method described above, or / and to execute the sparse matrix parallel computing method described above.

[0019] In summary, the above-described technical solutions conceived in this invention can achieve the following beneficial effects: (1) This invention effectively alleviates the memory overhead of dynamic assembly: by introducing a front-end cross-linked list data structure and a two-stage dimensionality reduction transformation algorithm, it avoids frequent memory reallocation and massive data migration during the assembly of nonlinear stiffness matrices, realizes efficient and compact reconstruction to one-dimensional continuous memory, and reduces memory overhead when facing dynamic assembly scenarios.

[0020] (2) The present invention realizes intelligent adaptive matching of storage format: relying on the row non-zero variance and diagonal occupancy evaluation model, it can dynamically switch between the universal Compressed Sparse Row (CSR) format and the vectorization friendly Diagonal (DIA) format, thereby achieving a better balance between memory continuity and computing throughput.

[0021] (3) This invention improves the scalability of distributed parallel computing: Before graph partitioning, elements on the main diagonal are forcibly removed to construct a pure graph topology without self-loops, and non-local dependent vertices are accurately extracted by combining a static multi-way recursive partitioning algorithm. This mechanism avoids expensive runtime dynamic load negotiation by generating a deterministic communication schedule table, enabling deep overlap between underlying non-blocking communication and local matrix calculation, effectively alleviating the cross-node memory access latency caused by indirect addressing.

[0022] Overall, through innovative data structure transformation, heuristic format adaptation, and deep communication-computation overlap mechanism, the storage compression rate and parallel computing scalability of large sparse matrices are significantly improved. Attached Figure Description

[0023] Figure 1 The main flowchart of a sparse matrix adaptive storage and parallel computing method based on vertex cutting and two-stage compression provided in an embodiment of the present invention is shown.

[0024] Figure 2 This is a schematic diagram of an adaptive storage and parallel computing method for sparse matrices based on vertex cutting and two-stage compression, provided in an embodiment of the present invention.

[0025] Figure 3 The diagram illustrates the principle of dynamic assembly and two-stage dimensionality reduction transformation of the cross-linked list provided in this embodiment of the invention.

[0026] Figure 4 A flowchart of heuristic feature diagnosis and adaptive format decision-making provided for embodiments of the present invention.

[0027] Figure 5 This is a schematic diagram of self-loop-free graph topology extraction and multi-path recursive static segmentation provided in an embodiment of the present invention.

[0028] Figure 6 This is a diagram of a non-local dependency vertex discovery and non-blocking communication scheduling architecture provided in an embodiment of the present invention.

[0029] Figure 7 This is a schematic diagram illustrating the relationship between the performance of sparse matrix-vector multiplication and the number of non-zero elements in the matrix, as provided in an embodiment of the present invention.

[0030] Figure 8 This is a schematic diagram showing the distribution of the test matrix memory ratio and computation speed ratio provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0032] In this invention, the terms "first," "second," etc., used in the invention and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0033] Example 1 like Figure 1 and Figure 2 As shown, this embodiment provides a method for adaptive storage and parallel computing of sparse matrices based on vertex cutting and two-stage compression. This embodiment mainly illustrates the application of this method to a distributed computing server (or cluster computing nodes) as an example. It can be understood that this method can also be downgraded to a local single-machine terminal, and can also be applied to a system including terminals and cloud servers, and implemented through communication and interaction between the terminal and the server.

[0034] In this embodiment, the method sequentially includes: a cross-linked list dynamic assembly stage, a two-stage traversal compression stage, a heuristic feature diagnosis and format adaptive selection stage, a vertex-cutting-based graph segmentation stage, and an asynchronous communication and computation overlap stage.

[0035] Specifically, the system first acquires the global sparse matrix data. In practical engineering applications, this data can be acquired by parsing a standardized external matrix storage file or by directly receiving a discrete triplet coordinate data stream dynamically generated by the nonlinear finite element solver during the assembly phase. Since this initial data is usually unordered and contains a large number of repeated coordinates (which need to be accumulated), the system initializes a cross-linked list data structure in memory to dynamically assemble the elements in the global sparse matrix data. This cross-linked list data structure dynamically states the global sparse matrix data as a pre-buffer, supporting flexible insertion and accumulation of elements, as well as the determination of zero elements in the matrix. After assembly, the system uses a two-stage traversal algorithm to efficiently reduce the dimensionality of the discrete nodes in the cross-linked list to a Compressed Sparse Row (CSR) format stored in one-dimensional contiguous memory, resulting in a CSR format matrix. Subsequently, for the micro-topology of the matrix, the system calculates its row non-zero variance and diagonal occupancy. When the diagonal occupancy exceeds a preset threshold (e.g., 0.6), it triggers memory reallocation and adaptively converts the CSR-formatted matrix to a diagonal (DIA) storage format, achieving adaptive matrix storage. Before entering distributed parallel computing, the system extracts the topology of the main clean graph without self-loops corresponding to the stored matrix and performs multi-way recursive partitioning to generate a distributed mapping array that determines data ownership. Then, it independently constructs a set of non-local dependent vertices on each computing node. Finally, based on the generated communication schedule table, the underlying message passing interface performs non-blocking data exchange, while the upper-layer computing threads synchronously perform local matrix-vector multiplication, achieving deep overlap between computation and communication.

[0036] like Figure 3 As shown, in the initial dynamic assembly stage, to address the frequent memory reallocation problem caused by the high-frequency accumulation and insertion of matrix elements in nonlinear solutions, this embodiment constructs a cross-linked list in memory containing row head pointer arrays and column head pointer arrays. The logical structure of its underlying computer data nodes (global sparse matrix data) is instantiated as a composite storage unit containing a data record field and a memory addressing field, that is, the sparse matrix data is discretized into node data in a cross-linked list composed of row and column linked lists. Specifically, the logical structure of the current node in the cross-linked list includes: 1) Position index variable, configured as integer data, is used to represent the spatial logical coordinates of the non-zero element represented by the current node in the global sparse matrix, specifically including row index and column index; 2) Numerical variables are configured as double-precision floating-point data to accurately store the actual size of the non-zero element; 3) The first direction pointer (right pointer, belonging to the row linked list) is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same row as the current node during dynamic assembly; 4) The second direction pointer (down pointer, belonging to the column linked list) is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same column as the current node during dynamic assembly.

[0037] When using an orthogonal list to store a sparse matrix, each non-zero element in the matrix corresponds to a node, and this node belongs to two linked lists simultaneously: Row-linked list: Links all non-zero elements in the same row in ascending column order.

[0038] Column-linked list: Links all non-zero elements in the same column in ascending order of row number.

[0039] In the default initialization state of the cross-linked list nodes, the aforementioned numerical variables are set to zero, and both the first and second direction pointers are suspended and point to empty memory addresses. When a new element with coordinates and a numerical value is received, the system traverses along the first direction pointer of the row linked list. If a matching column index is found, the value is incremented in place; if not, a new node is dynamically allocated, and its predecessor pointer is modified to simultaneously anchor it to both the row and column linked lists. Specifically, when it is necessary to insert or update an element (i, j, value) (i.e., the element at the i-th row and j-th column of the matrix has the value value), the logic is as follows: Search: In the row list of row i, search for a node whose column number is equal to j, in column order. If found, it means that a non-zero element (i, j, old_val) already exists at that position, so perform an in-place update—add the value field of the node directly, that is, update the value of the node to val_new = val_old + value. If not found, it means that the current value at that position is zero, and a new node needs to be inserted, so perform the "Insert New Node" step below.

[0040] Inserting a new node: Dynamically allocate a new node `newNode`, with row and column number (i, j) and initial value `value`; insert `newNode` into: In the row list of row i: after the node with the largest column number less than j (i.e., maintaining the column-incrementing order within the row); In the column list of column j: after the node with the largest row number less than i (i.e., maintaining the row-incrementing order within the column); The first and second pointers of the new node point to its successor in the row and column linked lists, respectively.

[0041] In this way, the global sparse matrix is ​​transformed into a cross-linked list structure, which enables flexible insertion and accumulation of elements in the sparse matrix. It only stores non-zero elements, saving storage space. At the same time, it supports sparse structure changes and has strong dynamics.

[0042] After assembly, a two-phase traversal algorithm is performed on the dynamically assembled cross-linked list to reduce its format (compress). The core of this algorithm is to avoid multiple dynamic memory expansions; its specific logic is as follows: 1) First stage traversal (calculating prefix sum and establishing row pointers): Set the number of rows in the global sparse matrix to... The number of columns is Then, assign a length of to each row of the global sparse matrix. A temporary statistical array is initialized to zero. Each row of the cross-linked list is traversed sequentially. For the current row, upon encountering a valid node (the node containing a non-zero element), the non-zero element represented by the current node is added to the corresponding row's statistical value (representing the current row number). This sum is used as the value at the corresponding position in the current row's temporary statistical array, resulting in an updated temporary statistical array for the current row. For example, if the current node's coordinates are (i, j), the non-zero element at (i, j) is added to i, and this sum is used as the value at the j-th position in the i-th row's temporary statistical array. For the position of the zero element in the sparse matrix, the value in the temporary statistical array at that position is empty. Subsequently, the prefix sum of the current row, i.e., the total number of non-zero elements in the current row, is calculated based on this statistical array. This is used to generate a core line pointer array in compressed sparse line (CSR) format, which is composed of... An array of pointers to elements is used to indicate the number of non-zero elements in the current row. At this point, the system has a clear understanding of the exact total number of non-zero elements in the entire matrix, so it allocates a fixed-length column index array and a value array in memory at once to store the non-zero elements of each row, resulting in a compressed matrix after row dimensionality reduction (a matrix in which zero elements do not occupy memory).

[0043] 2) Second-stage traversal (contiguous memory filling, filling CSR column index and value arrays): The system copies a row pointer array (the core row pointer array in CSR format) as a dynamic insertion offset pointer for each row to determine the initial insertion offset for each row. The cross-linked list (COO array) is traversed again from the beginning. When the current node of the current row is read, the column index and value of the current node are written to the corresponding positions in the column index array and value array according to the insertion offset of the current row. Then, the insertion offset of the current row is incremented by one and used as the insertion offset for the next node. After the traversal is completed, the physical storage space is compacted and reconstructed, resulting in a CSR format matrix. In the variables of the second stage, as the offset of each row gradually increases, the space and memory occupied by each element decrease accordingly, achieving dimensionality reduction in the column direction.

[0044] like Figure 4 As shown, to achieve a balance between universality and vectorization performance, this system establishes a heuristic diagnostic model based on mathematical statistics to determine whether the dimensionality-reduced matrix should be stored in CSR format or DIA format. Let the number of rows in the global sparse matrix be... , No. The number of non-zero elements in a row is The system first calculates the average number of non-zero elements in each row. Then, the row nonzero variance used to evaluate the multithreaded load dispersion is calculated. : To perform structural diagnosis on the obtained CSR format matrix, The larger the variance, the more uneven the distribution of non-zero elements between rows. In this case, adaptively converting the CSR format matrix to a DIA (diagonal) format matrix for storage can improve the efficiency of subsequent parallel computing. Conversely, the smaller the variance, the more balanced the distribution between rows (the number of non-zero elements in all rows is close to the mean). In this case, directly using the CSR format matrix for storage has a more significant advantage. Simultaneously, the diagonal occupancy rate of the CSR format matrix is ​​calculated. (An important metric in matrix storage). The diagonal offset is determined based on the difference between the column and row indices in the CSR-formatted matrix. The CSR-formatted matrix is ​​scanned to identify all sets of diagonals containing non-zero elements. For each diagonal in this set, the total number of theoretically valid positions within the matrix dimension is calculated. Then, the total number of non-zero elements actually filling these diagonals was counted. Generate diagonal occupancy rate : In this embodiment, the preset decision threshold for diagonal occupancy rate is 0.6. This is true if and only if... When this happens, the system triggers memory reallocation, adaptively converting the matrix from CSR format to DIA (diagonal) format to maximize the use of the processor's SIMD vectorized instructions; if If so, the CSR format is retained as a robust benchmark for parallel computing.

[0045] Before entering distributed computing, the system needs to map the algebraic matrix into a computational topology graph model. To ensure that the graph partitioning algorithm can accurately evaluate the amount of communication edge cutting across computing nodes, the interference of the matrix's main diagonal elements on the topology mapping must be eliminated. In the implementation of undirected graph topology extraction and self-loop filtering, the underlying computer traversal and diagnostic logic is specifically instantiated into the following control flow steps: The scanning interval is defined by extracting the starting memory offsets of the current target column and the next column from the column pointer array in the compressed matrix (either the CSR or DIA format matrix stored above). This determines the loop traversal interval for the non-zero elements of the target column. In other words, the interval between the starting memory offsets of the previous and next target columns is used as the loop traversal interval for the non-zero elements of the target column. Position feature analysis involves reading the target row index to which the current non-zero element belongs from the corresponding row index array during each iteration of the loop traversal interval (loop traversal of the non-zero elements of the target column) based on the current iteration step size. The system uses a self-loop filtering mechanism to compare the target row index with the current target column index. If the two values ​​are equal, the non-zero element is determined to be located on the main diagonal of the matrix. The system identifies this as a redundant self-loop connection at the graph topology level and triggers a branch jump instruction to stop the subsequent processing of the current non-zero element (the current element does not participate in subsequent parallel computation), and directly proceeds to the next iteration (traversal of the next non-zero element). The effective degree count is accumulated. If the two values ​​are not equal, the non-zero element is determined to be a valid topological connection edge across vertices. The system then uses the target row index as the memory address index, which is stored in a pre-initialized row feature statistics array to count the number of non-zero elements in the target row. In the pre-initialized row feature statistics array, the count of non-zero elements corresponding to the target row (the number of non-zero elements in the target row) is incremented, and the next iteration begins. Here, the number of non-zero elements in the target row is the number of non-zero elements remaining after removing the non-zero elements in the target row that are located on the main diagonal of the compressed matrix. In memory, the system maintains a one-dimensional database to record how many "valid communication edges" each row has. When the code confirms that the currently scanned non-zero element is not a main diagonal element, it is a valid cross-node dependency edge. The system then finds the corresponding row in the database and increments its count by 1.

[0046] After traversing all positions in all rows, based on the non-zero element counts of all rows, a graph topology is obtained that contains only elements located on the non-main diagonal of the matrix, which is the filtered and purified graph topology.

[0047] The filtered and purified graph topology is input into a multi-path recursive graph partitioning engine, such as... Figure 5 As shown, the engine adopts a heuristic strategy based on vertex cutting to optimize load balancing by minimizing the amount of communication edge cutting across partitions, and outputs a distributed mapping array with a length equal to the number of global vertices in the filtered and purified graph topology.

[0048] like Figure 6As shown, after each distributed node (each element in the distributed mapping array) obtains its own node identifier, it constructs a parallel computation and communication pipeline based on the previously generated distributed mapping array. First, each node traverses the filtered and purified global vertices of the graph topology, filtering out the set of local vertices belonging to the current node. For each local vertex in this set, it searches for all its adjacent vertices in the adjacency array of the filtered and purified graph topology. By querying the node identifiers of these adjacent vertices in the distributed mapping array, the system performs a strict comparison: if the node identifier of an adjacent vertex (i.e., its own node identifier) ​​is not equal to the node identifier of the current node, it indicates that the local computation depends on data stored on external nodes. The system adds this adjacent vertex to the "non-local dependent vertex set" and records its source node identifier (i.e., its own node identifier), thereby establishing a point-to-point communication scheduling table. This communication scheduling table records the mapping relationship between the current node and its corresponding "non-local dependent vertex set". During parallel computation, the local vertices of the current node call the corresponding adjacent vertices (external nodes) from the "non-local dependent vertex set" based on the communication scheduling table for parallel computation.

[0049] If the ownership identifier (i.e., the self-node identifier) ​​of an adjacent vertex is equal to the self-node identifier of the current node, then the local vertex of the current node will directly perform subsequent parallel computation, that is, without communication branching, it will directly perform computation branching.

[0050] In this embodiment of the invention, the system initiates a parallel branch execution strategy in the subsequent distributed matrix-vector calculation loop: 1) In communication branching, when the ownership identifier (i.e., the self-node identifier) ​​of an adjacent vertex is not equal to the self-node identifier of the current node, the system, based on the communication scheduling table, calls the underlying asynchronous receiving interface in advance to obtain the non-local dependent vertex data sent by the adjacent nodes (adjacent vertices) of the current node, and calls the asynchronous sending interface to distribute the non-local dependent data packets outward. 2) In the computation branch, the processor does not need to wait for external data to arrive. It directly performs local sparse matrix-vector multiplication operations on the "pure local matrix blocks" where all adjacent vertices (corresponding to the current node) are located locally. That is, when the ownership identifier (i.e., the self-node identifier) ​​of an adjacent vertex is equal to the self-node identifier of the current node, the local vertex of the current node directly performs local sparse matrix-vector multiplication operations. After the local operation is completed, the system calls the synchronous blocking interface to confirm that all asynchronous send and receive tasks have been transmitted in the network layer. Finally, the system concatenates the received external data (data of adjacent vertices) with the local data and performs multiplication calculations on the remaining vertices in the boundary region. Specifically, in global computation, a computation node (such as node A) only stores a portion of the globally supported data (local data) in its memory. However, some elements in the matrix region it is responsible for computing need to be multiplied by (external data) stored in other nodes (such as node B). While waiting, node A allocates a separate contiguous space in memory called the node-free and memory-occupied space. After TCP transmits the data from node B, the system sequentially "pastes" this data into the local memory space, forming a complete and continuous bandwidth that can be directly used by the CPU. While node A waits for network transmission, it simultaneously completes the calculations for the "purely local part" that does not require external data. Once the data is organized, the remaining "boundary areas" that were temporarily shelved due to missing data can be used to perform the final multiplication and addition operations using the newly pieced-together data, thus completing all the tasks for this node.

[0051] Combination Figure 7 and Figure 8 The experimental test results shown are as follows: Figure 7 The horizontal axis in the figure represents the number of non-zero elements in the matrix (NNZ), and the vertical axis GFLOP / s represents the SpMV performance, i.e., the performance of matrix-vector multiplication. Figure 8 The horizontal axis, Number of Processes, represents the total number of processes; the vertical axis, Speedup (relative to p=2), represents the communication ratio, i.e., the memory ratio; and the vertical axis, Parallel Efficiency (%), represents the computation speed ratio, i.e., the parallel efficiency. This represents one of the functional improvements implemented in this embodiment of the invention. The technical solution of this invention demonstrates significant beneficial effects: for test samples with extremely low sparsity, a storage compression rate of over 99.9% is achieved after two-stage compression; after using static graph partitioning and establishing a communication scheduling table on multi-core nodes, cross-boundary communication volume is reduced by 88% to 91%; and after achieving communication and computation overlap through non-blocking transmission and reception, the computational throughput of unstructured matrix multiplication tasks is significantly improved.

[0052] Example 2 Based on the same inventive concept as the above embodiments, this invention also provides a sparse matrix adaptive storage and parallel computing device based on vertex cutting and two-stage compression. The device's architecture is divided into five core virtual functional modules, including: 1) The initial dynamic assembly module is used to initialize a cross-linked list in memory for the dynamic assembly of sparse matrix elements in dynamically generated sparse matrix data in nonlinear problems.

[0053] 2) Two-stage compression conversion module, which is used to reduce the dimension of the dynamically assembled cross-linked list data structure to a one-dimensional continuous compressed sparse row (CSR) format through a two-stage traversal algorithm, thus completing two-stage compression.

[0054] 3) Heuristic diagnosis and adaptive module, which is used to build a heuristic diagnosis model based on row non-zero variance and diagonal occupancy rate. When the row non-zero variance or diagonal occupancy rate meets the preset threshold, it is adaptively converted to diagonal storage format; otherwise, the compressed sparse row format is retained.

[0055] 4) Vertex cutting and segmentation module, which is used to extract the clean graph topology after removing self-loop elements on the main diagonal, and perform multi-way recursive graph segmentation based on vertex cutting to generate a distributed mapping array from global vertices to distributed computing nodes.

[0056] 5) Asynchronous communication and parallel computing module, which is used to identify cross-boundary edges based on distributed mapping array to build a communication scheduling table, and realize deep overlap between non-blocking data exchange and local matrix calculation.

[0057] For the specific implementation of each module, please refer to the description of the corresponding steps in Embodiment 1 above, which will not be repeated here.

[0058] Example 3 This embodiment also provides a computer device, including a processor, a memory, a network interface, and a database connected via a system bus. The processor of this computer device provides computational and control capabilities, performing local computation and non-blocking scheduling of sparse matrices; the memory includes a non-volatile storage medium and internal memory, storing an operating system and a computer program, which, when executed by the processor, implements the steps of the method in Embodiment 1 above; the network interface is used for asynchronous data exchange with external computing nodes.

[0059] The relevant technical solutions are the same as above, and will not be repeated here.

[0060] Example 4 This embodiment provides a computer-readable storage medium on which a computer program is stored. When executed by a processor, the computer program implements the steps of the sparse matrix adaptive storage and parallel computing method based on vertex cutting and two-stage compression described in Embodiment 1 above. The medium may include a non-volatile storage carrier with the characteristic of maintaining the stability of internal code logic for a long time.

[0061] The relevant technical solutions are the same as above, and will not be repeated here.

[0062] Example 5 This embodiment provides a computer program product, including a computer program / instruction sequence, which, when allocated or downloaded to the execution memory of one or more processors, causes the processor to execute the steps of the method in Embodiment 1 above.

[0063] The relevant technical solutions are the same as above, and will not be repeated here.

[0064] Those skilled in the art will readily understand that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A sparse matrix adaptive storage method based on two-stage compression, characterized in that, include: Initialize a cross-linked list in memory to dynamically assemble the currently generated sparse matrix; Assign a length of to each row of the dynamically assembled cross-linked list. A temporary statistics array is initialized to zero; each row of the cross-linked list is traversed sequentially, and for each valid node encountered, the non-zero element value represented by the valid node is added to the current row number, and this sum is used as the value at the corresponding position in the temporary statistics array of the current row; the count of non-zero elements in each row is calculated based on the temporary statistics array of each row. This generates an array of row pointers in compressed sparse line (CSR) format; where, The number of columns in the sparse matrix is ​​given by the number of columns, and the effective nodes are those that represent the non-zero elements in the sparse matrix. The row pointer array is used as the dynamic insertion offset pointer for each row to determine the initial insertion offset for each row. Then, each row of the cross-linked list is traversed sequentially. When the current valid node of the current row is reached, the column index and value of the current valid node are written into the corresponding positions of the column index array and the value array according to the insertion offset of the current row. Then, the insertion offset of the current row is incremented by one and used as the insertion offset of the next valid node. After the traversal is completed, a matrix in CSR format is obtained. Calculate the row nonzero variance of the matrix in the CSR format. and the diagonal occupancy of the matrix in CSR format. ;judge or If the value exceeds a preset threshold, the CSR format matrix is ​​converted to a DIA format matrix so that the sparse matrix is ​​stored in the DIA format; otherwise, the sparse matrix is ​​stored in the CSR format.

2. The sparse matrix adaptive storage method according to claim 1, characterized in that, During the initialization of the cross-linked list in memory for dynamic assembly of the currently generated sparse matrix, when inserting or updating the element (i, j, value) with the value value at the i-th row and j-th column of the sparse matrix in the current node of the cross-linked list, the following operation is performed: In the row list of row i, search for a node with column index j in column index order. If found, it means that a non-zero element (i, j, old_val) already exists at that position, so perform an in-place update: update the value of the current node to the sum of value and old_val. If not found, it means that the current position is zero, so dynamically allocate a new node newNode with row and column index (i, j) and initial value value. Insert newNode into the row list of row i after the largest column index node with column index less than j, and into the column list of column j after the largest row index node with row index less than i. The first and second pointers of newNode point to its successor in the row and column lists, respectively. The first pointer is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same row as the current node during dynamic assembly. The second pointer is configured as a memory address reference variable, used to link and point to the next non-zero element node in the same column as the current node during dynamic assembly.

3. The sparse matrix adaptive storage method according to claim 1 or 2, characterized in that, The row non-zero variance for: in, Let be the number of rows in the sparse matrix. This represents the average number of non-zero elements in each row of the cross-linked list.

4. The sparse matrix adaptive storage method according to claim 1 or 2, characterized in that, diagonal occupancy rate The calculation method is as follows: The diagonal offset is determined based on the difference between the column index and the row index in the CSR format matrix; based on the diagonal offset, the CSR format matrix is ​​scanned to identify all diagonal sets containing non-zero elements; For each diagonal in the set of diagonals, calculate the total number of theoretically valid positions for it within the matrix dimension. ; And count the total number of non-zero elements actually filled on all diagonals. Generate diagonal occupancy rate : .

5. A parallel computation method for sparse matrices based on vertex cutting, characterized in that, include: S1. Based on the column pointer array in the compressed matrix column format, extract the interval from the starting memory offset corresponding to the current column to the ending memory offset corresponding to the next column as the loop traversal interval for the non-zero elements of the current column; wherein, the compressed matrix is ​​a CSR format matrix or a DIA format matrix stored using the sparse matrix adaptive storage method described in any one of claims 1-4. S2. During the traversal of the current non-zero element, based on the current iteration step size, read the row index to which the current non-zero element belongs from the corresponding row index array; determine whether the row index is equal to the current column index. If so, determine that the current non-zero element is located on the main diagonal of the compressed matrix and terminate the participation of the current non-zero element in subsequent parallel computation; if not, use the row index as the memory addressing subscript and perform an incrementing accumulation operation on the non-zero element count value corresponding to the current row in the pre-initialized row feature statistics array; wherein, the memory addressing subscript is used to determine the non-zero element count value corresponding to the current row in the row feature statistics array. S3. Take the next non-zero element as the current non-zero element, repeat S2, until all non-zero elements in the compression matrix have been traversed, and obtain the filtered and purified graph topology after removing the non-zero elements located on the main diagonal of the compression matrix, which is determined by the count value of the remaining non-zero elements in the compression matrix. S4. Input the filtered and purified graph topology into the multi-path recursive graph partitioning engine, optimize load balancing by minimizing the amount of communication edge cutting across partitions, and output a distributed mapping array with a length equal to the number of global vertices of the filtered and purified graph topology; wherein, the multi-path recursive graph partitioning engine adopts a heuristic strategy based on vertex cutting. S5. Perform parallel sparse matrix-vector multiplication operations based on the distributed mapping array.

6. The sparse matrix parallel computing method according to claim 5, characterized in that, Parallel sparse matrix-vector multiplication based on the distributed mapping array includes: Each element in the distributed mapping array is taken as a node, and each node obtains its own node identifier in the distributed mapping array. Each node traverses the global vertices of the filtered and purified graph topology to obtain a local vertex set belonging to the current node; for each local vertex in the local vertex set, it searches for all its neighboring vertices in the adjacency array of the filtered and purified graph topology, and queries the self-node identifier of the neighboring vertex in the distributed mapping array. Determine whether the self-node identifier of the adjacent vertex is equal to the self-node identifier of the current node. If they are not equal, add the adjacent vertex to the set of non-local dependent vertices to establish a communication scheduling table representing the mapping relationship between the current node and its corresponding set of non-local dependent vertices, and execute the following communication branch pipeline; if they are not equal, directly execute the following computation branch pipeline. Communication branch pipeline: Based on the communication scheduling table, the underlying asynchronous receiving interface is called to obtain the non-local dependent vertex data sent by the non-local dependent vertex of the current node, and the asynchronous sending interface is called to distribute the non-local dependent vertex data outward, so as to realize asynchronous sending and receiving; Calculate the branch pipeline: The current node and its corresponding adjacent vertices directly perform sparse matrix-vector multiplication. After the operation is completed, the synchronous blocking interface is called. When the asynchronous sending and receiving tasks of the communication branch pipeline have been transmitted in the network layer, the received non-local dependent vertex data is used to perform sparse matrix-vector multiplication with the current node.

7. A sparse matrix adaptive storage system based on two-stage compression, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the sparse matrix adaptive storage method according to any one of claims 1-4.

8. A sparse matrix parallel computing system based on vertex cutting, characterized in that, Includes computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the sparse matrix parallel computing method according to claim 5 or 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the sparse matrix adaptive storage method as described in any one of claims 1-4, or / and the sparse matrix parallel computing method as described in claim 5 or 6.

10. A computer program product, characterized in that, Includes a computer program that, when run on a computer, causes the computer to perform the sparse matrix adaptive storage method according to any one of claims 1-7, or / and to perform the sparse matrix parallel computation method according to claim 5 or 6.

Images