Deployment mapping analysis system and application of graph neural network in sparse memory computation
By deploying and mapping an analysis system for graph neural networks in sparse in-memory computing, the efficiency reduction problem caused by sparse computing is solved. The efficiency of the in-memory computing platform is improved by optimizing the computation order and data rearrangement through a simulator.
Patent Information
- Application Number
- CN202511835698.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-08
- Publication Date
- 2026-02-17
AI Technical Summary
Sparse computing reduces deployment efficiency in in-memory computation of graph neural networks, and existing technologies have failed to effectively address the optimization issues of computational and data handling in sparse computing.
This paper provides a deployment mapping analysis system for graph neural networks in sparse in-memory computation. By inputting hardware and network parameters through a simulator, the system performs sparse data rearrangement and counts of non-zero rows, analyzes mapping options for different computation orders and matrix multiplications, and selects the optimal mapping method to reduce computation and data transfer.
It improves the utilization of the in-memory computing platform, reduces the amount of computation and data transfer, and enhances the deployment efficiency of sparse graph neural networks.
Smart Images
Figure CN121543672A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of graph neural network, and particularly relates to a deployment mapping analysis system of a graph neural network in sparse in-memory computing and application. BACKGROUND
[0002] A graph convolution network (GCN) is a kind of deep learning model specially processing graph structure data, and can capture complex relationships of nodes, edges and overall structure in a graph. The calculation of the GCN mainly involves three matrices, namely an adjacency matrix A, a feature value matrix X and a weight matrix W. The GCN needs to calculate the multiplication of the three matrices, that is, AXW.
[0003] In-memory computing (CIM) is to embed computing capability into a memory array, and directly perform data calculation on the original site, so as to reduce data transfer. When the CIM is applied to the GCN, the efficiency of deploying the GCN model can be improved, but there is sparse calculation in the GCN, which reduces the efficiency of deploying the GCN. SUMMARY
[0004] The purpose of the application is to provide a deployment mapping analysis system of a graph neural network in sparse in-memory computing, so as to solve the problems existing in the prior art.
[0005] To achieve the above purpose, the application provides a deployment mapping analysis system of a graph neural network in sparse in-memory computing, comprising:
[0006] The simulator inputs hardware parameters, network parameters and dataset data, and outputs data transfer amount and calculation amount corresponding to different mapping options. First, sparse-aware data reorder (based on sparse data reordering) is input, and the A matrix and the X matrix are rearranged, so that sparse CIM can be used. Then, the non-zero rows counter (non-zero row counter) is used to calculate the number of non-zero rows of the A matrix and the X matrix after rearrangement. Then, the calculation amount of different calculation sequences and the analysis of matrix multiplication mapping are performed. According to the output result of the simulator, the most suitable mapping option can be selected, so as to reduce the calculation amount and the data transfer amount.
[0007] The application is based on a multi-address sparse CIM. First, the execution sequence of the basic operators (aggregation and combination) of the GCN is analyzed, and then the matrix multiplication of the basic operation of the GCN is further analyzed.
[0008] Optionally, the principle of the simulator for analyzing the deployment of the GCN is as follows:
[0009] The computation of GCN mainly involves three matrices: the adjacency matrix A, the eigenvalue matrix X, and the weight matrix W. GCN requires calculating the multiplication of these three matrices, which is AXW.
[0010] Exploring the Computation Order of GCN: Different computation orders lead to different computational costs. The specific computational costs of (AX)W and A(XW) are a hardware and software issue, influenced by model size, dataset parameters, and hardware execution methods. Some academic GCN accelerators have analyzed the computational costs of GCN under different computation orders, but these analyses were all performed on intensive computing engines. This invention, taking sparse CIM as an example, explores the impact of GCN execution order on computational costs on sparse CIM platforms.
[0011] Deployment Analysis of Matrix Multiplication in GCN: When the computation order is A(XW), during the computation of (XW), the weight matrix W is stored as fixed weights in the CIM, while the feature matrix X is input into the CIM after sparsification. During the computation of A(XW), the intermediate result (XW) is stored as weights in the CIM, and the adjacency matrix A is calculated after sparsification. The mapping from matrix multiplication to the CIM consists of two steps: inter-tile mapping and intra-tile mapping. The former focuses on how to distribute the looptiling tiles to multiple CIMs, while the latter involves the specific computation process of a single tile within a CIM. We define the load of a CIM as a tile. For a multi-CIM architecture, the load of all CIMs is defined as a tilegroup.
[0012] Optional, Inter-tile deployment: Consider inter-tile loop interchange and inter-tile loop unrolling. Let the three dimensions of matrix multiplication be M, N, and P, with corresponding indices i, j, and k. The output dimensions correspond to M and N, while the inner product dimension corresponds to P. Set the size of the tile_group of matrix A to TG. M *TG P The size of the tile group in matrix B is TG. P *TG N Let the size of matrix A be S. M S P The size of matrix B is S P S N .
[0013] inter-tile-loop-interchange: Prioritizing the three dimensions will have different effects:
[0014] Prioritizing the P-dimensional dimension: Calculating the inner product first reduces the amount of data that needs to be temporarily stored and the total psum. This ensures that at most only the TG (Transmission Toll) is cached on-chip. M *TG N If other dimensions are prioritized, the number of psums will be large; however, the overhead is that each new group may need to load new data.
[0015] Prioritize the M dimension: it allows for the reuse of the W matrix;
[0016] Prioritize N dimensions: Enable the reuse of the X matrix;
[0017] Inter-tile-loop-unrolling: Block unrolling refers to multiplying the matrix along three dimensions, including the inner product dimension, and then multiplying multiple CIMs in parallel for multiple tiles. This fully utilizes the parallelism between CIM macros.
[0018] With a fixed number of CIM macros, the number of tile expansions is also fixed, equal to the number of CIM macros. However, tile expansion offers three selectable dimensions; utilizing only one, two, or all three dimensions are all options that need discussion. Furthermore, when expanding multiple dimensions, the number of expansions for each dimension also needs to be considered. To reduce design space and better quantify this issue, we assign priorities to the three dimensions, i.e., expansion priorities. Dimensions with higher expansion priorities will be expanded first. That is, if the number of CIM macros N... CIM If the number of blocks is less than the number of blocks in this dimension, then only expand this dimension first; only if there are N blocks... CIM Only when the dimension exceeds this priority will the next priority dimension be introduced for expansion.
[0019] Optional Intra-tile deployment includes: naturally expanding the P and N dimensions for DCIM.
[0020] The technical effects of this invention are as follows: This invention provides a simulator that can analyze the deployment options of GCN operators on an in-memory computing platform. Given hardware and software parameters, the simulator calculates the computational load and data transfer load brought about by different deployment options, thereby helping designers and developers to choose appropriate mapping methods, improve the utilization of the in-memory computing platform, and reduce the computational load and data transfer load. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0023] Figure 1 Schematic diagram of sparse CIM structure
[0024] Figure 2 This is a schematic diagram illustrating the calculation of the number of non-zero rows in a matrix based on sparse CIM.
[0025] Figure 3 A schematic diagram of the deployment for matrix computation;
[0026] Figure 4 This is a schematic diagram of the simulator provided by the present invention. Detailed Implementation
[0027] Various exemplary embodiments of the present invention will now be described in detail. This detailed description should not be considered as a limitation of the present invention, but rather as a more detailed description of certain aspects, features, and embodiments of the present invention.
[0028] It should be understood that the terminology used in this invention is merely for describing particular embodiments and is not intended to limit the invention. Furthermore, with respect to numerical ranges in this invention, it should be understood that each intermediate value between the upper and lower limits of the range is also specifically disclosed. Every smaller range between any stated value or intermediate value within a stated range, and any other stated value or intermediate value within said range, is also included in this invention. The upper and lower limits of these smaller ranges may be independently included or excluded from the range.
[0029] Various modifications and variations can be made to the specific embodiments described in this specification without departing from the scope or spirit of the invention, as will be apparent to those skilled in the art. Other embodiments derived from this specification will also be obvious to those skilled in the art. This application specification and embodiments are merely exemplary.
[0030] The terms “include,” “including,” “have,” “contain,” etc., used in this article are all open-ended terms, meaning that they include but are not limited to.
[0031] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0032] like Figure 1 - Figure 4 As shown, this embodiment provides a deployment mapping analysis system for sparse in-memory graph neural networks, including:
[0033] The data acquisition module is used to acquire the hardware parameters of the in-memory computing platform, the network parameters of the graph neural network, and the dataset parameters.
[0034] The data rearrangement module is used to rearrange the A and X matrices of the graph neural network;
[0035] The non-zero row counting module is used to calculate the number of non-zero rows in the rearranged A and X matrices;
[0036] The calculation order selection module is used to select the matrix calculation order among the adjacency matrix A, the eigenvalue matrix X, and the weight matrix W;
[0037] The deployment mapping analysis module is used to perform computational statistics and deployment mapping analysis for different computation orders, and select the optimal deployment mapping method based on the output results of the deployment mapping analysis module.
[0038] This embodiment provides a simulator for analyzing the mapping options of graph neural networks (GCNs) on sparse in-memory computing (CIM) systems, and for analyzing the utilization of GCN operators on the in-memory computing platform. Given hardware and software parameters, the simulator calculates the computational and data transfer volumes associated with different mapping options, thereby helping designers and developers select appropriate mapping methods to improve the utilization of the in-memory computing platform and reduce computational and data transfer volumes.
[0039] This embodiment is based on sparse CIM, whose macrocells are multi-address in-memory computation macrocells, and the indices of each row of candidate word lines (WL) are different. As a result, the memory cells read in the subarray no longer correspond to the same offset address, so as to support more flexible weighted read operations and eliminate unnecessary zero-value calculations. For example: Assuming the number of input channels in CIM is M, each row of matrix A is split into multiple sub-blocks of length N. The length of the last sub-block can be a positive integer less than or equal to N (e.g., when a row of matrix A contains 11 elements and (N=4), it is split into two sub-blocks containing 4 elements each and the last one containing 3 elements, with 0s added to any missing positions). In each of the split sub-blocks, data at the same position across sub-blocks (because the weights corresponding to these data are all in the subarray of the same row) (analogous to "data in the same column of a table") can be arbitrarily swapped (because the indices of the candidate word lines (WL) in each row are different, each row can select different weight data to participate in the calculation, thus matching the data exchange of the input). Taking the split sub-blocks of the input matrix row in the figure as an example, through the swapping operation, the scattered non-zero values (such as "1, 2, 4, 5" in the example) can be gathered as much as possible into the earlier sub-blocks, so that zero values are more concentrated in the subsequent sub-blocks, thereby improving the spatial locality of effective data and facilitating the efficient access and calculation of non-zero data by the CIM hardware. In the elimination step, if a sub-block containing all zeros (i.e., all elements in the sub-block are 0) exists after the swaps, such a sub-block does not require any multiplication or addition operations and can be skipped directly; for example... Figure 2 As shown, CIM originally required three calculations for the three sub-blocks after splitting. By eliminating all-zero sub-blocks, only two calculations are needed in the end, thereby reducing the redundant calculation overhead caused by zero values and significantly improving the calculation efficiency of GCN sparse data under the CIM architecture.
[0040] This embodiment is based on a multi-address sparse CIM. First, it analyzes the execution order of the basic GCN operators (aggregation and combination), and then further analyzes the matrix multiplication, the basic operation of GCN.
[0041] This simulator analyzes the principles of GCN deployment, specifically including:
[0042] The computation of GCN mainly involves three matrices: the adjacency matrix A, the eigenvalue matrix X, and the weight matrix W. GCN requires calculating the multiplication of these three matrices, which is AXW.
[0043] Exploring the Computation Order of GCN: Different computation orders lead to different computational costs. The specific computational costs of (AX)W and A(XW) are a hardware and software issue, influenced by model size, dataset parameters, and hardware execution methods. Some academic GCN accelerators have analyzed the computational costs of GCN under different computation orders, but these analyses were all performed on intensive computing engines. This invention, taking sparse CIM as an example, explores the impact of GCN execution order on computational costs on sparse CIM platforms.
[0044] Deployment Analysis of Matrix Multiplication in GCN: When calculating (XW), the weight matrix W is stored as fixed weights in the CIM, while the feature matrix X is input into the CIM after sparsification. When calculating A(XW), the intermediate result (XW) is stored as weights in the CIM, and the adjacency matrix A is calculated after sparsification. The mapping from matrix multiplication to the CIM consists of two steps: inter-tile mapping and intra-tile mapping. The former focuses on how to distribute the tiles after loop tiling to multiple CIMs, while the latter involves the specific calculation process of a single tile within a CIM. We define the load of a CIM as a tile. For a multi-CIM architecture, the load of all CIMs is defined as a tile group.
[0045] Inter-tile deployment: Consider inter-tile loop interchange and inter-tile loop unrolling. Let the three dimensions of matrix multiplication be M, N, and P, with corresponding indices i, j, and k. The output dimensions correspond to M and N, while the inner product dimension corresponds to P. Set the size of the tile group of matrix A to TG. M *TG P The size of the tile group in matrix B is TG. P *TG N Let the size of matrix A be S. M S P The size of matrix B is S P S N .
[0046] inter-tile-loop-interchange:
[0047] Prioritizing these three dimensions will have different effects:
[0048] Prioritizing the P dimension: Calculating the inner product first reduces the number of psums that need to be temporarily stored. This ensures that at most TGM*TGN psums are cached on-chip. Prioritizing other dimensions would result in many more psums; however, this incurs the overhead of potentially loading new data for each new group.
[0049] Prioritize the M dimension: it allows for the reuse of the W matrix;
[0050] Prioritize N dimensions: Enable the reuse of the X matrix;
[0051] Inter-tile loop unrolling:
[0052] Block expansion refers to multiplying a matrix in three dimensions, including the inner product dimension, and then multiplying multiple CIMs in parallel for multiple tiles. This fully utilizes the parallelism between CIM macros.
[0053] With a fixed number of CIM macros, the number of unfolded tiles is also fixed, equal to the number of CIM macros. However, tile unfolding offers three possible dimensions; utilizing only one, two, or all three dimensions are all options that need discussion. Furthermore, when unfolding multiple dimensions, the number of unfolded tiles for each dimension also needs to be considered. To reduce design space and better quantify this issue, we assign priorities to the three dimensions, i.e., unfolding priorities. Dimensions with higher unfolding priorities are unfolded first. That is, if the number of CIM macros (NCIM) is less than the number of tiles in a given dimension, then only that dimension is unfolded first; only when the NCIM is greater than that dimension will the next higher priority dimension be introduced for unfolding.
[0054] The three dimensions are defined as M, N, and P. At this time, the length and width of A are M and P, the length and width of B are P and N, and the length and width of C are M and N.
[0055] The expansion of the three dimensions brings different advantages, and the results are shown in Table 1.
[0056] Prioritize expanding the P dimension, which means increasing its size. This will lead to As it increases, Decrease.
[0057] Prioritize expanding the M dimension, which means increasing its size. This will lead to As it increases, Decrease.
[0058] First expand the N-dimensional dimension, which means increasing the size. This will lead to As it increases, Decrease.
[0059] Table 1
[0060] A load quantity / tile group time B load quantity / tile group time Maximum psum preservation quantity / tile group quantity MNP MPN NMP NPM PMN 1 PNM 1
[0061] Intra-tile deployment: For DCIM, it naturally expands the P-dimensional and N-dimensional dimensions.
[0062] Mapping simulator: The architecture of a mapping simulator is as follows Figure 4 As shown, the simulator takes hardware parameters, network parameters, and dataset data as input and outputs the data transfer and computational costs corresponding to different mapping options. First, it inputs sparse-aware data reorder to reorder matrices A and X to utilize sparse CIM. Then, a non-zero rows counter is used to calculate the number of non-zero rows in matrices A and X after reordering. Following this, computational costs for different calculation orders and matrix multiplication mappings are analyzed. Based on the simulator's output, the most suitable mapping option can be selected, helping designers and developers choose appropriate mapping methods to improve the utilization of the in-memory computing platform and reduce computational and data transfer costs.
[0063] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A deployment mapping analysis system for graph neural networks in sparse in-memory computation, characterized in that, include: The data acquisition module is used to acquire the hardware parameters of the in-memory computing platform, the network parameters of the graph neural network, and the dataset parameters. The data rearrangement module is used to rearrange the A and X matrices of the graph neural network; The non-zero row counting module is used to calculate the number of non-zero rows in the rearranged A and X matrices; The calculation order selection module is used to select the matrix calculation order among the adjacency matrix A, the eigenvalue matrix X, and the weight matrix W; The deployment mapping analysis module is used to perform computational statistics and deployment mapping analysis for different computation orders, and select the optimal deployment mapping method based on the output results of the deployment mapping analysis module.
2. The system according to claim 1, characterized in that, The matrix calculation order includes (AX)W and A(XW).
3. The system according to claim 1, characterized in that, The deployment mapping analysis module specifically includes: The parameter setting module is used to set the loop mapping parameters; The inter-block cyclic rearrangement module is used to determine the computation priority of different dimensions of the matrix; The inter-block loop unrolling module is used to multiply the matrix in three dimensions, including the inner product dimension, and then perform parallel computation of multiple loads on multiple in-memory computing platforms. Deploy the compute module to perform inter-tile mapping and intra-tile mapping.
4. The application of the deployment mapping analysis system for sparse in-memory computation of graph neural networks as described in any one of claims 1-3 in the deployment analysis of GCN operators.