A graph neural network computing method and device based on resistive memory

Abstract

An embodiment of the present invention provides a graph neural network calculation method and device based on resistive memory, the method comprising: for any layer of the graph neural network, analyzing the relative size of the processing delay of the graph data to be calculated in the resistive memory array in the layer under the weighted in-situ calculation mode and the hybrid in-situ calculation mode, and selecting the mode with the smallest delay as the calculation mode of the layer; in the weighted in-situ calculation mode, mapping the adjacency matrix of the graph data and the weight parameters of the graph neural network as in-situ data of the layer of the graph neural network to the corresponding resistive memory array respectively, so as to use the node features of the graph neural network as input data to perform calculations with the corresponding in-situ data; in the hybrid in-situ calculation mode, mapping the adjacency matrix and node features of the graph data as in-situ data of the layer of the graph neural network to the corresponding resistive memory array respectively, so as to use the weight parameters as input data to perform calculations with the corresponding in-situ data.

Description

Technical Field

[0001] The present invention relates to the field of accelerated processing technology of neural networks, specifically to the field of processing technology of graph neural networks on a computing device based on resistive random access memory, and more specifically to a graph neural network computing method and device based on resistive random access memory. Background Art

[0002] Graph Learning, especially graph neural networks, is an emerging technology for processing large attribute graphs, which can be used to represent various related random data such as text, images and videos. It has been proven effective in a large number of applications such as social network analysis, e-commerce recommendation, biomolecule characterization, etc. Real-world graph data (such as e-commerce data on Taobao and social network data on WeChat) are usually large and sparse, and graph learning tasks for processing these graph data are mostly data-intensive, so when deploying them on general-purpose processors, they are limited by the bottleneck of dense data movement.

[0003] The accelerator based on RRAM realizes in-situ computing of data by mapping the selected static data onto the RRAM array, which greatly reduces the huge data movement overhead from the memory to the computing unit of the general-purpose processor. Therefore, it is a technology that is very suitable for data-intensive tasks and has broad application prospects. At present, in the design of accelerators for neural networks, accelerators based on RRAM can achieve energy efficiency ratios of several orders of magnitude higher than traditional CMOS accelerators. Therefore, it can be foreseen that the in-memory computing technology of RRAM is also suitable for the acceleration of graph learning applications. At present, research in this area at home and abroad is still in its early stages.

[0004] Before using an accelerator based on RRAM for calculation, it is usually necessary to write a portion of static data into the RRAM calculation array in advance to achieve in-situ calculation. Figure 1 As shown in the left image in the dashed box, the neural network model, i.e., the weight value, is usually pre-programmed in the resistive memory array as static data for in-situ calculation. This mode is called the weight in-situ calculation mode. It avoids the data movement of the weight value from the memory to the calculation unit. On the contrary, Figure 1The image on the right side of the dashed box is called the data in-situ computing mode, which maps the model input to the RRAM. The weight in-situ computing mode is the best choice for traditional neural network accelerators based on RRAM (such as convolutional neural networks). There are several reasons for this: 1. The input features of the convolutional neural network are much smaller than the model parameters. Keeping the model parameters static in memory has less benefit in terms of data movement overhead than the parameters. 2. The input feature values ​​of the convolutional neural network will be continuously updated layer by layer as the calculation process progresses. The update of the feature values ​​will repeatedly write them into the RRAM unit, which brings huge write operation energy consumption overhead, while the weight value does not change. Therefore, the weight mode only requires read operations with lower energy consumption overhead. 3. In typical convolutional neural network applications, such as image or speech recognition, input feature values ​​are rarely reused, so it is difficult to benefit from in-situ feature value data processing.

[0005] As shown in the following two formulas, there are some differences between convolutional neural networks and single-layer operations in graph learning:

[0006] Graph Learning: X l+1 =σ(AX l W l )

[0007] Convolutional Neural Networks: X 1+1 =σ(X 1 W l )

[0008] Where l represents the number of layers and σ() is the activation function. Unlike convolutional neural networks, a single-layer operation in graph learning includes two consecutive matrix multiplications with three matrix operands, including the graph data adjacency matrix A, the graph feature matrix X, and the weight value matrix W. These two matrix multiplications need to be calculated on the resistive memory respectively, and the execution order of the two operations and the selection of which two of the three operands are mapped to the resistive memory (i.e., the selection of the in-situ computing mode) will directly affect the graph learning computational overhead. In addition, the characteristics of the operands, i.e., matrices, in graph learning are also different from those in convolutional neural networks. First, the adjacency matrix A can be very large and sparse in big data applications such as social networks, and can also be small in graph classification applications such as skeleton recognition and MUTAG datasets. Second, the graph feature matrix X essentially represents the features of the graph. In different graph learning tasks, the size range of the feature matrix varies greatly with the size of the graph. In addition, the size of the weight value matrix in graph learning depends on the input feature dimension and the output feature dimension, and can sometimes be very compact, such as kilobits, or even several orders of magnitude smaller than the input graph size. For example, some graph learning models do not exceed three layers. Finally, unlike convolutional neural networks, in some graph learning tasks such as recommendation systems, input graph features and graph adjacency matrices are reused, so they are suitable for mapping to the RRAM array as static data for repeated calls, thereby significantly reducing data movement overhead. Therefore, in graph learning tasks, weight values ​​are not the main data movement overhead, and the weight in-situ calculation mode is not always suitable for graph learning calculations based on RRAM. In short, the choice of in-situ calculation mode, that is, deciding which part of the model input should be stored as static data in the RRAM array, will result in different data movement and write operation overheads. Therefore, flexible static data (in-situ data) selection is crucial to the energy efficiency of graph learning task processing based on RRAM accelerators. Summary of the invention

[0009] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a graph neural network calculation method and device based on resistive random access memory.

[0010] The objective of the present invention is achieved through the following technical solutions:

[0011] According to a first aspect of the present invention, there is provided a graph neural network computing method for a graph neural network computing device based on resistive memory, comprising: for any layer of the graph neural network, analyzing the relative sizes of processing delays of graph data to be calculated in the resistive memory array in the layer under a weighted in-situ computing mode and a hybrid in-situ computing mode, and selecting the mode with the smallest delay as the computing mode of the layer; in the weighted in-situ computing mode, mapping the adjacency matrix of the graph data and the weight parameters of the graph neural network as in-situ data of the layer of the graph neural network to the corresponding resistive memory array respectively, so as to use the node features of the graph neural network as input data to perform operations with the corresponding in-situ data; in the hybrid in-situ computing mode, mapping the adjacency matrix and the node features of the graph data as in-situ data of the layer of the graph neural network to the corresponding resistive memory array respectively, so as to use the weight parameters as input data to perform operations with the corresponding in-situ data.

[0012] In some embodiments of the present invention, for the first layer of the graph neural network, its processing delay only considers the computational delay.

[0013] In some embodiments of the present invention, the calculation mode of the first layer of the graph neural network is determined by the following formula:

[0014]

[0015] Among them, a k Indicates the number of nodes or average number of nodes in the k-th layer of the graph neural network. Represents the resistive memory array with the k-th layer node feature X k As the delay required to read the weight value of the kth layer when the input data is used, DAC_res represents the resolution of the digital-to-analog conversion, c k Represents the weight parameter of the kth layer The number of columns of the corresponding dimension matrix c, Represents the resistive memory array with the k-th layer weight parameter W k The delay required to read the node features of the kth layer as the input signal, k=1; wherein, if the value of t is greater than zero, it means that the processing delay of this layer in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected; otherwise, the weighted in-situ computing mode is selected.

[0016] In some embodiments of the present invention, for other layers of the graph neural network, the processing latency takes into account the computation latency and the write latency of node feature updates.

[0017] In some embodiments of the present invention, the calculation mode of other layers of the graph neural network is determined by the following formula:

[0018]

[0019] Among them, a k Indicates the number of nodes or average number of nodes in the k-th layer of the graph neural network. Represents the resistive memory array with the k-th layer node feature X k As the delay required to read the weight value of the kth layer when the input data is used, DAC_res represents the resolution of the digital-to-analog conversion, c k Represents the weight parameter of the kth layer The number of columns of the corresponding dimension matrix c, Represents the resistive memory array with the k-th layer weight parameter W k As the input signal, β represents the delay required to read the node features of the kth layer, and k≠1; where t ′ If the value of is greater than zero, it means that the processing delay of the layer data in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected. Otherwise, the weighted in-situ computing mode is selected.

[0020] In some embodiments of the present invention, mapping the in-situ data to the corresponding resistive memory array includes a sparse data mapping method and a dense data mapping method; wherein, the sparse data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data into multiple sub-graphs, deleting the empty sub-graphs with zero values, and mapping the non-empty sub-graphs to the resistive memory array in sequence; the dense data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data directly according to the size of the resistive memory array and mapping it.

[0021] In some embodiments of the present invention, when performing in-situ data mapping, the adjacency matrix as the in-situ data is mapped to the resistive random access memory array for storage by using a sparse data mapping method.

[0022] In some embodiments of the present invention, when performing in-situ data mapping, weight parameters of the in-situ data are mapped to the resistive random access memory array for storage in a dense data mapping manner.

[0023] In some embodiments of the present invention, for the first layer of the graph neural network, when performing in-situ data mapping, if the sparsity of the node features serving as the in-situ data is greater than a predetermined sparsity threshold, a sparse data mapping method is used to map it to the resistive memory array for storage; otherwise, a dense data mapping method is used to map it to the resistive memory array for storage.

[0024] In some embodiments of the present invention, for other layers of the graph neural network, when performing in-situ data mapping, the node features of the in-situ data are mapped to the resistive memory array for storage using a dense data mapping method.

[0025] According to a second aspect of the present invention, a graph neural network computing device based on resistive random access memory is provided, comprising a resistive random access memory array for storing and computing graph data, and a processing unit, characterized in that the processing unit is configured to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0027] Figure 1 A schematic diagram of a weight in-situ method and a data in-situ method in a data configuration process according to an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a weighted in-situ calculation mode and a hybrid in-situ calculation mode according to an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the principles of dense mapping and sparse mapping according to an embodiment of the present invention;

[0030] Figure 4 A schematic diagram of a graph neural network computing device based on a resistive random access memory according to an embodiment of the present invention;

[0031] Figure 5 It is a schematic diagram of the configuration of the in-situ calculation mode and mapping mode of data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] As mentioned in the background technology section, the choice of in-situ computing mode will result in different data movement and write operation overheads. Flexible static data (in-situ data) selection is crucial for the energy efficiency of graph learning task processing based on RRAM accelerator. In order to achieve energy-efficient RRAM graph learning calculations, based on the weight in-situ computing mode in the traditional RRAM-based graph neural network computing device (RRAM-based graph convolutional neural network accelerator), the present invention proposes a hybrid in-situ computing mode, that is, the weight in-situ mode and the data in-situ mode are configured on the RRAM at the same time, thereby realizing two key matrix multiplication calculations in one layer of graph learning operations. In addition, different graph learning target tasks have different computing characteristics and are therefore suitable for different in-situ computing modes. In order to find and practice optimized in-situ computing modes for different graph learning tasks, the present invention proposes to analyze the relative sizes of the processing delays of the graph data to be calculated in the resistive memory array under the weighted in-situ computing mode and the hybrid in-situ computing mode for each layer of the graph neural network, and select the mode with the smallest delay as the computing mode of this layer, which is equivalent to a task-adaptive graph learning computing selection strategy, which is used to select the data configuration scheme with the smallest processing delay, so as to efficiently process various graph learning computing tasks.

[0034] Before specifically introducing the embodiments of the present invention, some of the terms used therein are explained as follows:

[0035] Graph data refers to a graph structure that uses nodes and edges to represent the relationships between multiple nodes. Graph data is usually stored in a graph database.

[0036] Adjacency matrix, a two-dimensional array used to store the relationship between nodes.

[0037] Weight parameters refer to the model parameters of the graph neural network model.

[0038] Node features refer to the feature vector of a node.

[0039] In the process of graph neural network calculation, there are two kinds of weight data, one is the adjacency matrix, the other is the weight parameters of the graph neural network (parameters of the graph neural network model), and one kind of non-weight data, namely node features. First, we introduce two in-situ configuration methods, see Figure 1, DAC (Digital to Analog Converter) means digital-to-analog converter, which can convert digital signals into analog voltage signals; S&H (Sample and Hold) means sample and hold circuit, which is used to sample and save continuous analog signals; when performing digital-to-analog conversion on analog signals, a certain conversion time is required. During this conversion time, the analog signal must remain basically unchanged to ensure the conversion accuracy. The sample and hold circuit is a circuit that implements this function; ADC (Analog to Digital Converter) means analog-to-digital converter, which is used to convert an input analog signal into an output digital signal; in the weight in-situ mode, the weight is configured as in-situ data, and the data (node ​​features) is used as input; in the data in-situ mode, the data (node ​​features) is configured as in-situ data, and the weight (adjacency matrix or weight parameter) is used as input. Figure 1 Taking the weight in-situ method as an example, after configuration, the calculation process includes: inputting the digital signal representing the input data (i.e., the input activation value in the graph neural network) into the DAC for digital-to-analog conversion, and obtaining the corresponding analog voltage value and adding it to each row of the storage calculation array; the input analog voltage value and the resistive memory on the array obtain the dot product and current value on the column according to Kirchhoff's principle, and the magnitude of the current value is the matrix-vector product of the input data and the weight value pre-stored on the array; through the sampling and holding circuit, the obtained current result is sampled and kept unchanged to facilitate subsequent analog-to-digital conversion; the current value of the analog signal is converted into analog-to-digital through the ADC to obtain the final digital calculation result.

[0040] According to the above two in-situ configuration methods, the present invention proposes two different in-situ computing modes, wherein the weighted in-situ computing mode refers to storing two weight data as in-situ data (static data) in the corresponding resistive memory array during the calculation process of the corresponding layer of the graph neural network; the hybrid in-situ computing mode refers to storing one weight data and node features as in-situ data in the corresponding resistive memory array during the calculation process of the corresponding layer of the graph neural network. All possible data flows in a graph convolution layer are as follows: Figure 2 As shown, Figure 2 a and Figure 2 b are two different weight in-situ calculation modes, Figure 2 c and Figure 2d shows two different hybrid in-situ computing modes. The gray array represents the RRAM array that stores weight data as in-situ data, the black array represents the RRAM array that stores node features as in-situ data, A represents the adjacency matrix, W represents the weight parameter of the graph neural network, X' represents the output node feature value, and T represents transposition. Figure 2 Taking b as an example, A and W are pre-programmed into the resistive memory array ReRAM as static data. When the voltage of the node characteristic X representing the input data is applied to the array row storing W, the array column produces the matrix multiplication result XW of X and W. Then XW is transposed to (XW) T , added as input to the array storing A, and the graph convolution result X' of this layer is obtained. T The whole process is X′= ((AXW) T ) T =((XW) T (A) T ) T The process of hybrid in-situ computing mode is as follows Figure 2 c, the adjacency matrix A (representing the weight in situ) and the node feature X (representing the data in situ) are selected as static data and mapped to the data storage array based on the resistive random access memory; wherein, the transposed weight W is first T Input to storage X T The array is obtained from (XW) T , and then add it as a new input to storage A T The second matrix multiplication is performed on the array rows of , and the output is the calculation result of the convolution layer of the graph as X′ T The whole process is X′ T =((AXW) T ) T =((W T X T )(A) T ), X′ T Then transpose to get the required output node feature X′.

[0041] Here we introduce a specific task-adaptive graph learning calculation mode selection strategy. Before proposing this strategy, the applicant analyzed the graph learning task. Because the two operands of a matrix multiplication cannot be static data at the same time, the matrix calculation order of graph learning limits the selection of static data. The present invention preferably implements graph convolution operations in the calculation order of A×(X×W), because the calculation order of (A×X)×W will lead to a significant increase in the number of operations and more data movement overhead. In the calculation order of A×(X×W), only the mixed in-situ calculation mode of the adjacency matrix A and the input eigenvalue X and the weighted in-situ calculation mode of the adjacency matrix A and the weight W are allowed to be selected simultaneously. In both modes, the adjacency matrix A is static by default, so the focus of the following strategy is to choose whether to map X or W as static data to the data storage array in the (X×W) operation. Since different layers in a graph learning model (graph neural network) may have different characteristics, they need to be configured layer by layer. According to one embodiment of the present invention, a graph neural network calculation method is applied to a graph neural network calculation device based on a resistive memory, including: for each layer of the graph neural network, the relative size of the processing delay of the graph data to be calculated in the resistive memory array in the weight in-situ calculation mode and the hybrid in-situ calculation mode is analyzed layer by layer, and the mode with the smallest delay is selected as the calculation mode of the layer; for the layer that selects the weight in-situ calculation mode, the adjacency matrix of the graph data and the weight parameters of the graph neural network are respectively mapped as in-situ data to different resistive memory arrays, and the node features of the graph neural network are used as input data to calculate with the corresponding in-situ data; for the layer that selects the hybrid in-situ calculation mode, the adjacency matrix and node features of the graph data are respectively mapped as in-situ data to different resistive memory arrays, and the weight parameters are used as input data to calculate with the corresponding in-situ data. Preferably, when performing the weight in-situ calculation mode, the input data is configured to first perform a matrix multiplication operation with the weight parameter, and the result is then matrix multiplied with the adjacency matrix. The technical solution of this embodiment can at least achieve the following beneficial technical effects: The present invention simultaneously supports the hierarchical weighted in-situ calculation mode and the hybrid in-situ calculation mode, and supports the adaptive selection configuration of the in-situ calculation mode according to the processing delay of each layer, so as to efficiently process various graph learning calculation tasks.

[0042] In the task adaptive computing mode selection process based on the RRAM array, layer-by-layer selection is divided into two types: first-layer selection and other-layer selection, because all static data of the first layer are pre-mapped to the RRAM storage array, and the feature update of other layers in the hybrid in-situ computing mode will bring additional RRAM write overhead.

[0043] Since the static data of the first layer will be pre-programmed into the RRAM array before calculation, and there is no need to move data from the main memory, the configuration delay is not considered. In the operation of in-situ calculation, the corresponding calculation mode is adaptively selected according to the delay T of the RRAM in-situ calculation. Specifically, when the matrix A with a size of m×p is multiplied with the matrix B with a size of p×n, the delay T of the in-situ calculation mode of A is A and B in-situ calculation mode delay T B They are defined as follows:

[0044]

[0045]

[0046] Among them, T read (n) is the delay required when the RRAM read input is n bits, Bitwidth(m) is the bit width of m, and DAC_res is the resolution of the digital-to-analog conversion. Therefore, for the (X×W) operation, the comparison weight value W is the in-situ calculation delay under static data. The delay of in-situ calculation with input eigenvalues ​​as static data The following example uses the resolution of the digital-to-analog conversion as 1 bit, which can be converted to calculate a k ×T read (Bitwidth(X k ))-c k ×T read (Bitwidth(W k )). If the difference is positive, it means that T W >T X , in this case, the delay of selecting X as static data is smaller, and the hybrid in-situ calculation mode should be used; if the difference is negative, it means that T W <T X At this time, choosing W as static data has a smaller delay, and the weight in-situ calculation mode should be adopted.

[0047] According to one embodiment of the present invention, the processing delay corresponding to the first layer of the graph neural network only considers the calculation delay. Assuming that the resolution of the digital-to-analog conversion is 1 bit, the calculation mode of the first layer of the graph neural network is determined by the following formula:

[0048] t=a k ×T read (Bitwidth(X k ))-c k ×T read (Bitwidth(W k ));

[0049] Among them, if the value of t is greater than zero, it means that the processing delay of this layer in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected. Otherwise, the weighted in-situ computing mode is selected. When there is only one graph data in this calculation, a k represents the number of nodes or average number of nodes in the kth layer of the graph neural network, T read (Bitwidth(X k ) represents the resistive memory array with the k-th node feature X k The latency required to read the weight value of the kth layer as input data, c k Represents the weight parameter of the kth layer The number of columns of the corresponding dimension matrix c, T read (Bitwidth(W k )) represents the resistive memory array with the kth layer weight parameter W k The delay required to read the node features of the kth layer as the input signal, k = 1. For example, determine the input node features of the graph as i=1,...,m,the weight parameters of the graph neural network are Among them, n is the number of layers of the graph neural network, m is the number of input graphs, and a i , b are the dimensions of the node features, and a is the number of rows in the matrix corresponding to the node features. i and the number of columns b. In the preparation phase, the average number of nodes in multiple graph inputs in the multi-graph task is calculated. For node-type tasks with only one large-scale graph input (i.e., when m=1), this stage can be skipped. That is, when there is only one graph data in this calculation, a k Represents the node features of the kth layer of the graph neural network The number of rows in the corresponding matrix a (equivalent to the number of nodes in the graph data); when there are multiple graph data in this calculation, a k The average number of rows of the matrix representing the node features of multiple graph data in the kth layer of the graph neural network (equivalent to the average number of nodes in multiple graph data). The technical solution of this embodiment can at least achieve the following beneficial technical effects: for each layer of the graph neural network, the processing delay only considers the calculation delay, which helps to reduce the amount of calculation in the calculation mode selection process, improve processing efficiency, and reduce total processing energy consumption.

[0050] For other layers, a parameter β is introduced to measure the write latency overhead caused by node feature updates:

[0051] β=#row×T write ;

[0052] Where #row represents the total number of rows in a single RRAM array, T write represents the single write time of the resistive memory. For other layers, the input feature value is the in-situ calculation delay under static data. So calculate If the value is positive, it means that T W >T X , in this case, the delay of selecting X as static data is smaller, and the hybrid in-situ calculation mode should be used; if the value is negative, it means that T W < T X At this time, the delay of selecting W as static data is smaller, and the weight in-situ calculation mode should be adopted.

[0053] According to one embodiment of the present invention, the processing delay corresponding to other layers of the graph neural network takes into account the calculation delay and the write delay of the node feature update. Assuming that the resolution of the digital-to-analog conversion is 1 bit, the calculation mode of other layers of the graph neural network is determined by the following formula:

[0054] t′=a k ×T read (Bitwidth(X k ))-c k ×T read (Bitwidth(W k ))-β;

[0055] Among them, if the value of t′ is greater than zero, it means that the processing delay of the layer of data in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected. Otherwise, the weighted in-situ computing mode is selected, k≠1, and β represents the write delay required to update the node features. Assuming that the graph neural network is a 3-layer network, then k=2, 3 here. The technical solution of this embodiment can at least achieve the following beneficial technical effects: the processing delay corresponding to other layers of the graph neural network takes into account the calculation delay and the write delay of the node feature update, which helps to consider the corresponding delay according to the write characteristics of the resistive memory, so as to comprehensively select the optimal calculation mode and improve the processing efficiency of the graph data.

[0056] Since the graph data may be sparse, if it is mapped without discrimination, it may not only cause excessive configuration delay, but also waste precious RRAM resources. According to one embodiment of the present invention, when performing corresponding in-situ data mapping, a sparse data mapping method or a dense data mapping method is selected for the in-situ data; wherein, the sparse data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data into multiple sub-graphs, deleting the empty sub-graphs with zero values, and mapping the non-empty sub-graphs to the RRAM array in sequence; the dense data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data directly according to the size of the RRAM array and mapping it.

[0057] Since the adjacency matrix is ​​usually sparse, determining the mapping mode of the adjacency matrix will reduce efficiency. According to an embodiment of the present invention, when performing in-situ data mapping, the adjacency matrix as the in-situ data is mapped to the resistive random access memory array using a sparse data mapping method for storage.

[0058] Since the weight parameters of the graph neural network are usually dense, determining the mapping mode of the weight parameters will reduce efficiency. According to one embodiment of the present invention, when performing in-situ data mapping, the weight parameters as in-situ data are mapped to the resistive memory array using a dense data mapping method for storage.

[0059] Unlike the adjacency matrix and weight parameters, the node features of the first layer of the graph neural network may be sparse or dense. If they are mapped without discrimination, it will greatly affect the computing efficiency and waste computing resources. According to one embodiment of the present invention, for the first layer of the graph neural network, when performing in-situ data mapping, if the sparsity of the node features as in-situ data is greater than the predetermined sparse threshold, a sparse data mapping method is used to map them to the resistive memory array for storage, otherwise, a dense data mapping method is used to map them to the resistive memory array for storage. Preferably, the value range of the predetermined sparse threshold is 80% to 95%. For example, the predetermined sparse threshold is set to 90%. The node features of other layers of the graph neural network are usually dense. According to one embodiment of the present invention, for other layers of the graph neural network, when performing in-situ data mapping, the node features as in-situ data are mapped to the resistive memory array for storage using a dense data mapping method.

[0060] For example, suppose the size of a matrix is ​​256*256, and the size of the data storage calculation array is 128*128;

[0061] Among them, under the dense data mapping method: ① divide the 256*256 matrix into 4 blocks of 128*128 size; ② map them to 4 data storage and computing arrays respectively.

[0062] In the sparse data mapping mode: ① Divide the 256*256 matrix according to the sub-graph size (the sub-graph size is usually 4*4 / 8*8 / 16*16, 4*4 is taken as an example here) to obtain 4096 blocks; ② Remove the blocks in which all elements in the 4096 small blocks are zero values ​​to obtain the remaining non-zero blocks; ③ Map the remaining blocks to the data storage and calculation array in sequence.

[0063] For a graphic illustration, see Figure 3 , the 24×24 matrix and data storage computing array are used for simplified explanation. The rectangle represents a 4×4 matrix subgraph, and the circle represents a 4×4 resistive memory unit. In dense mapping (corresponding to the dense data mapping mode), the 24×24 matrix is ​​directly mapped to the data storage computing array; in sparse mapping (corresponding to the sparse data mapping mode): the blocks in the 24×24 matrix where all elements are zero are removed to obtain the remaining non-zero blocks; the remaining blocks are mapped sequentially (continuously mapped on the array, without leaving zero-valued resistive memory units in the middle) to the matrix.

[0064] According to one embodiment of the present invention, a graph neural network computing device based on resistive memory is provided, comprising a resistive memory array for storing and computing graph data, and the graph neural network computing device is configured to execute the method described in the foregoing embodiment. According to one embodiment of the present invention, see Figure 4 , a graph neural network computing device, including: a data storage computing module, including multiple resistive memory arrays, wherein the resistive memory arrays are used to store in-situ data and complete matrix-vector multiplication thereof with input data.

[0065] Preferably, the data storage computing module is configured as follows:

[0066] Supports at least two computing modes, including:

[0067] The weight in-situ calculation mode, in which the node features as input data are matrix-vector multiplied with the weight parameters and adjacency matrix as in-situ data;

[0068] A hybrid in-situ computation mode, in which the weight parameters as input data are matrix-vector multiplied with the node features and adjacency matrix as in-situ data;

[0069] For each layer of the graph neural network, the mode with the smallest delay is selected as the calculation mode of the layer according to the processing delay of the layer in the corresponding calculation mode. The data storage calculation module includes multiple data storage arrays.

[0070] According to one embodiment of the present invention, the graph neural network computing device further includes: an index control module, a multi-way selection module, a multi-way selection module, a shift accumulation module, an activation function module, a digital-to-analog conversion module, an analog-to-digital conversion module, a register module, a static random access memory and a combination thereof. The resolution of the digital-to-analog conversion mentioned above refers to the resolution of the digital-to-analog conversion module and the analog-to-digital conversion module.

[0071] The index control module is used to store the row and column address selection signals of the resistive memory array corresponding to the original data, and input the corresponding selection signals to the multi-way selection module corresponding to the data storage array during the calculation process.

[0072] Multi-way selection module (corresponding to Figure 4 In the row multiplexing and column multiplexing), according to the row and column addresses of the input selection signal, the corresponding row and column units of the resistive memory array are selected;

[0073] The shift-and-accumulate module is used to shift and accumulate the results obtained by performing matrix-vector multiplication on the input data and the original data to obtain a dot product calculation result.

[0074] The activation function module is used to activate the selected dot product calculation result and perform the calculation of the corresponding neuron response value.

[0075] Digital-to-analog conversion module (not included in Figure 4 Shown, refer to Figure 1 A DAC) is used to convert a digital signal representing input data (input activation value) into a corresponding analog voltage value, which will be loaded on the corresponding row of the resistive memory array.

[0076] Analog-to-digital conversion module (not included in Figure 4 Shown, refer to Figure 1 ADC), which is used to convert the analog current value corresponding to the dot product calculation result obtained by matrix-vector multiplication into a digital value; the dot product calculation result here may be obtained through the activation function module or directly from the shift-accumulation module, depending on whether the activation function module is required to activate the corresponding dot product calculation result in the task corresponding to the corresponding graph neural network; after the last layer of calculation is completed, the result obtained by the analog-to-digital conversion module is the final result data, which represents the embedded representation (feature vector) of each node in the graph data.

[0077] The register module is used to temporarily store input data to be input into the data storage computing module or result data output from the data storage computing module. For example, the register module is used to temporarily store a small amount of input data to be input into the data storage computing device for calculation, or output result data of a matrix multiplication operation that has been completed.

[0078] Static random access memory is used to store graph data, weight parameters of graph neural networks and output result data. For example, static random access memory is used to store a large amount of input data that is about to be input into the data storage computing device for calculation, or output result data that has completed the matrix multiplication operation.

[0079] According to one embodiment of the present invention, the graph neural network computing device may further include a processing unit ( Figure 4 (not shown), the processing unit is used to select an adaptive computing mode for the corresponding layer of the graph neural network, and configure the data storage computing module according to the computing mode and / or control the configured storage computing module to perform the operation of the corresponding graph neural network. The processing unit is used to control the storage computing module to perform the operation of the corresponding graph neural network, the index control module, the multi-way selection module, the multi-way selection module, the shift accumulation module, the activation function module, the digital-to-analog conversion module, the analog-to-digital conversion module, the register module, and the static random access memory to work together to complete the operation of the corresponding graph neural network.

[0080] It can be seen from the above implementation modes that Figure 5, the present invention realizes the process of adaptively selecting strategies according to the tasks corresponding to the graph neural network and the graph data, configures the corresponding in-situ computing mode (mixed in-situ or weighted in-situ) for each layer (1, 2, ..., N layers) of the graph neural network, and performs sparse mapping or dense mapping for the selected in-situ data according to the corresponding mapping mode; thereby improving the processing efficiency of the graph data. In the processing of graph data based on resistive random access memory, the selected in-situ data is statically mapped to the resistive random access memory array. When the input data adds the corresponding voltage signal to the array row, the matrix multiplication in a certain layer of the graph neural network can be completed once. The output signal of the array will pass through the analog-to-digital conversion module, the shift accumulation module and the excitation function module to finally obtain the output of the layer, and temporarily store it in the static random access memory to be used as the node feature of the next layer calculation. From the above scheme, it can be seen that the advantages of the present invention are: the present invention considers two angles of different computing characteristics of graph learning tasks in the selection of computing mode: in-situ computing mode and data mapping strategy, which effectively reduces the data movement energy consumption overhead and storage overhead in combination with specific task characteristics, while reducing computing delays, and will not have any impact on the calculation results of graph learning tasks, thus realizing energy-efficient graph learning task calculations. The present invention supports flexible configuration of in-situ computing modes and mapping methods in hardware, thereby supporting task adaptive selection strategies, and providing hardware implementation for energy-efficient graph learning task calculations.

[0081] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0082] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0083] Computer readable storage medium can be a tangible device that holds and stores instructions used by an instruction execution device. Computer readable storage medium can include, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0084] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A graph neural network computing method for a graph neural network computing device based on resistive random access memory, characterized in that: include: For any layer of the graph neural network, analyze the relative processing delay of the graph data to be calculated in the resistive memory array in the layer under the weighted in-situ calculation mode and the hybrid in-situ calculation mode, and select the mode with the smallest delay as the calculation mode of the layer; In the weight in-situ calculation mode, the adjacency matrix of the graph data and the weight parameters of the graph neural network are respectively mapped as in-situ data to the corresponding resistive memory array for the layer of the graph neural network, so as to use the node features of the graph neural network as input data to perform operations with the corresponding in-situ data; In the hybrid in-situ computing mode, the adjacency matrix and node features of the graph data of the layer of the graph neural network are respectively mapped to the corresponding resistive memory array as in-situ data, so that the weight parameters are used as input data to perform operations with the corresponding in-situ data.

2. The method according to claim 1, characterized in that For the first layer of the graph neural network, its processing delay only considers the computational delay.

3. The method according to claim 2, characterized in that The calculation mode of the first layer of the graph neural network is determined by the following formula: Among them, a k Indicates the number of nodes or average number of nodes in the k-th layer of the graph neural network. Represents the resistive memory array with the k-th layer node feature X k As the delay required to read the weight value of the kth layer when the input data is used, DAC_res represents the resolution of the digital-to-analog conversion, c k Represents the weight parameter of the kth layer The number of columns of the corresponding dimension matrix c, Represents the resistive memory array with the k-th layer weight parameter W k The delay required to read the node features of the kth layer as an input signal, k = 1; Among them, if the value of t is greater than zero, it means that the processing delay of this layer in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected; otherwise, the weighted in-situ computing mode is selected.

4. The method according to claim 1, characterized in that: For other layers of the graph neural network, the processing latency takes into account the computation latency and the write latency of node feature updates.

5. The method according to claim 4, characterized in that The calculation mode of other layers of the graph neural network is determined by the following formula: Among them, a k Indicates the number of nodes or average number of nodes in the k-th layer of the graph neural network. Represents the resistive memory array with the k-th layer node feature X k As the delay required to read the weight value of the kth layer when the input data is used, DAC_res represents the resolution of the digital-to-analog conversion, c k Represents the weight parameter of the kth layer The number of columns of the corresponding dimension matrix c, Represents the resistive memory array with the k-th layer weight parameter W k The latency required to read the node features of the kth layer as the input signal, β represents the write latency required to update the node features, k≠1; Among them, if the value of t′ is greater than zero, it means that the processing delay of the layer data in the hybrid in-situ computing mode is smaller, and the hybrid in-situ computing mode is selected; otherwise, the weighted in-situ computing mode is selected.

6. The method according to any one of claims 1 to 5, characterized in that: Mapping the in-situ data to the corresponding resistive memory array includes a sparse data mapping method and a dense data mapping method; The sparse data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data into multiple sub-graphs, deleting the empty sub-graphs with zero values, and mapping the non-empty sub-graphs to the resistive memory array in sequence; The dense data mapping method refers to dividing the matrix corresponding to the corresponding in-situ data directly according to the size of the resistive random access memory array and mapping it.

7. The method according to claim 6, characterized in that When performing in-situ data mapping, the adjacency matrix as the in-situ data is mapped to the resistive random access memory array for storage in a sparse data mapping manner.

8. The method according to claim 6, characterized in that When performing in-situ data mapping, weight parameters of the in-situ data are mapped to the resistive random access memory array for storage in a dense data mapping manner.

9. The method according to claim 6, characterized in that For the first layer of the graph neural network, when performing in-situ data mapping, if the sparsity of the node features serving as the in-situ data is greater than a predetermined sparsity threshold, a sparse data mapping method is used to map it to the resistive random access memory array for storage; otherwise, a dense data mapping method is used to map it to the resistive random access memory array for storage.

10. The method according to claim 9, characterized in that For other layers of the graph neural network, when performing in-situ data mapping, the node features as the in-situ data are mapped to the resistive memory array for storage using a dense data mapping method.

11. A graph neural network computing device based on resistive memory, comprising a resistive memory array for storing and computing graph data, and a processing unit, characterized in that: The processing unit is configured to execute the method according to any one of claims 1 to 10.

Images