Graph structure data processing method, accelerator, storage medium and program product

Abstract

The invention provides a graph structure data processing method which can be applied to the technical field of artificial intelligence and deep learning. The processing method comprises the steps that a compression feature matrix and a state feature matrix are read, the compression feature matrix is obtained by compressing row position information and numerical value information of at least one element in a sparse feature matrix based on a preset compression strategy, and the state feature matrix represents pre-processed state features of the compression feature matrix; first matrix operation is executed on the compression feature matrix and the weight matrix to obtain a first intermediate matrix, second matrix operation is executed on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, and the first matrix operation comprises parallel operation and delay operation executed by a plurality of gating algorithms based on an accelerator; and obtaining a target matrix according to the first intermediate matrix and the second intermediate matrix so as to perform a target task by using the target matrix. The invention also provides an accelerator, a storage medium and a program product.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and deep learning technology, and in particular to a graph structure data processing method, accelerator, storage medium and program product. Background Art

[0002] Graph structured data is widely used in social network analysis, resource recommendation, knowledge graph, supply chain management, and bioinformatics. In related technologies, graph neural networks are used to capture the topological structure and node features of graph structured data, which can support more complex data analysis tasks. However, the data analysis process in related technologies has high requirements for the system cache and is difficult to adapt to the computing needs of data with different sparsity. Summary of the invention

[0003] In view of the above problems, the present invention provides a graph structure data processing method, accelerator, device, equipment, storage medium and program product.

[0004] According to a first aspect of the present invention, a method for processing graph structured data is provided, which is applied to a graph neural network accelerator, wherein the graph structured data includes a sparse feature matrix corresponding to the graph structured data to be processed, including: reading a compressed feature matrix and a state feature matrix, wherein the compressed feature matrix is ​​obtained by compressing the row position information and numerical information of at least one element in the sparse feature matrix based on a preset compression strategy, and the state feature matrix represents the state characteristics of the compressed feature matrix in previous processing; performing a first matrix operation on the compressed feature matrix and a weight matrix to obtain a first intermediate matrix, and performing a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes parallel operations and delay operations performed based on multiple gating algorithms of the accelerator, and the first intermediate matrix includes multiple parallel intermediate matrices obtained based on parallel operations, and a delayed intermediate matrix obtained based on delay operations; obtaining a target matrix based on the first intermediate matrix and the second intermediate matrix, so as to use the target matrix to perform a target task.

[0005] The second aspect of the present invention provides a graph structure data processing device, including: a reading module, used to read a compressed feature matrix and a state feature matrix, wherein the compressed feature matrix is ​​obtained by compressing the row position information and numerical information of at least one element in a sparse feature matrix based on a preset compression strategy, and the state feature matrix represents the state characteristics of the compressed feature matrix in previous processing; an operation module, used to perform a first matrix operation on the compressed feature matrix and the weight matrix to obtain a first intermediate matrix, and perform a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes a parallel operation and a delayed operation, and the first intermediate matrix includes a parallel intermediate matrix and a delayed intermediate matrix; a task execution module, used to obtain a target matrix based on the first intermediate matrix and the second intermediate matrix, so as to use the target matrix to perform a target task.

[0006] The third aspect of the present invention provides a graph neural network accelerator, comprising: a memory; and a processor, configured to execute the above-mentioned graph structure data processing method according to instructions stored in the memory.

[0007] A fourth aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0008] The fifth aspect of the present invention further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0009] The sixth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0011] Figure 1A An example diagram showing the compression of sparse data using a coordinate list format is shown;

[0012] Figure 1B A schematic diagram showing the structure of a long short-term memory network unit is shown;

[0013] Figure 2 An application scenario diagram of a graph structure data processing method, an accelerator, a storage medium, and a program product according to an embodiment of the present invention is shown;

[0014] Figure 3A flowchart of a method for processing graph structure data according to an embodiment of the present invention is shown;

[0015] Figure 4 An example schematic diagram of a sparse feature matrix compression process according to an embodiment of the present invention is shown;

[0016] Figure 5A An example block diagram of a graph neural network accelerator according to an embodiment of the present invention is shown;

[0017] Figure 5B A schematic diagram showing an example of the hardware architecture of an accelerator according to an embodiment of the present invention is shown;

[0018] Figure 6 It shows a structural block diagram of a graph structure data processing device according to an embodiment of the present invention;

[0019] Figure 7 A block diagram of an electronic device suitable for implementing a graph structure data processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0020] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0021] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0022] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0023] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0024] In the technical solution of the present invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0025] In some examples, when using graph neural network acceleration methods to process sparse data, the processor is required to automatically clear invalid zero values; or use a compression algorithm to eliminate zero values ​​in the data.

[0026] For example, the processor automatically filters non-zero data and only retains valid data for calculation. Although this method can output valid non-zero data, since the input is original sparse data, the output rate will be affected by the calculation process of eliminating zero values, and it does not actually increase the effective rate of non-zero data entering the calculation unit.

[0027] For example, the row address is modified to the end-of-row marker to reduce storage requirements. However, this method does not take into account the special case in sparse big data where all row values ​​are zero. The calculation process will still traverse these rows, resulting in unnecessary computing overhead.

[0028] Figure 1A An example diagram showing the compression of sparse data using a coordinate list format is shown.

[0029] like Figure 1A As shown, the data is compressed using the coordinate list format 100A, and then the compressed data is sent to the processor for calculation. Although this compression method can eliminate zero values ​​in the data, it will also increase the row index 101 and column index 102 of non-zero data. The index values ​​of the row index 101 and the column index 102 occupy a large storage space, and the storage overhead of the index value is multiple times that of the data itself. If the data (value) 103 is one byte, for the address index of a very large amount of data, generally 4 bytes are required, and the total size of the row and column items is 8 bytes corresponding to one byte of data, and the storage space occupied is very large.

[0030] In some examples, for the gating algorithm of the long short-term memory network, some calculation results need to be cached during the calculation process, and the next step of data can be calculated after all the corresponding calculation data are completed.

[0031] Figure 1BA schematic diagram of the structure of a long short-term memory network unit is shown.

[0032] like Figure 1B As shown, combined with the gating algorithm formula of the long short-term memory network, input h (t-1) After that, we can calculate i synchronously t , f t , g t , o t , but in calculating h t When c t , and then calculate tanh(c t ), which is consistent with the previously calculated o t Calculate h t In the related art, by t The value remains in the cache, waiting for tanh(c t ). If the amount of data is small, caching is usually acceptable, but for the intermediate results of matrix calculations of graph neural networks with large amounts of data, the demand for storage resources is large and will increase with the increase of computational parallelism.

[0033] Based on the above problems, the present invention provides a graph structure data processing method, accelerator, device, equipment, storage medium and program product, which are applied to a graph neural network accelerator, wherein the graph structure data includes a sparse feature matrix corresponding to the graph structure data to be processed, including: reading a compressed feature matrix and a state feature matrix, wherein the compressed feature matrix is ​​obtained by compressing the row position information and numerical information of at least one element in the sparse feature matrix based on a preset compression strategy, and the state feature matrix represents the state characteristics of the compressed feature matrix in the previous processing; performing a first matrix operation on the compressed feature matrix and the weight matrix to obtain a first intermediate matrix, and performing a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes a parallel operation and a delayed operation, and the first intermediate matrix includes a parallel intermediate matrix and a delayed intermediate matrix; obtaining a target matrix according to the first intermediate matrix and the second intermediate matrix, so as to use the target matrix to perform a target task.

[0034] According to an embodiment of the present invention, a compressed feature matrix is ​​obtained by compressing the row position information and numerical information of the elements in the sparse feature matrix, thereby performing matrix operations on the compressed compressed feature matrix, weight matrix and state feature matrix. The operation process fully considers the sparsity of the sparse feature matrix and the density of other matrices (including the weight matrix, the state feature matrix, and the first intermediate matrix) in the graph structure data, and is compatible with the calculation characteristics of sparse matrices and dense matrices. Since the first intermediate matrix is ​​obtained by performing parallel operations and delayed operations on the compressed feature matrix and the weight matrix based on multiple gating algorithms of the accelerator, the first intermediate matrix is ​​dynamically generated according to demand, realizing cacheless pipeline calculation, avoiding the occupation of storage space by intermediate results, and further improving the calculation efficiency and utilization of system resources.

[0035] Figure 2 An application scenario diagram of a graph structure data processing method, an accelerator, a storage medium, and a program product according to an embodiment of the present invention is shown.

[0036] like Figure 2 As shown, the application scenario according to this embodiment may include a terminal device 201, a server 202, an external storage unit 203, and a network 204. The network 204 is used to provide a medium for a communication link between the terminal device 201, the server 202, and the external storage unit 203. The network 204 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0037] The user can use the terminal device 201 to interact with the server 202 through the network 204 to receive or send messages, etc. The terminal device 201 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, etc.

[0038] The server 202 can be used to process the graph structure data. For example, it can read different types of feature matrices from the external storage unit 203, perform matrix operations on different matrices according to the operation instructions, and send the operation results to the external storage unit 203 for storage, or display the operation results through the terminal device 201. For example, the server 202 can recommend resources to the user based on the operation results, or analyze the relevant physical examination data to achieve the classification and prediction of the cause of the disease.

[0039] It should be noted that the graph structure data processing method provided in the embodiment of the present invention can generally be executed by the server 202. Accordingly, the graph structure data processing device provided in the embodiment of the present invention can generally be set in the server 202. The graph structure data processing method provided in the embodiment of the present invention can also be executed by a server or server cluster that is different from the server 202 and can communicate with the server 202. Accordingly, the graph structure data processing device provided in the embodiment of the present invention can also be set in a server or server cluster that is different from the server 202 and can communicate with the server 202.

[0040] It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Any number of terminal devices, external storage units, networks and servers may be provided according to implementation requirements.

[0041] Figure 3 A flowchart of a method for processing graph structured data according to an embodiment of the present invention is shown.

[0042] like Figure 3 As shown, the graph structure data processing method of this embodiment is applied to a graph neural network accelerator, the graph structure data includes a sparse feature matrix corresponding to the graph structure data to be processed, and the graph structure data processing method includes operations S310 to S330.

[0043] In operation S310, a compression feature matrix and a state feature matrix are read, wherein the compression feature matrix is ​​obtained by compressing row position information and value information of at least one element in the sparse feature matrix based on a preset compression strategy, and the state feature matrix represents the state characteristics of the compression feature matrix before processing.

[0044] In an embodiment of the present invention, the external storage unit may be a memory corresponding to the accelerator for storing data to be processed or processed result data (including but not limited to graph structure data, compressed feature matrix and state feature matrix), and the accelerator may be a processor that processes the graph data structure using a preset algorithm. The row position information of an element may characterize the position of the element in the current row information, including the end-of-row position or the non-end-of-row position. The numerical information may characterize that the element is a zero value or a non-zero value. The state characteristics of the previous processing may characterize the state characteristics obtained by processing the previous cycle corresponding to the current calculation cycle in the graph structure data processing method.

[0045] In operation S320, a first matrix operation is performed on the compression feature matrix and the weight matrix to obtain a first intermediate matrix, and a second matrix operation is performed on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes a parallel operation and a delayed operation, and the first intermediate matrix includes a parallel intermediate matrix and a delayed intermediate matrix.

[0046] In an embodiment of the present invention, the processing method of graph structure data can be a graph data processing algorithm based on a long short-term memory network (Long Short-Term Memory, LSTM), and the gating algorithm and the related state algorithm in the long short-term memory network perform corresponding matrix operations on different matrices. The gating algorithm can be multiple, and different intermediate matrices can be obtained according to different gating algorithms. The related state algorithm can include a memory unit state algorithm and a hidden state algorithm.

[0047] For example, based on the multiple gating algorithms of the accelerator and the operation duration information corresponding to the gating algorithms, the compressed feature matrix and the weight matrix can be respectively parallelized and delayed at different times according to the cacheless pipeline computing requirements to obtain a first intermediate matrix. For example, based on the memory unit state algorithm, a second matrix operation is performed on the state feature matrix and the obtained first intermediate matrix to obtain a second intermediate matrix.

[0048] In operation S330, a target matrix is ​​obtained according to the first intermediate matrix and the second intermediate matrix, so as to perform a target task using the target matrix.

[0049] In an embodiment of the present invention, the target matrix may be a final output result matrix obtained after performing multiple cycles of operations. After obtaining the first intermediate matrix and the second intermediate matrix, the activation function may be used to perform a matrix multiplication operation on the first intermediate matrix and the second intermediate matrix to obtain a target matrix. After obtaining the target matrix, resource recommendations, event predictions, or relationship reasoning may be performed on the target object according to the target matrix based on actual needs.

[0050] For example, by using graph structure data processing methods, the interaction sequence between users and items and the social relationships between users can be analyzed to achieve more accurate resource recommendations. For example, by analyzing the time series characteristics of user behavior and extracting the relationship structure between users with graph neural networks, user behavior patterns can be predicted or potential friends can be recommended. For example, by analyzing the temporal dynamic changes of biological sequence data (such as gene sequences) and combining graph neural networks, the structural information of biological networks (such as protein interaction networks) can be processed, and hidden patterns in biological networks can be analyzed for disease prediction and drug discovery.

[0051] According to an embodiment of the present invention, a compressed feature matrix is ​​obtained by compressing the row position information and numerical information of the elements in the sparse feature matrix, thereby performing matrix operations on the compressed compressed feature matrix, weight matrix and state feature matrix. The operation process fully considers the sparsity of the sparse feature matrix and the density of other matrices (including the weight matrix, the state feature matrix, and the first intermediate matrix) in the graph structure data, and is compatible with the calculation characteristics of sparse matrices and dense matrices. Since the first intermediate matrix is ​​obtained by performing parallel operations and delayed operations on the compressed feature matrix and the weight matrix based on multiple gating algorithms of the accelerator, the first intermediate matrix is ​​dynamically generated according to demand, realizing cacheless pipeline calculation, avoiding the occupation of storage space by intermediate results, and further improving the calculation efficiency and utilization of system resources.

[0052] It can be understood that the above has already described how to perform graph structure data processing methods, and the following will describe how to compress sparse feature matrices.

[0053] According to an embodiment of the present invention, the feature of the element includes at least one of an all-zero row feature and a non-all-zero row feature, and the non-all-zero row feature includes at least one non-zero element; the processing method also includes: when the feature of the current element is an all-zero row feature or a non-all-zero row feature, determining the label value corresponding to the all-zero row feature based on the label correspondence relationship, or determining the label value corresponding to the non-zero element, the label correspondence relationship characterizing the relationship between multiple label values ​​and the all-zero row feature and the non-zero element.

[0054] In the embodiment of the present invention, considering the uncertain length of the input data of the graph neural network, the large amount of data for a single matrix calculation, and the irregularity of data sparsity, the matrix multiplication in the LSTM is performed in a row multiplication manner. For example, taking a row (such as L1) in the sparse feature matrix with only two non-zero elements as an example, the first non-zero element a in L1 and the corresponding weight row W in the weight matrix (such as W) are l1 (including multiple weight values) perform multiplication and addition operations, and accumulate the second non-zero element b and the corresponding weight row W l2 The multiplication and addition results can be used to calculate the first row of data of the final data result. In this way, there is no limit on the length of the sparse feature matrix to be input, and only non-zero data needs to be screened out for effective calculation.

[0055] In the related art, before performing row multiplication on a sparse feature matrix, it is necessary to extract valid data (non-zero data) in the sparse feature matrix (for example, automatically filter by a processor or compress data by compression software), but these methods are difficult to effectively solve the problem of storage space occupation. In order to solve this technical problem, according to the characteristics of row multiplication, the present invention can obtain a row of valid data of the output result after performing a complete row multiplication calculation each time. Therefore, for the row index value (label value), it is sufficient to determine which data is the last data of the current row.

[0056] In an embodiment of the present invention, each element in the sparse feature matrix may include an all-zero row feature and a non-all-zero row feature based on the case where the value is zero or non-zero. The all-zero row feature may characterize the feature that all elements in the row corresponding to the target element are zero elements, and the non-all-zero row feature may characterize the feature that the elements in the row corresponding to the target element include both zero elements and non-zero elements. The representation method of the label value may be determined according to the actual situation, for example, using bits to represent the all-zero row feature and the non-all-zero row feature of the element.

[0057] According to an embodiment of the present invention, in the characteristics of matrix-matrix row multiplication, by recording the index of the last data in each row, the boundary of each row can be directly determined, avoiding additional traversal operations, and making row operations on sparse matrices more efficient, especially in the calculation of large-scale sparse matrices. In matrix multiplication, only the non-zero elements of each row need to be processed, avoiding meaningless calculations of zero elements, reducing the amount of calculations, and improving calculation efficiency.

[0058] According to an embodiment of the present invention, a label value includes a first label value and a second label value; determining a label value corresponding to an all-zero row feature or determining a label value corresponding to a non-zero element based on a label correspondence relationship includes: when the feature of a current element is an all-zero row feature, determining that the label value corresponding to the all-zero row feature is a second label value; when the feature of the current element is a non-all-zero row feature, determining that the label value corresponding to the non-zero element is a first label value based on numerical information and row position information.

[0059] In an embodiment of the present invention, the first label value may be a label value corresponding to a non-zero element in a non-all-zero row feature; the second label value may be a label value of a row corresponding to an all-zero row feature. For example, 0 represents a non-all-zero row, and 1 represents an all-zero row; if the current row is an all-zero row, no calculation is required, but a row of all-zero data (expressed as 10) needs to be added during matrix operations; for non-all-zero rows, two bits can be used to represent them, the first bit, 0 represents a non-all-zero row, 1 represents an all-zero row, and the second bit, 0 represents a non-last data in the row, and 1 represents the last data in the row.

[0060] According to an embodiment of the present invention, the method further includes: processing zero elements in non-all-zero row features, and determining a compressed feature matrix based on the first label value and the second label value.

[0061] In an embodiment of the present invention, considering that there is a resource overhead problem of traversing the zero elements in the non-all-zero row features during the calculation process, the present invention updates the zero elements in the non-all-zero row features, and then compresses the sparse matrix based on different label values ​​to obtain a compressed feature matrix.

[0062] According to an embodiment of the present invention, zero elements in non-all-zero row features are processed, and a compressed feature matrix is ​​determined based on a first label value and a second label value, including: when the feature of the current element is a non-all-zero row feature, deleting the zero elements in the current row; converting the non-zero elements into the first label value and converting the non-all-zero row features into the second label value in sequence to obtain a compressed feature matrix.

[0063] Figure 4 An example schematic diagram of a sparse feature matrix compression process according to an embodiment of the present invention is shown.

[0064] like Figure 4 As shown, in 400, the row information in the original sparse feature matrix 401 can be updated from the original four bytes to two bits in the compressed feature matrix 402, where the left bit, 0 represents a non-all-zero row, and 1 represents an all-zero row, which is compatible with the case of all-zero rows in the sparse feature matrix; the right bit, 0 represents the last element of a non-current row, and 1 represents the last element of the current row.

[0065] When performing row multiplication and accumulation operations on a row of data, if the element is not the last element of the current row, the operation and accumulation operations continue. When the last element of the current row is identified, the result of the accumulation operation is output as a row of the final output result, and the accumulation buffer is cleared, and the row multiplication operation of the next row is performed. In this way, the storage space of the index value of the row after the compressed data can be compressed to 1 / 16 of the original, and the bit width of the row data will not increase with the increase of the data volume, indicating 2 bits.

[0066] According to an embodiment of the present invention, by using two bits to represent the row information in the compressed feature matrix, the last element of each row can be quickly determined, so that only necessary valid information needs to be processed in the row multiplication operation, which is compatible with the operation requirements of sparse matrices and dense matrices of graph structure data. By reducing unnecessary data storage and operations, the efficiency and performance of matrix operations are further improved.

[0067] In an embodiment of the present invention, the calculation unit structure of matrix multiplication may include multiple multipliers and adders. Matrix operations may be performed by a hardware accelerator (e.g., a field programmable gate array). For example, by inputting the data to be input into the multiplier for multiplication, the multiplier corresponds to a row or a column of the matrix, and multiple multiplication operations may be performed simultaneously through parallel work, and each multiplier may receive an element from a different matrix for multiplication; thereby, the adder is used to accumulate the results from the corresponding multiplier, and multiple adders may be operated in parallel to process multiple accumulation operations simultaneously. Through the multiplication and addition operations of the matrix elements, only one element in the sparse matrix and one row in the corresponding weight matrix are multiplied each time, and accumulation is performed until a row of data in the sparse feature matrix is ​​calculated. After the end-of-row flag of the current element is detected, a row of data of the matrix that has been calculated can be represented and output, and the cache in the calculation unit is cleared.

[0068] According to an embodiment of the present invention, by Figure 4 The compression method shown and the above-mentioned computing unit structure can use the compressed feature matrix without decompression, and directly read and perform matrix multiplication and addition operations from the memory (such as double data rate synchronous dynamic random access memory) in sequence. Since the data has no line wrapping and selective reading, the read and write bandwidth of the memory can be maximized, and the data only needs to be read once to complete all matrix calculations, thereby reducing the number of memory accesses and further improving the efficiency of data processing and calculation.

[0069] It can be understood that the above has described how to compress the sparse feature matrix, and the following will describe how to read the feature matrix.

[0070] According to an embodiment of the present invention, reading a compressed feature matrix and a state feature matrix includes: based on storage address information, sequentially reading elements in the compressed feature matrix from a first storage unit, and obtaining a target element and target row information when the read label value is an end-of-row label value, the end-of-row label value being used to characterize that the element is the last non-zero element in the target row information; and reading a state feature matrix from a second storage unit based on storage address information and a mapping relationship, the mapping relationship characterizing a correspondence between state features in the state feature matrix and the target element.

[0071] In an embodiment of the present invention, the first storage unit can be used to store graph structure data, label values ​​corresponding to elements, and compressed feature matrices after compression. The intermediate matrix obtained during the matrix operation of the compressed feature matrix can be stored in the second storage unit. For example, the second storage unit can be used to store a state feature matrix corresponding to the LSTM algorithm, which may include a memory cell state matrix and a hidden state matrix.

[0072] For example, based on the storage address information, sequentially reading the elements in the compressed feature matrix from the first storage unit may include: setting an index or pointer in the first storage unit to point to the starting position of the data pointer array; sequentially reading the values ​​of non-zero elements according to the address in the data pointer array, and detecting whether each element is an end-of-row tag value when reading each element. The end-of-row tag value is usually a specific mark (such as 1) used to indicate the end of a row; when the end-of-row tag value is detected, the value of the current element and the row information in which it is located are recorded.

[0073] According to an embodiment of the present invention, by reading elements in a compressed feature matrix based on storage address information and obtaining information about the current element and its row when the end-of-row tag value is read, large-scale sparse data can be effectively processed and analyzed.

[0074] According to an embodiment of the present invention, the processing method also includes: when the target matrix is ​​obtained, clearing the operation cache corresponding to the first matrix operation and the second matrix operation in the internal storage unit of the accelerator, and performing the matrix operation of the next cycle, the internal storage unit being an on-chip storage unit of the accelerator.

[0075] In an embodiment of the present invention, after completing a matrix operation cycle, some intermediate calculation results and temporary data exist in the internal storage unit of the processor. The method of clearing the operation cache corresponding to the first matrix operation and the second matrix operation in the internal storage unit of the accelerator can be specifically determined according to actual conditions.

[0076] For example, the address space of the internal storage unit can be reset or cleared, or new data can be directly written to overwrite the old data before the start of a new operation cycle, or a specific instruction can be sent to the accelerator to instruct it to clear the internal storage unit so as to perform the matrix operation of the next cycle. After clearing the old operation cache, the new matrix data can be loaded into the internal storage unit, and the accelerator operates on the new matrix data according to a preset algorithm (such as matrix multiplication operation).

[0077] According to an embodiment of the present invention, the state feature matrix includes a hidden state matrix, the algorithm of the accelerator is a gating algorithm, the gating algorithm includes a first gating algorithm and a second gating algorithm; a first matrix operation is performed on the compressed feature matrix and the weight matrix to obtain a first intermediate matrix, and a second matrix operation is performed on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, including: determining a first moment for parallel operation, and determining a second moment for second matrix operation based on the first moment, and determining a third moment for delayed operation based on the second moment; at the first moment, based on the first gating algorithm, parallel operations are performed on the compressed feature matrix and the weight matrix to obtain multiple parallel intermediate matrices; at the second moment, multiple parallel intermediate matrices and the hidden state matrix are used to obtain the second intermediate matrix, and at the third moment, based on the second gating algorithm, a delay operation is performed on the compressed feature matrix and the weight matrix to obtain a delayed intermediate matrix, so that the moment information of obtaining the second intermediate matrix is ​​the same as the moment information of obtaining the delayed intermediate matrix.

[0078] In an embodiment of the present invention, considering the gating algorithm based on the long short-term memory network, the matrix operation needs to be calculated in a certain order and cannot be fully parallelized. To solve this technical problem, the present invention combines the gating algorithm of the long short-term memory network, the calculation characteristics of row multiplication, and the advantage of directly using the compressed feature matrix without decoding, and realizes corresponding matrix operations at multiple times by using a single data delay unit, so as to achieve the time alignment of the intermediate calculation results of the matrix operation, and the conditions for calculating the next step of calculation data can be calculated without caching.

[0079] In an embodiment of the present invention, a shift register can be used based on a long short-term memory network computing structure to control the input delay of data so that the time information of different intermediate matrix operation results is the same, so as to avoid using other data cache units. The method of implementing single data input delay may include: determining multiple time information for different matrix operations, including a first time for parallel operation and a second time for second matrix operation, so that a third time for delayed operation can be determined based on the second time. After determining different times, the input delay of data can be achieved by always controlling, serial input and output, cascading, resetting and enabling control.

[0080] For example, the clock frequency of the shift register can be adjusted through the clock signal to control the speed of data movement, thereby achieving control of input delay. Or by introducing data bits one by one at the data input end, data can be shifted into the register at the required time, thereby achieving precise control of the timing of data input. For example, through reset control, the register content can be cleared at the target time, while enable control can start or stop data shifting when needed.

[0081] According to an embodiment of the present invention, by controlling the input delay of a single data to time-align the intermediate results, data pipeline calculation can be ensured without caching the intermediate result matrix, thereby improving the computing efficiency and reducing the overall cache requirements, and the required delay unit will not increase resource loss as the computing parallelism increases.

[0082] According to an embodiment of the present invention, the first gating algorithm includes a first row multiplication operation and a first addition operation, the weight matrix includes weight row information corresponding to non-zero elements, and the weight row information includes multiple weight values; at a first moment, the first gating algorithm is used to perform parallel operations on the compressed feature matrix and the weight matrix to obtain multiple parallel intermediate matrices, including: at a first moment, a first row multiplication operation is performed on at least one non-zero element and the weight row information to obtain multiple first intermediate eigenvalues; a first addition operation is performed on the multiple first intermediate eigenvalues ​​to obtain multiple parallel intermediate matrices.

[0083] In an embodiment of the present invention, the first gating algorithm may include an input gate algorithm, a forget gate algorithm, and a related state algorithm in a long short-term memory network. At the first moment, a first row multiplication operation may be performed on the non-zero elements in the compressed feature matrix and the weight information to obtain a plurality of first intermediate eigenvalues, and then a first addition operation may be performed on the plurality of first intermediate eigenvalues ​​to obtain a plurality of parallel intermediate matrices. For example, for the input gate algorithm, at time t, h (t−1) With weight information W hi Perform multiplication and add x t With weight information W ii Perform the first multiplication operation to obtain multiple first intermediate eigenvalues, based on which W ii x t , W hi h (t−1) Perform the first addition operation with other intermediate eigenvalues ​​or dense feature matrices to obtain the intermediate input matrix i t At the same time, based on the corresponding algorithms, the intermediate forgetting matrix f corresponding to the forget gate algorithm and the candidate cell state algorithm can be obtained t and the intermediate state matrix g t , and then obtain the parallel intermediate matrix.

[0084] According to an embodiment of the present invention, a parallel intermediate matrix includes an intermediate input matrix, an intermediate forgetting matrix and an intermediate state matrix; at a second moment, a second intermediate matrix is ​​determined using multiple parallel intermediate matrices and a hidden state matrix, including: performing multiplication operations on the intermediate input matrix, the intermediate forgetting matrix, the intermediate state matrix and the hidden state matrix at the second moment to obtain a second intermediate matrix.

[0085] In an embodiment of the present invention, the second moment may be the same moment as a moment when the parallel intermediate matrix is ​​calculated. After obtaining the intermediate input matrix, the intermediate forgetting matrix, and the intermediate state matrix, the intermediate input matrix and the intermediate state matrix may be subjected to matrix multiplication to obtain a first result, and the read hidden state matrix and the intermediate forgetting matrix may be subjected to matrix multiplication to obtain a second result, thereby performing matrix addition on the first result and the second result to obtain a second intermediate matrix.

[0086] For example, the intermediate input matrix i t and the intermediate state matrix g t Perform matrix multiplication to obtain the first result i t g t , and the hidden state matrix c (t−1) and the intermediate forgetting matrix f t Perform matrix multiplication to obtain the second result f t c (t−1) .

[0087] According to an embodiment of the present invention, the second gating algorithm includes a second row multiplication operation and a second addition operation; at a third moment, a delay operation is performed on the compressed feature matrix and the weight matrix based on the second gating algorithm to obtain a delayed intermediate matrix, including: at the third moment, a second row multiplication operation is performed on at least one non-zero element and the weight row information to obtain multiple second intermediate eigenvalues; a second addition operation is performed on the multiple second intermediate eigenvalues ​​to obtain a delayed intermediate matrix.

[0088] In an embodiment of the present invention, the second gating algorithm may represent an output gate algorithm. After performing a second row multiplication operation on the non-zero elements and the weight row information through the output gate algorithm and the single data delay unit to obtain a plurality of second intermediate eigenvalues, a second addition operation is performed to obtain a delayed intermediate matrix, so that the time when the delayed intermediate matrix is ​​obtained is the same as the time when the second intermediate matrix is ​​obtained.

[0089] According to an embodiment of the present invention, by achieving the same time as the second intermediate matrix when the delayed intermediate matrix is ​​obtained based on a single data delay unit based on a gating algorithm, the time synchronization of data when flowing between computing units can be ensured, thereby reducing waiting time and improving overall computing efficiency. Since no additional cache is required to store intermediate results, the demand for storage resources can be reduced, thereby reducing the cost and power consumption of the system. The single data delay unit helps to optimize the data flow, making the data smoother during the calculation process and reducing the bottleneck of data transmission. This design makes the system easier to expand, and each computing unit can process data independently without relying on a centralized cache mechanism. By reducing the reliance on the cache, system errors caused by cache consistency problems can be reduced, further improving the reliability of the system.

[0090] According to an embodiment of the present invention, a graph neural network accelerator includes a computing array, which includes a plurality of computing units arranged in a one-dimensional direction, and the number of computing units is greater than or equal to the number of columns contained in a weight matrix.

[0091] In an embodiment of the present invention, considering the fact that the compressed feature matrix in the present invention has a small restriction on the length of the matrix during the matrix operation, it is only necessary to meet the storage requirements of the weight matrix corresponding to the non-zero elements. By setting a plurality of operation units in the accelerator according to actual conditions, the operation units are arranged in a one-dimensional direction, and the number of operation units is designed to be greater than or equal to the number of columns contained in the weight matrix, it can be ensured that when performing matrix multiplication operations, the operation array can process all columns of the weight matrix at one time, thereby improving the calculation efficiency.

[0092] In a feasible embodiment, the graph structure data processing method may also include: storing the weight matrix in an internal memory by column; reading the target weight at the target position in the column information stored in the internal memory in parallel, and determining the weight row corresponding to the feature.

[0093] According to an embodiment of the present invention, by reusing the weight matrix on the processor, the reuse of the weight matrix is ​​maximized, the number of memory accesses can be reduced, the reuse rate of data can be improved, and power consumption can be greatly reduced. At the same time, the reuse of the weight matrix helps to optimize the data flow, making the data smoother during the calculation process and reducing the bottleneck of data transmission.

[0094] Figure 5A An example block diagram of a graph neural network accelerator according to an embodiment of the present invention is shown.

[0095] like Figure 5A As shown, the graph neural network accelerator 500A of this embodiment includes: a memory 51; a processor 52, configured to execute the above-mentioned graph structure data processing method according to the instructions stored in the memory 51.

[0096] The memory 51 may store different types of instructions and data. The instructions may include read instructions, data input delay instructions, and operation instructions. The data may include graph structure data, sparse feature matrices, compressed feature matrices, state feature matrices, and target matrices obtained through operations.

[0097] The processor 52 may include a processing unit 521 and a storage unit 522. The processing unit 521 is used to perform corresponding data processing according to various instructions. The storage unit 522 may be an on-chip storage unit integrated on the processing unit 521. The storage unit 522 may also be used to store weight matrices and parameters of graph neural network models.

[0098] For example, the processor 52 may execute a graph structure data processing method according to instructions stored in the memory 51 , such as the processor 52 performing matrix operations using multipliers and adders based on a gating algorithm.

[0099] Figure 5B A schematic diagram showing an example of the hardware architecture of an accelerator according to an embodiment of the present invention is shown.

[0100] like Figure 5B As shown, in 500B, the processor 52 inputs a delay instruction according to the data stored in the storage unit 522, and sends the first operation instruction to the forget gate operation unit 5211, the input gate operation unit 5212 and the first state operation unit 5213 in the processing unit 521 based on the first gating algorithm at the first moment; sends the second operation instruction to the second state operation unit 5214 in the processing unit 521 at the second moment, and sends the third operation instruction to the output gate operation unit in the processing unit 521 based on the delay unit 523 at the third moment based on the second gating algorithm, so that the intermediate operation result is obtained at the same moment for the calculation of the third state operation unit 5215, and the operation result is obtained, and the operation result is output to the double data rate memory (Double Data Rate SDRAM, DDR). According to actual needs, the storage unit 522 may include multiple storage units that meet different transmission rates, such as storage unit 1, storage unit 2, storage unit 3 and storage unit 4.

[0101] According to the embodiments of the present disclosure, according to the data input delay instruction stored in the storage unit 522, the operation process meets the sequential calculation requirements of the matrix operation, and there is no need to use the internal cache, and the cacheless pipeline calculation is directly implemented to avoid the calculation delay. The graph neural network accelerator can efficiently execute the graph structure data processing method through the collaborative work of the memory 51 and the processor 52, thereby realizing the accelerated operation of the graph neural network.

[0102] Based on the above graph structure data processing method, the present invention also provides a graph structure data processing device. Figure 6 The device is described in detail.

[0103] Figure 6 A structural block diagram of a graph structure data processing device according to an embodiment of the present invention is shown.

[0104] like Figure 6 As shown, the graph structure data processing device 600 of this embodiment includes a reading module 610 , a calculation module 620 and a task execution module 630 .

[0105] The reading module 610 is used to read the compressed feature matrix and the state feature matrix, wherein the compressed feature matrix is ​​obtained by compressing the row position information and the value information of at least one element in the sparse feature matrix based on a preset compression strategy, and the state feature matrix represents the state characteristics of the compressed feature matrix in the previous processing. In one embodiment, the reading module 610 can be used to perform the operation S210 described above, which will not be repeated here.

[0106] The operation module 620 is used to perform a first matrix operation on the compression feature matrix and the weight matrix to obtain a first intermediate matrix, and to perform a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes a parallel operation and a delay operation, and the first intermediate matrix includes a parallel intermediate matrix and a delay intermediate matrix. In one embodiment, the operation module 620 can be used to perform the operation S220 described above, which will not be repeated here.

[0107] The task execution module 630 is used to obtain a target matrix according to the first intermediate matrix and the second intermediate matrix so as to execute the target task using the target matrix. In one embodiment, the task execution module 630 can be used to execute the operation S230 described above, which will not be described in detail here.

[0108] According to an embodiment of the present invention, based on the reading module 610, the operation module 620 and the task execution module 630 in the graph structure data processing device, the compressed feature matrix is ​​obtained by compressing the row position information and numerical information of the elements in the sparse feature matrix, thereby performing matrix operations on the compressed compressed feature matrix, the weight matrix and the state feature matrix. The operation process fully considers the sparsity of the sparse feature matrix in the graph structure data and the density of other matrices (including the weight matrix, the state feature matrix, and the first intermediate matrix), and is compatible with the calculation characteristics of sparse matrices and dense matrices. Since the first intermediate matrix is ​​obtained by performing parallel operations and delayed operations on the compressed feature matrix and the weight matrix based on multiple gating algorithms of the accelerator, the first intermediate matrix is ​​dynamically generated according to demand, realizing cache-free pipeline calculation, avoiding the occupation of storage space by intermediate results, and further improving the calculation efficiency and utilization of system resources.

[0109] According to an embodiment of the present invention, the feature of the element includes at least one of an all-zero row feature and a non-all-zero row feature, and the non-all-zero row feature includes at least one non-zero element; the device also includes: a label value determination module, which is used to determine the label value corresponding to the all-zero row feature based on the label correspondence relationship, or determine the label value corresponding to the non-zero element when the feature of the current element is an all-zero row feature or a non-all-zero row feature, and the label correspondence relationship characterizes the relationship between multiple label values ​​and the all-zero row feature and the non-zero element.

[0110] According to an embodiment of the present invention, a label value includes a first label value and a second label value; and a label value determination module includes: a second label value determination submodule and a first label value determination submodule. The second label value determination submodule is used to determine that the label value corresponding to the all-zero row feature is the second label value when the feature of the current element is an all-zero row feature; and the first label value determination submodule is used to determine that the label value corresponding to the non-zero element is the first label value based on the numerical information and the row position information when the feature of the current element is a non-all-zero row feature.

[0111] According to an embodiment of the present invention, the device further includes: a zero element processing module, configured to process zero elements in non-all-zero row features, and determine a compressed feature matrix based on the first label value and the second label value.

[0112] According to an embodiment of the present invention, the zero element processing module includes: an element deletion submodule and an element conversion submodule. The element deletion submodule is used to delete the zero elements in the current row when the feature of the current element is a non-all-zero row feature; the element conversion submodule is used to convert the non-zero elements into the first label value and the non-all-zero row feature into the second label value in sequence to obtain a compressed feature matrix.

[0113] According to an embodiment of the present invention, the reading module 610 includes: an element reading submodule and a matrix reading submodule. The element reading submodule is used to read the elements in the compressed feature matrix from the first storage unit in sequence based on the storage address information, and obtain the target element and the target row information when the read label value is the end-of-row label value, and the end-of-row label value is used to indicate that the element is the last non-zero element in the target row information; the matrix reading submodule is used to read the state feature matrix from the second storage unit based on the storage address information and the mapping relationship, and the mapping relationship indicates the corresponding relationship between the state features in the state feature matrix and the target element.

[0114] According to an embodiment of the present invention, the processing device also includes: a clearing module, which is used to clear the operation cache corresponding to the first matrix operation and the second matrix operation in the internal storage unit of the accelerator when the target matrix is ​​obtained, and perform the matrix operation of the next cycle, and the internal storage unit is an on-chip storage unit of the accelerator.

[0115] According to an embodiment of the present invention, the state feature matrix includes a hidden state matrix, the algorithm of the accelerator is a gating algorithm, and the gating algorithm includes a first gating algorithm and a second gating algorithm; the operation module 620 includes: a time determination submodule, used to determine a first time for parallel operation, and determine a second time for second matrix operation based on the first time, and determine a third time for delay operation based on the second time; a parallel operation submodule, used to perform parallel operation on the compressed feature matrix and the weight matrix based on the first gating algorithm at the first time to obtain multiple parallel intermediate matrices; a delay operation submodule, used to obtain the second intermediate matrix using the multiple parallel intermediate matrices and the hidden state matrix at the second time, and perform delay operation on the compressed feature matrix and the weight matrix based on the second gating algorithm at the third time to obtain a delayed intermediate matrix, so that the time information of obtaining the second intermediate matrix is ​​the same as the time information of obtaining the delayed intermediate matrix.

[0116] According to an embodiment of the present invention, the first gating algorithm includes a first row multiplication operation and a first addition operation, the weight matrix includes weight row information corresponding to non-zero elements, and the weight row information includes multiple weight values; the parallel operation submodule includes: a row multiplication operation unit and an addition operation unit. The row multiplication operation unit is used to perform a first row multiplication operation on at least one non-zero element and the weight row information at a first moment to obtain multiple first intermediate eigenvalues; the addition operation unit is used to perform a first addition operation on the multiple first intermediate eigenvalues ​​to obtain multiple parallel intermediate matrices.

[0117] According to an embodiment of the present invention, the parallel intermediate matrix includes an intermediate input matrix, an intermediate forgetting matrix and an intermediate state matrix; the delay operation submodule includes: a multiplication operation unit, which is used to perform multiplication operations on the intermediate input matrix, the intermediate forgetting matrix, the intermediate state matrix and the hidden state matrix at a second moment to obtain a second intermediate matrix.

[0118] According to an embodiment of the present invention, the second gating algorithm includes a second row multiplication operation and a second addition operation; the delay operation submodule also includes: a second row multiplication operation unit and a second addition operation unit. The second row multiplication operation unit is used to perform a second row multiplication operation on at least one non-zero element and the weight row information at a third moment to obtain a plurality of second intermediate eigenvalues; the second addition operation unit is used to perform a second addition operation on the plurality of second intermediate eigenvalues ​​to obtain a delayed intermediate matrix.

[0119] According to an embodiment of the present invention, a graph neural network accelerator includes a computing array, which includes a plurality of computing units arranged in a one-dimensional direction, and the number of computing units is greater than or equal to the number of columns contained in a weight matrix.

[0120] According to an embodiment of the present invention, any multiple modules of the reading module 610, the operation module 620 and the task execution module 630 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the reading module 610, the operation module 620 and the task execution module 630 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or encapsulating the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the reading module 610, the operation module 620 and the task execution module 630 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be executed.

[0121] Figure 7 A block diagram of an electronic device suitable for implementing a graph structure data processing method according to an embodiment of the present invention is shown.

[0122] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present invention includes a first processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 to a random access memory (RAM) 703. The first processor 701 may, for example, include a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related processor group and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The first processor 701 may also include an onboard memory for caching purposes. The first processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0123] In RAM 703, various programs and data required for the operation of electronic device 700 are stored. Processor 701, ROM 702 and RAM 703 are connected to each other via bus 704. First processor 701 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 702 and / or RAM 703. It should be noted that the program may also be stored in one or more memories other than ROM 702 and RAM 703. First processor 701 may also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0124] According to an embodiment of the present invention, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to the bus 704. The electronic device 700 may further include one or more of the following components connected to the input / output (I / O) interface 705: an input portion 706 including a keyboard, a mouse, etc.; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 708 including a hard disk, etc.; and a communication portion 709 including a network interface card such as a LAN card, a modem, etc. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output (I / O) interface 705 as needed. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed into the storage portion 708 as needed.

[0125] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0126] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 702 and / or RAM 703 described above and / or one or more memories other than ROM 702 and RAM 703.

[0127] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the graph structure data processing method provided by the embodiment of the present invention.

[0128] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when the first processor 701 executes the computer program. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0129] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 709, and / or installed from the removable medium 711. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0130] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the first processor 701, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0131] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0132] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0133] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0134] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A method for processing graph structure data, applied to a graph neural network accelerator, wherein the graph structure data includes a sparse feature matrix corresponding to the graph structure data to be processed, characterized in that: The method comprises: Reading a compression feature matrix and a state feature matrix, wherein the compression feature matrix is ​​obtained by compressing row position information and value information of at least one element in the sparse feature matrix based on a preset compression strategy, and the state feature matrix represents state features of the compression feature matrix in the previous processing; Performing a first matrix operation on the compression feature matrix and the weight matrix to obtain a first intermediate matrix, and performing a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, wherein the first matrix operation includes a parallel operation and a delay operation, and the first intermediate matrix includes a parallel intermediate matrix and a delay intermediate matrix; A target matrix is ​​obtained according to the first intermediate matrix and the second intermediate matrix, so as to perform a target task using the target matrix.

2. The processing method according to claim 1, characterized in that: The characteristic of the element comprises at least one of an all-zero row characteristic and a non-all-zero row characteristic, wherein the non-all-zero row characteristic comprises at least one non-zero element; The processing method also includes: In the case where the feature of the current element is the all-zero row feature or the non-all-zero row feature, a label value corresponding to the all-zero row feature is determined based on a label correspondence relationship, or a label value corresponding to the non-zero element is determined, and the label correspondence relationship characterizes the relationship between multiple label values ​​and the all-zero row feature and the non-zero element.

3. The processing method according to claim 2, characterized in that: The label value includes a first label value and a second label value; The determining, based on the label correspondence relationship, a label value corresponding to the all-zero row feature, or determining a label value corresponding to the non-zero element, includes: When the feature of the current element is the all-zero row feature, determining that the label value corresponding to the all-zero row feature is the second label value; In a case where the feature of the current element is the non-all-zero row feature, a label value corresponding to the non-zero element is determined to be the first label value based on the numerical information and the row position information.

4. The processing method according to claim 3, characterized in that: The processing method also includes: Zero elements in the non-all-zero row features are processed, and the compressed feature matrix is ​​determined based on the first label value and the second label value.

5. The processing method according to claim 4, characterized in that: The processing of the zero elements in the non-all-zero row features and determining the compressed feature matrix based on the first label value and the second label value includes: When the feature of the current element is the non-all-zero row feature, deleting the zero element in the current row; The non-zero elements are sequentially converted into the first label values, and the non-all-zero row features are converted into the second label values, to obtain the compressed feature matrix.

6. The processing method according to any one of claims 1 to 5, characterized in that: The reading of the compression feature matrix and the state feature matrix includes: Based on the storage address information, the elements in the compressed feature matrix are read from the first storage unit in sequence, and when the read label value is an end-of-row label value, the target element and the target row information are obtained, and the end-of-row label value is used to indicate that the element is the last non-zero element in the target row information; The state feature matrix is ​​read from the second storage unit based on the storage address information and a mapping relationship, wherein the mapping relationship represents a corresponding relationship between the state features in the state feature matrix and the target element.

7. The processing method according to claim 1, characterized in that: The processing method also includes: When the target matrix is ​​obtained, the operation cache corresponding to the first matrix operation and the second matrix operation in the internal storage unit of the accelerator is cleared, and the matrix operation of the next cycle is performed, and the internal storage unit is an on-chip storage unit of the accelerator.

8. The processing method according to claim 2, characterized in that: The state feature matrix includes a hidden state matrix, the algorithm of the accelerator is a gating algorithm, and the gating algorithm includes a first gating algorithm and a second gating algorithm; Performing a first matrix operation on the compression feature matrix and the weight matrix to obtain a first intermediate matrix, and performing a second matrix operation on the state feature matrix and the first intermediate matrix to obtain a second intermediate matrix, including: Determine a first time for the parallel operation, determine a second time for the second matrix operation based on the first time, and determine a third time for the delay operation based on the second time; Performing the parallel operation on the compressed feature matrix and the weight matrix based on the first gating algorithm at a first moment to obtain a plurality of parallel intermediate matrices; At a second moment, the second intermediate matrix is ​​obtained using a plurality of the parallel intermediate matrices and the hidden state matrix, and at a third moment, the delay operation is performed on the compressed feature matrix and the weight matrix based on the second gating algorithm to obtain the delayed intermediate matrix, so that the moment information of obtaining the second intermediate matrix is ​​the same as the moment information of obtaining the delayed intermediate matrix.

9. The processing method according to claim 8, characterized in that: The first gating algorithm includes a first row multiplication operation and a first addition operation, the weight matrix includes weight row information corresponding to the non-zero elements, and the weight row information includes a plurality of weight values; At a first moment, the first gating algorithm is used to perform the parallel operation on the compressed feature matrix and the weight matrix to obtain a plurality of parallel intermediate matrices, including: Performing the first row multiplication operation on at least one of the non-zero elements and the weight row information at the first time to obtain a plurality of first intermediate eigenvalues; The first addition operation is performed on a plurality of the first intermediate eigenvalues ​​to obtain a plurality of the parallel intermediate matrices.

10. The processing method according to claim 8, characterized in that: The parallel intermediate matrix includes an intermediate input matrix, an intermediate forgetting matrix and an intermediate state matrix; Determining the second intermediate matrix using the plurality of parallel intermediate matrices and the hidden state matrix at a second moment includes: At the second moment, a multiplication operation is performed on the intermediate input matrix, the intermediate forgetting matrix, the intermediate state matrix and the hidden state matrix to obtain the second intermediate matrix.

11. The processing method according to claim 9, characterized in that: The second gating algorithm includes a second row multiplication operation and a second addition operation; Performing the delay operation on the compressed feature matrix and the weight matrix based on the second gating algorithm at a third moment to obtain the delayed intermediate matrix includes: Performing the second row multiplication operation on at least one of the non-zero elements and the weight row information at the third moment to obtain a plurality of second intermediate eigenvalues; The second addition operation is performed on a plurality of the second intermediate eigenvalues ​​to obtain the delayed intermediate matrix.

12. The processing method according to claim 1, characterized in that: The graph neural network accelerator includes a computing array, which includes a plurality of computing units arranged in a one-dimensional direction, and the number of the computing units is greater than or equal to the number of columns contained in the weight matrix.

13. A graph neural network accelerator, characterized in that: include: Memory; A processor configured to execute the method according to any one of claims 1 to 12 according to the instructions stored in the memory.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images