Neural network data operation acceleration device and method, equipment and medium

By dividing the feature matrix into submatrixes and generating index tables, localized operations of neural network data are implemented, and the problems of insufficient cache space and increased memory access in convolution operations are solved, which improves the computing efficiency.

CN118966293APending Publication Date: 2024-11-15HUNAN GREAT WALL GALAXY TECH CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410992516.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

When performing convolutional operations in the prior art, the increase in the matrix size leads to insufficient cache space and low computing efficiency, and the expansion of feature matrix data leads to an increase in the number of memory accesses, which cannot effectively improve the computing efficiency.

Method used

By dividing the feature matrix of the neural network into multiple feature submatrices in the C direction, and generating the filled img2col operation index table, map it back to the original matrix index table, realizing data localization operations, reducing the number of data accesses, and improving computing efficiency.

Benefits of technology

It effectively reduces cache usage and feature data access times, improves the computing efficiency of neural network operations, and is suitable for the acceleration of operation of various neural networks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118966293A_ABST
    Figure CN118966293A_ABST
Patent Text Reader

Abstract

The invention relates to a neural network data operation acceleration device and method, equipment and a medium, and the method comprises the steps: firstly obtaining the input data of neural network operation and Cin convolution kernel data, dividing a feature matrix SMF into N feature sub-matrixes MF in a C direction, generating an index table of img2col operation after pad for each feature sub-matrix MF, and carrying out the calculation of the img2col operation according to the index table. According to the method, firstly, an index table generating img2col operation is mapped back to an index table of an original feature matrix SMF, finally, localization of data of neural network operation is achieved, operation processing is carried out, multiple sliding windows can be calculated at a time, in other words, multiple pieces of data can be read at a time, and therefore the number of times of data access is greatly reduced, and the operation efficiency is greatly improved. Compared with the prior art, localization operation of the neural network is achieved, cache occupation of a computer is effectively reduced, the number of times of access to the feature data is reduced, and the method can be universally used for operation acceleration of various neural networks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of computer operation acceleration, and relates to a neural network data operation acceleration device, method, equipment and medium. Background Art

[0002] With the further development of artificial intelligence and big data technology, artificial intelligence involves more and more application fields, which helps the development of various industries and technical fields. At present, the main computational load of various neural networks comes from their convolution operations. The computational load of convolutional networks accounts for 90% of the computational load of the entire neural network. In current practical applications, convolution operations are basically converted into matrix multiplication operations, which are implemented through systolic arrays or general matrix multiplication (GEMM). However, with the development of deep neural networks, the size of network matrices (such as feature matrices and weight matrices) is also getting larger. When the matrix is ​​large, problems such as insufficient cache space, inflexible calculations, and low computational efficiency will be more prominent. At present, how to efficiently improve the computational efficiency of convolution operations has become a key issue to be solved.

[0003] In order to improve the computational efficiency of convolution operations, the feature data and weight data are usually rearranged through the img2col operation, and the rearranged data are then input into the matrix operation unit for computational processing. The final result of the matrix multiplication is the output of the neural network. The amount of weight data does not increase after the img2col operation, and the data can be rearranged. However, the feature matrix data will expand to several times the original after the img2col operation, which will face problems such as insufficient cache space and increased memory access times. Therefore, there is still a technical problem of low computational efficiency. Summary of the invention

[0004] In response to the problems existing in the above-mentioned traditional methods, the present invention proposes a neural network data operation acceleration device, a neural network data operation acceleration method, a computer device and a computer-readable storage medium, which can effectively improve the computing efficiency of the computer when performing neural network operations.

[0005] In order to achieve the above objectives, the embodiments of the present invention adopt the following technical solutions:

[0006] On the one hand, a neural network data operation acceleration device is provided, comprising:

[0007] The data acquisition module is used to obtain the input data of the neural network operation and Cin convolution kernel data; Cin is the number of input channels of the neural network;

[0008] The matrix partitioning module is used to divide the feature matrix of the neural network into N feature sub-matrices of size WHC0 in the C direction, and to divide the weight matrix of the neural network into N weight sub-matrices accordingly; W is the width of the feature sub-matrix, H is the height of the feature sub-matrix, C0 is an integer greater than 1, and N is an integer not less than 1;

[0009] Index generation module, used to generate the index table of img2col operation after all feature sub-matrices are filled;

[0010] The index mapping module is used to map the index table generated by the img2col operation back to the index table of the feature matrix;

[0011] The operation processing module is used to read the input data according to the index table of the characteristic submatrix for localization, read the data according to the index table of the small matrix of the characteristic submatrix and perform preprocessing, and then use the convolution kernel to complete the operation processing to obtain the operation results of N characteristic submatrices;

[0012] The result accumulation module is used to accumulate the operation results of the N feature sub-matrices as the output result of the final neural network operation.

[0013] On the other hand, a method for accelerating neural network data operation is also provided, comprising the steps of:

[0014] Get the input data of the neural network operation and Cin convolution kernel data; Cin is the number of input channels of the neural network;

[0015] Divide the feature matrix of the neural network into N feature sub-matrices of size WHC0 in the C direction, and divide the weight matrix of the neural network into N weight sub-matrices accordingly; W is the width of the feature sub-matrix, H is the height of the feature sub-matrix, C0 is an integer greater than 1, and N is an integer not less than 1;

[0016] Generate an index table of img2col operations after all feature sub-matrices are filled;

[0017] Map the index table of the generated img2col operation back to the index table of the feature matrix;

[0018] The input data is read according to the index table of the feature submatrix for localization, and the data is read according to the index table of the small matrix of the feature submatrix and preprocessed, and the convolution kernel is used to complete the operation processing to obtain the operation results of N feature submatrices;

[0019] The operation results of the N feature sub-matrices are accumulated as the output result of the final neural network operation.

[0020] On the other hand, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned neural network data operation acceleration method when executing the computer program.

[0021] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above-mentioned neural network data operation acceleration method are implemented.

[0022] One of the above technical solutions has the following advantages and beneficial effects:

[0023] The above-mentioned neural network data operation acceleration device, method, equipment and medium first obtain the input data of the neural network operation and Cin convolution kernel data, first divide the feature matrix into N feature sub-matrices in the C direction, and then for each feature sub-matrix, generate the index table of the img2col operation after padding, and then map the generated index table of the img2col operation back to the index table of the original feature matrix, and finally realize the localization of the data of the neural network operation and enter the operation processing, and can calculate multiple sliding windows at a time, that is, can read multiple data at a time, thereby greatly reducing the number of accesses to the data to greatly improve the operation efficiency. Compared with traditional technologies, it realizes the localization operation of the neural network, effectively reduces the cache occupancy of the computer and reduces the number of accesses to the feature data, and can be used for the operation acceleration of various neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic diagram of a module framework of a neural network data computing acceleration device in one embodiment;

[0026] Figure 2 is a schematic diagram of a calculation process of a matrix in one embodiment;

[0027] Figure 3 A schematic diagram of generating an index table in one embodiment;

[0028] Figure 4 Schematic diagram of a flow chart of a method for accelerating neural network data operations in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art of the present invention. The terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0030] It should be noted that the reference to "embodiment" in this article means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present invention. The presentation of this phrase at various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It will be appreciated by those skilled in the art that the embodiments described herein may be combined with other embodiments. The term "and / or" used in the specification of the present invention and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0031] The following will describe the implementation of the present invention in detail with reference to the accompanying drawings in the embodiment diagram of the present invention.

[0032] The img2col operation converts the padded feature matrix into a matrix form that is more suitable for convolution operations, that is, converting the image (im) into columns (col). Specifically, the img2col operation converts the input feature matrix into a new matrix in which each column represents a local area where a convolution kernel can be applied. This conversion allows the convolution operation to be expressed as matrix multiplication, thereby improving computational efficiency.

[0033] In traditional technology, the weight data is first rearranged into Kw*Kh sub-convolution kernel groups according to the rules, and then the feature data is rearranged according to the rearrangement rules of the weight data, and then Kw*Kh times are accumulated to output the result. Although the number of times the feature data is read is greatly reduced, there is still a problem of occupying too much cache at one time, and the localization problem is not solved. Kw and Kh are both integers greater than or equal to 1, where Kw*Kh refers to the width w and height h of the convolution (Kenel) kernel, which is generally 3*3.

[0034] In one embodiment, Figure 1As shown, a neural network data operation acceleration device 100 is provided, which may include a data acquisition module 11, a matrix division module 13, an index generation module 15, an index mapping module 17, an operation processing module 19 and a result accumulation module 21. Among them, the data acquisition module 11 is used to obtain the input data and Cin convolution kernel data of the neural network operation; Cin is the number of input channels of the neural network. The matrix division module 13 is used to divide the feature matrix of the neural network (denoted as SMF) into N feature sub-matrices (denoted as MF) of size WHC0 in the C direction, and divide the weight matrix of the neural network into N weight sub-matrices correspondingly; W is the width of the feature sub-matrix MF, H is the height of the feature sub-matrix MF, C0 is an integer greater than 1, and N is an integer not less than 1. The index generation module 15 is used to generate an index table of the img2col operation after all the feature sub-matrices MF are filled. The index mapping module 17 is used to map the generated index table of the img2col operation back to the index table of the feature matrix SMF. The operation processing module 19 is used to read the input data according to the index table of the feature submatrix MF for localization, read the data according to the index table of the small matrix (recorded as mf) of the feature submatrix MF and perform preprocessing, and then use the convolution kernel to complete the operation processing to obtain the operation results of N feature submatrices MF. The result accumulation module 21 is used to accumulate the operation results of the N feature submatrices MF as the output result of the final neural network operation.

[0035] It can be understood that the overall design concept of the present invention can be as follows: first obtain the input data of the neural network operation and Cin convolution kernel data, first divide the original feature matrix SMF into N (1≤N) feature sub-matrices MF in the C direction, the i-th feature sub-matrix is ​​recorded as MFi, i takes a value from 1 to N, C0 is an integer greater than 1, and the weight matrix is ​​similarly generated for each weight sub-matrix. Among them, dividing the feature matrix SMF into N feature sub-matrices MF in the C direction means dividing the original feature matrix SMF according to the dimension of the input channel (Cin). Specifically, the feature matrix SMF is generally a four-dimensional tensor and its shape is (N, Cin, H, W), where N is the batch size, Cin is the number of input channels, H is the height of the feature matrix SMF, and W is the width of the feature matrix SMF. Then, since a feature submatrix MF is still relatively large, for each feature submatrix MF, an index table of the img2col operation after padding (the index table of each feature submatrix MF is consistent) is generated to divide the feature submatrix MF into multiple small matrices mf, and then the index table of the generated img2col operation (that is, the index table corresponding to the small matrix, also called the index table small matrix mi) is mapped back to the index table of the original feature matrix SMF. Finally, the localization of the data of the neural network operation is realized and the operation processing step is entered, in which the input data is read according to the index table of the feature submatrix MF to realize localization.

[0036] Operation processing steps: read data according to the index table of the small matrix mf and perform preprocessing, which includes shifting, splicing and filling, so as to realize conversion and calculation at the same time. To calculate the results of m convolution sliding windows, it is necessary to calculate Kw*Kh times, accumulate the results of Kw*Kh matrix multiplications to obtain the accumulated results of the small matrix mf, and use the accumulated results of the small matrix mf as part of the output results of the neural network operation. After calculating all small matrices mf, the operation results of a feature submatrix MF are obtained, and the operation results of N feature submatrices MF are accumulated as the output results of the final neural network operation. The value of the number m of convolution sliding windows is related to the performance of the AI ​​core. The appropriate value can be taken according to the performance of the AI ​​core to meet the processing efficiency requirements under the performance constraints of the AI ​​core.

[0037] The above-mentioned neural network data operation acceleration device 100 first obtains the input data of the neural network operation and Cin convolution kernel data, first divides the feature matrix SMF into N feature sub-matrices MF in the C direction, and then generates an index table of the img2col operation after padding for each feature sub-matrix MF, and then maps the generated index table of the img2col operation back to the index table of the original feature matrix SMF, and finally realizes the localization of the data of the neural network operation and enters the operation processing, and can calculate multiple sliding windows at a time, that is, can read multiple data at a time, thereby greatly reducing the number of accesses to the data to greatly improve the operation efficiency. Compared with traditional technologies, it realizes the localization operation of the neural network, effectively reduces the cache occupancy of the computer and reduces the number of accesses to the feature data, and can be used for the operation acceleration of various neural networks.

[0038] More specifically, after obtaining the input data of the neural network and Cin convolution kernel data, the original feature matrix SMF is first divided into multiple feature sub-matrices MF on the Cin scale, that is, the original feature matrix SMF is divided on the input channel dimension to obtain multiple feature sub-matrices MF. The shape of these feature sub-matrices MF is usually (N, Csub, H, W), where Csub is the number of channels of each feature sub-matrix MF, which can be divided as needed to achieve computational acceleration. For all feature sub-matrices MF, an index table of the img2col operation after padding is generated. The specific process is: each feature sub-matrix MF is zero-filled through the pad operation to ensure that edge information is not lost during the convolution process, and the filled feature sub-matrix MF is converted into a column matrix through the img2col operation. In this way, the convolution operation can be converted into matrix multiplication, thereby improving computational efficiency. For example Figure 2 As shown, A MN and B NK All are matrix elements. When generating the index table, the original position index of the matrix elements of the feature submatrix MF is recorded in the conversion process so that the original data can be restored or referenced when needed. The index table of the img2col operation is common to all feature submatrices MF, and then the index table generated by the img2col operation is mapped back to the index table of the original feature matrix SMF, and finally the data is localized and enters the operation processing step.

[0039] In one embodiment, preprocessing includes shifting, splicing and padding. It can be understood that in the operation processing step, data is read and localized according to the index table of the feature submatrix MF to divide the large index table matrix (recorded as MI, which is the index table matrix corresponding to the feature submatrix MF) into small index table matrices mi, and then the data is read in according to the small index table matrix mi for preprocessing. The preprocessing includes shifting, splicing and padding to achieve conversion and calculation. To calculate the results of m (such as m = 16) convolution sliding windows, it is necessary to calculate Kw*Kh times, accumulate the matrix multiplication results of Kw*Kh times to obtain the accumulated result, and use the accumulated result as a partial output result of the neural network operation; after calculating all the small matrices, the operation result of a feature submatrix MF is obtained, and finally the operation results of N feature submatrices MF are accumulated as the output result of the final neural network operation.

[0040] Among them, the size of the original feature matrix SMF is W*H*C, W is the matrix width, H is the matrix height, C is the number of channels, Kw is the length of the convolution kernel, Kh is the width of the convolution kernel, Cout represents the number of convolution kernels, C0 can be set dynamically, and the size of C0 in this embodiment is 16. Fij represents the element of the i-th row and j-th column in the W*H plane of the feature map.

[0041] In the specific process of dividing the large matrix into small matrices, for a certain weight element W in the convolution kernel mt In the convolution process, the sliding window participates in the calculation as many times as it slides. When stride=1, the data involved in the calculation are adjacent when the same weight element slides. Therefore, multiple sliding windows can be calculated at one time, that is, multiple data can be read in at one time, which greatly reduces the number of data accesses and greatly improves the computing efficiency.

[0042] Each row of the index table matrix MI represents a weight element W of the convolution kernel. mt The coordinates of the data corresponding to the m-times movement, assuming that the size of the convolution kernel is 3*3, then the 1-9 rows of the large index table matrix MI are the coordinates of the data corresponding to the m-times movement of the nine weight elements of the convolution kernel, and so on for all the remaining rows. The large matrix is ​​divided into small matrices here, which means that the large index table matrix MI is divided into multiple small index table matrices mi by row, and the small matrix multiplication is calculated each time, thereby realizing the localization of operations, reducing the cache occupancy problem, and greatly reducing the number of accesses to feature data. At the same time, it is applicable to a variety of networks, realizing conversion and calculation.

[0043] The process of dividing the original feature matrix SMF into multiple feature sub-matrices MF on the Cin scale includes:

[0044] The original feature matrix SMF is divided into multiple feature sub-matrices MF on the scale of Cin, and the W and H of each feature sub-matrix MF remain unchanged. If Cin is less than C0, it is padded with 0 and then divided into multiple feature sub-matrices MF; if Cin is greater than C0, it is directly divided into multiple feature sub-matrices MF, so that the depth of the elements Aij of each feature sub-matrix MF is C0, and the insufficient ones are still padded with 0.

[0045] The weight matrix is ​​also divided into multiple weight sub-matrices according to the above rules, and the size of the weight sub-matrix d after division is (Kw*Kh*C0)*Cout.

[0046] Since the row vectors in the matrix operated in this embodiment are divided into a fixed length C0 from the channel C direction, when generating N*C0 (i.e., N and the depth is C0) feature sub-matrices MF, the discreteness of the elements in the matrix W and H directions is reduced to only the discreteness in the H direction, thereby reducing the probability of conflict generated by reading feature data from the memory, thereby improving the performance of reading the original feature matrix SMF data.

[0047] In one embodiment, regarding the process of generating an index table for the img2col operation after padding, specifically: the img2col operation implements convolution through matrix multiplication, so that a large number of convolution calculations in the convolution layer can be converted into the parallelism of the matrix operation itself. Therefore, the processor only needs to efficiently implement matrix multiplication to be able to efficiently perform convolution operations. The expansion method of the img2col operation is to expand each input feature submatrix MF (that is, a convolution window) into a row (or a column) to generate a new input feature small matrix, the number of which is the same as the number of input feature submatrices MF. At the same time, the weight submatrix in the convolution kernel is expanded into a column (or a row), and multiple weight small matrices can be arranged into multiple columns. Only the parameters such as W, H, Cin, Cout, stride and pad of the feature matrix need to be input for conversion. The size of the index table (which can be recorded as S) after conversion is ((W out *H out ) / m*Kw*Kh, m), where Sij represents the element in the i-th row and j-th column in the plane of the feature map. The value stored in the element is the coordinate of the feature submatrix MF assumed to have been padded. Then the generated index table manipulated by img2col is mapped back to the index table of the original feature matrix SMF. If the Sij coordinate exists after padding, zero is used to represent this coordinate.

[0048] The above W out and H out The calculation formula is as follows:

[0049] W out=(W-Kw+2*pad) / stride+1

[0050] H out =(H-Kh+2*pad) / stride+1

[0051] Among them, Kw, Kh are the length and width of the convolution kernel, stride is the step length, and W out is the number of sliding windows on W, H out is the number of sliding windows on H.

[0052] As attached Figure 3 As shown, the left part of the figure is a two-dimensional graph of the feature submatrix MF. The outermost circle of 0 represents the pad data, and the middle one is the real data. The right part of the figure is a two-dimensional graph of the index table of the img2col operation. -1 represents the pad data, which is not the real data.

[0053] Among them, the preprocessing includes shifting, splicing and filling. The size of the matrix multiplication is fixed, and its size is determined by the hardware level. In this embodiment, a fixed size of data is read at a time. When the stride is 1, all the read data are valid data. When the stride = 2, only half of the data is valid data. Here, multiple shifts and splicing of data will be rearranged into data for a small matrix multiplication; when there is insufficient valid data to be rearranged into data for a small matrix multiplication, the insufficient data will be filled with zeros.

[0054] Among them, regarding the process of accumulating the matrix multiplication results to obtain the cumulative result, specifically, when the convolution kernel slides in the feature matrix SMF, each time a feature submatrix MF with the same size as the weight submatrix is ​​drawn in the input feature map, and the corresponding elements are multiplied and accumulated with the weight submatrix (dot product operation). To calculate the results of m (such as m = 16) convolution windows, it is necessary to calculate Kw*Kh times, accumulate the Kw*Kh times of small matrix multiplication results to obtain the cumulative result, and use the accumulated result obtained by accumulating the small matrix multiplication results as part of the output result of the neural network operation.

[0055] Finally, in the process of accumulating the operation results of the N feature sub-matrices MF as the output result of the final neural network operation, specifically, after all the small matrices of each feature sub-matrix MF are calculated, the convolution output of each feature sub-matrix MF is obtained, and the N feature sub-matrices MF are traversed. Finally, the operation results of the N feature sub-matrices MF are accumulated as the output result of the final neural network operation.

[0056] Each module in the above-mentioned neural network data operation acceleration device 100 can be fully or partially implemented by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of a device with data processing functions in the form of hardware, or can be stored in the memory of the aforementioned device in the form of software, so that the processor can call and execute the operations corresponding to the above modules. The aforementioned device can be, but is not limited to, various types of computer devices already available in the art.

[0057] In one embodiment, Figure 4 As shown, a method for accelerating neural network data operation is provided, which may include the following processing steps S10 to S20:

[0058] S10, obtaining input data of the neural network operation and Cin convolution kernel data; Cin is the number of input channels of the neural network;

[0059] S12, dividing the feature matrix of the neural network into N feature sub-matrices of size WHC0 in the C direction, and correspondingly dividing the weight matrix of the neural network into N weight sub-matrices; W is the width of the feature sub-matrix, H is the height of the feature sub-matrix, C0 is an integer greater than 1, and N is an integer not less than 1;

[0060] S14, generating an index table of the img2col operation after all feature sub-matrices are filled;

[0061] S16, mapping the index table of the generated img2col operation back to the index table of the feature matrix;

[0062] S18, reading the input data according to the index table of the feature submatrix for localization, reading the data according to the index table of the small matrix of the feature submatrix and preprocessing, and then completing the operation processing using the convolution kernel to obtain the operation results of N feature submatrices;

[0063] S20, accumulating the operation results of the N feature sub-matrices as the output result of the final neural network operation.

[0064] The above-mentioned neural network data operation acceleration method first obtains the input data of the neural network operation and Cin convolution kernel data, first divides the feature matrix SMF into N feature sub-matrices MF in the C direction, and then generates an index table of the img2col operation after padding for each feature sub-matrix MF, and then maps the generated index table of the img2col operation back to the index table of the original feature matrix SMF, and finally realizes the localization of the data of the neural network operation and enters the operation processing, and can calculate multiple sliding windows at a time, that is, can read multiple data at a time, thereby greatly reducing the number of accesses to the data to greatly improve the operation efficiency. Compared with traditional technologies, it realizes the localization of neural network operations, effectively reduces the computer's cache occupancy and reduces the number of accesses to feature data, and can be used for the operation acceleration of various neural networks.

[0065] In one embodiment, preprocessing includes shifting, splicing, and padding.

[0066] In one embodiment, in the process of using the convolution kernel to complete the operation processing, the number of sliding windows on W is:

[0067] W out =(W-Kw+2*pad) / stride+1

[0068] The number of sliding window slides on H is:

[0069] H out =(H-Kh+2*pad) / stride+1

[0070] Among them, Kw is the length of the convolution kernel, Kh is the width of the convolution kernel, pad is the padding of the feature submatrix MF, and stride is the step size.

[0071] For the specific limitations of the neural network data operation acceleration method, please refer to the corresponding limitations of the neural network data operation acceleration device 100 mentioned above, which will not be repeated here.

[0072] It should be understood that although Figure 4 The steps in the method are shown in sequence as indicated by the arrows, but the steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of the steps, and the steps can be executed in other orders. Figure 4 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequentially, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0073] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the following processing steps when executing the computer program: obtaining input data and Cin convolution kernel data of a neural network operation; Cin is the number of input channels of the neural network; dividing the feature matrix SMF of the neural network into N feature sub-matrices MF in the C direction, and correspondingly dividing the weight matrix of the neural network into N weight sub-matrices; W is the width of the feature matrix SMF, H is the height of the feature matrix SMF, C0 is an integer greater than 1, and N is an integer not less than 1; generating an index table of the img2col operation after all feature sub-matrices MF are filled; mapping the generated index table of the img2col operation back to the index table of the feature matrix SMF; reading the input data according to the index table of the feature sub-matrix MF and performing preprocessing, using the convolution kernel to complete the operation processing, and obtaining the operation results of the N feature sub-matrices MF; accumulating the operation results of the N feature sub-matrices MF as the output result of the final neural network operation.

[0074] In one embodiment, when the processor executes the computer program, it can also implement the steps or sub-steps added in each embodiment of the above-mentioned neural network data operation acceleration method.

[0075] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following processing steps are implemented: obtaining input data and Cin convolution kernel data of a neural network operation; Cin is the number of input channels of the neural network; dividing the feature matrix SMF of the neural network into N feature sub-matrices MF in the C direction, and correspondingly dividing the weight matrix of the neural network into N weight sub-matrices; W is the width of the feature matrix SMF, H is the height of the feature matrix SMF, C0 is an integer greater than 1, and N is an integer not less than 1; generating an index table of the img2col operation after all feature sub-matrices MF are filled; mapping the generated index table of the img2col operation back to the index table of the feature matrix SMF; reading the input data according to the index table of the feature sub-matrix MF and performing preprocessing, using the convolution kernel to complete the operation processing, and obtaining the operation results of the N feature sub-matrices MF; accumulating the operation results of the N feature sub-matrices MF as the output result of the final neural network operation.

[0076] In one embodiment, when the computer program is executed by a processor, it can also implement the steps or sub-steps added in each embodiment of the above-mentioned neural network data operation acceleration method.

[0077] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus dynamic random access memory (RambusDRAM, referred to as RDRAM) and interface dynamic random access memory (DRDRAM).

[0078] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0079] The above embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection of the invention. It should be pointed out that, for those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the attached claims.

Claims

1. A neural network data computing acceleration device, characterized in that: include: The data acquisition module is used to obtain the input data of the neural network operation and Cin convolution kernel data; Cin is the number of input channels of the neural network; The matrix partitioning module is used to divide the feature matrix of the neural network into N feature sub-matrices of size WHC0 in the C direction, and to divide the weight matrix of the neural network into N weight sub-matrices accordingly; W is the width of the feature sub-matrix, H is the height of the feature sub-matrix, C0 is an integer greater than 1, and N is an integer not less than 1; Index generation module, used to generate the index table of img2col operation after all feature sub-matrices are filled; The index mapping module is used to map the index table generated by the img2col operation back to the index table of the feature matrix; An operation processing module is used to read the input data according to the index table of the characteristic submatrix for localization, read the data according to the index table of the small matrix of the characteristic submatrix and perform preprocessing, and then use the convolution kernel to complete the operation processing to obtain the operation results of N characteristic submatrices; The result accumulation module is used to accumulate the operation results of the N feature sub-matrices as the output result of the final neural network operation.

2. The neural network data calculation acceleration device according to claim 1, characterized in that: The preprocessing includes shifting, splicing and filling.

3. The neural network data calculation acceleration device according to claim 1 or 2, characterized in that: In the process of using the convolution kernel to complete the operation, the number of sliding windows on W is: IN out =(W-Kw+2*pad) / stride+1 The number of sliding window slides on H is: H out =(H-Kh+2*pad) / stride+1 Among them, Kw is the length of the convolution kernel, Kh is the width of the convolution kernel, pad is the feature sub-matrix padding, and stride is the step size.

4. A method for accelerating neural network data operations, characterized in that: Includes steps: Get the input data of the neural network operation and Cin convolution kernel data; Cin is the number of input channels of the neural network; Divide the feature matrix of the neural network into N feature sub-matrices of size WHC0 in the C direction, and divide the weight matrix of the neural network into N weight sub-matrices accordingly; W is the width of the feature sub-matrix, H is the height of the feature sub-matrix, C0 is an integer greater than 1, and N is an integer not less than 1; Generate an index table of img2col operations after all feature sub-matrices are filled; Map the index table of the generated img2col operation back to the index table of the feature matrix; The input data is read according to the index table of the characteristic submatrix for localization, the data is read according to the index table of the small matrix of the characteristic submatrix and preprocessed, and the operation is completed by using the convolution kernel to obtain the operation results of N characteristic submatrices; The operation results of the N feature sub-matrices are accumulated as the output result of the final neural network operation.

5. The neural network data operation acceleration method according to claim 4, characterized in that: The preprocessing includes shifting, splicing and filling.

6. The neural network data operation acceleration method according to claim 4 or 5, characterized in that: In the process of using the convolution kernel to complete the operation, the number of sliding windows on W is: IN out =(W-Kw+2*pad) / stride+1 The number of sliding window slides on H is: H out =(H-Kh+2*pad) / stride+1 Among them, Kw is the length of the convolution kernel, Kh is the width of the convolution kernel, pad is the feature sub-matrix padding, and stride is the step size.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the neural network data operation acceleration method described in any one of claims 4 to 6 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the neural network data operation acceleration method described in any one of claims 4 to 6 are implemented.

Citation Information

Cited By

  • Hardware acceleration method and device for sparse matrix multiplication operation of convolutional neural layer

    CN120179975A