A data processing method, device, terminal and storage medium

Abstract

The present invention discloses a data processing method, device, terminal and storage medium, which includes inputting data, calculating the hash of the input data, and obtaining the fingerprint of the input data; querying whether there is a convolution result of the same input data in the data container hash_map according to the hash of the input data, and if the query has a convolution result of the same data, directly returning the convolution result, otherwise entering the next step; calculating the convolution of the input data; saving the hash of the input data and the convolution result in the data container hash_map in the form of key-value, wherein the hash value is used as the key and the convolution result is used as the value, and then returning the convolution result. When similar data is input or the convolution layer in the middle of the convolutional neural network calculates and generates similar data, and the results of these data have been calculated before, there is no need to recalculate, and only the previously calculated results need to be obtained from the recorded historical results, thereby reducing the amount of calculation, improving the processing speed, and reducing power consumption.

Description

Technical Field

[0001] The present invention relates to data processing technology, and in particular to a data processing method, device, terminal and storage medium. Background Art

[0002] The main component of Convolutional Neural Networks (CNN) is the convolution layer, which is a data processing method that performs convolution operations on input data. Convolution operations are mainly implemented by multiplication and addition. Multiplication instructions are multi-clock cycle instructions in the CPU (Central Processing Unit). The more complex the convolutional neural network is and the more convolutional layers it has, the more multiplication operations are performed on the neural network inference calculation, which means it takes longer.

[0003] In order to speed up the reasoning speed of convolutional neural networks, people have proposed various technical solutions: 1. Implement convolution calculations through unique hardware circuits, such as NPU (Neural Network Processing Unit); 2. Reduce the number of multiplications to a certain extent through algorithms, such as the winograd algorithm. The implementation method through hardware circuits relies on hardware. Although it has great advantages in performance and power consumption, it is not flexible enough in application and cannot be used on devices without relevant hardware; algorithms that reduce the complexity of convolution operations are relatively flexible in application, but performance and power consumption depend to a large extent on the implementation of the algorithm. For example, although the winograd algorithm reduces the number of multiplication calculations, it still requires a large number of multiplications. Summary of the invention

[0004] In order to solve at least one technical problem existing in the above-mentioned prior art, the present invention provides a data processing method, device, terminal and storage medium.

[0005] To achieve the above object, the technical solution of the present invention is:

[0006] In a first aspect, the present invention provides a data processing method, comprising:

[0007] Input data, calculate the hash of the input data, and obtain the fingerprint of the input data;

[0008] According to the hash of the input data, query whether there is a convolution result of the same input data in the data container hash_map. If there is a convolution result of the same data, return the convolution result directly, otherwise proceed to the next step;

[0009] Calculate the convolution of the input data;

[0010] The hash of the input data and the convolution result are saved in the data container hash_map in the form of key-value, with the hash value as the key and the convolution result as the value, and then the convolution result is returned.

[0011] In a second aspect, the present invention provides a data processing device, comprising:

[0012] Input module, used to input data;

[0013] The hash calculation module is used to calculate the hash of the input data and obtain the fingerprint of the input data;

[0014] The data container hash_map stores the convolution result, which corresponds to the hash value;

[0015] The query module is used to query whether there is a convolution result of the same input data in the data container hash_map. If there is a convolution result of the same data, the convolution result is directly returned, otherwise the convolution calculation module is triggered;

[0016] Convolution calculation module, used to calculate the convolution of input data;

[0017] The saving module saves the hash of the input data and the convolution result into the data container hash_map in the form of key-value, where the hash value is used as the key and the convolution result is used as the value, and then returns the convolution result.

[0018] In a third aspect, the present invention provides a data processing terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0019] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described above are implemented.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The fingerprint information of the input data of the convolutional neural network is obtained, and the scheme of querying the historical calculation results based on the fingerprint information is used to reduce repeated calculations and improve the processing speed of convolution and neural networks; at the same time, the hash of the input data is calculated to improve the processing speed and IP utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the flow chart of convolution data processing;

[0023] Figure 2 It is a flowchart of neural network data processing;

[0024] Figure 3 It is a schematic diagram of the composition of a data processing device;

[0025] Figure 4 It is a schematic diagram of the query module composition;

[0026] Figure 5 Schematic diagram of the composition of the data processing terminal. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0028] The whole or part of the input data of the convolution is sometimes consistent with the whole or part of the previously input data. If the previous convolution calculation results are saved, then when encountering the same input, the previous results can be queried and then returned, thereby reducing calculations, increasing processing speed, and reducing power consumption. For example, if the neural network has been detecting the images captured by the camera in the same direction, because the content captured by the camera is the same, then the data in the processing process of the neural network is close or consistent, and the intermediate calculation results can be reused. The specific implementation method is introduced below.

[0029] See also Figure 1 As shown, this embodiment provides a data processing method applied to a convolutional layer, and its implementation steps include:

[0030] During the first run, a hash value (hash, hash algorithm, hash algorithm maps a binary value of any length to a smaller binary value of a fixed length, this small binary value is called a hash value) and a data container corresponding to the convolution result are established;

[0031] Input data, calculate the hash of the input data, and obtain the fingerprint of the input data;

[0032] According to the hash of the input data, query whether there is a convolution result of the same input data in the data container hash_map. If there is a convolution result of the same data, return the convolution result directly, otherwise proceed to the next step;

[0033] Calculate the convolution of the input data;

[0034] The hash of the input data and the convolution result are saved in the data container hash_map in the form of key-value, with the hash value as the key and the convolution result as the value, and then the convolution result is returned.

[0035] In this way, through the above steps, the fingerprint information of the input data of the convolutional neural network is obtained, and the solution of querying the historical calculation results based on the fingerprint information is used to reduce repeated calculations and improve the processing speed of convolution and neural networks; at the same time, the hash of the input data is calculated to improve the processing speed and IP utilization.

[0036] For neural networks, the implementation method is as follows Figure 2 As shown:

[0037] When running for the first time, a data container corresponding to the hash value of the neural network input data and the recognition result is established;

[0038] Before inference calculation, the hash of the input data is calculated to obtain the fingerprint of the input data;

[0039] According to the hash, check whether there is a recognition result of the same input data in the data container. If there is a recognition result of the same data, return the recognition result directly, otherwise proceed to the next step;

[0040] Calculate the recognition result of the input data, the implementation method of which may include the convolution layer implementation method introduced above;

[0041] The hash of the input data and the recognition result are saved in the data container in the form of key-value, with the hash value as the key and the recognition result as the value, and then the recognition result is returned.

[0042] The following takes the convolutional neural network for recognizing input images as an example to illustrate the detailed implementation method of this solution.

[0043] Define the hash table of the neural network N as T. T is implemented using C++STL's hash_map. The key is the hash value of the network input, and the value is the network output result corresponding to the hash.

[0044] The hash table of the convolutional layer of the neural network N is T L1 ,T L2 ...T Ln They also use C++STL's hash_map implementation, where the key is the hash value of the convolutional layer input data, and the value is the convolutional layer output data corresponding to the hash value. The number of hash tables for the convolutional layer can be determined according to the size of the convolutional layer input to prevent convolutional layers with small computational complexity from being added to the processing of this solution. For example, a hash table is added to the convolutional layer with an input larger than W*H, and no hash table is added to the others.

[0045] The processing of a convolutional layer with input greater than W*H*C is as follows:

[0046] Create a hash_map container for the current convolutional layer. The hash_map can be empty or load historical data from a file. When data is input, perform the following processing steps.

[0047] Step 1: Scale the input data to a dimension of 9*8*C in width and height;

[0048] Step 2: Convert the input data into single-channel data by adding the values ​​of each channel and then averaging the final result, such as:

[0049]

[0050] At this time, we get data with a dimension of 9*8*1:

[0051]

[0052] Step 3: Calculate the difference between the two elements before and after each row of data, that is, [d 00 d 01 …d 08 ] after processing to obtain [d 00 -d 01 d 01 -d 02 …d 07 -d 08 ], where if each difference is greater than 0, it takes the value of 1, otherwise it takes the value of 0. The final result is a matrix data of dimension 8*8 containing only 0 or 1:

[0053]

[0054] Step 4: The value of each element in step 3 is 0 or 1. Consider each row of the matrix as a byte value. After converting the value into hexadecimal, 8 rows of data get a hash value h of length 8.

[0055] Step 5: Use h to query in hash_map. If the result is found, the result is directly passed to the next layer as the output of the current convolutional layer, otherwise go to the next step; the query comparison method is as follows: 1) h' is a hash value selected from hash_map, and the difference of each bit in h and h' is compared; 2) Count the number of difference bits. If the number of differences is less than the set threshold D, it is considered that the result is found, otherwise compare the next hash value in hash_map.

[0056] Step 6: Calculate the convolution of the input data;

[0057] Step 7: Save the convolution result and h into hash_map, with h as the key and the convolution result as the value;

[0058] Step 8: Output the convolution result to the next layer.

[0059] If the program exits, you can choose to save the hash_map to a file for use the next time the program runs.

[0060] The neural network process is as follows:

[0061] Create a hash_map container for the current neural network. The hash_map can be empty or load historical data from a file. Assuming the input is a W*H*3 RGB image, when there is data input, perform the following processing steps.

[0062] Step 1: Scale the input image to an image of size 9*8;

[0063] Step 2: grayscale the 9*8 image using the same method as above;

[0064] Step 3: Calculate the difference between the two elements before and after each row of the grayscale image, [d 00 d 01 …d 08 ] after processing to obtain [d 00 -d 01 d 01 -d 02 …d 07 -d 08 ], where if each difference is greater than 0, the value is 1, otherwise the value is 0, and the final result is data of dimension 8*8;

[0065] Step 4: Treat each line as a byte of data and convert it to hexadecimal. The 8 lines of data will get a hash value h with a length of 8.

[0066] Step 5: Use h to query in hash_map. If a result is found, the result is directly output as the result of the current neural network. Otherwise, go to the next step. The query comparison method is the same as above.

[0067] Step 6: The data is passed to the neural network, and the neural network recognizes the result. The neural network can be implemented using the convolution layer processing method introduced above. That is, the layer in the neural network that meets the above convolution processing requirements can be processed using the above steps 1-8;

[0068] Step 7: Save the recognition result and h into hash_map, with h as the key and the convolution result as the value;

[0069] Step 8: Return the recognition result.

[0070] If the program exits, you can choose to save the hash_map to a file for use by the neural network the next time the program runs.

[0071] In summary, compared with the prior art, the present invention has the following technical advantages:

[0072] When similar data is input or the convolutional layer in the middle of the convolutional neural network generates similar data, and the results of these data have been calculated before, there is no need to recalculate. You only need to obtain the previously calculated results from the recorded historical results, thereby reducing the amount of calculation, improving processing speed, and reducing power consumption.

[0073] Embodiment 2:

[0074] See also Figure 3 As shown, the data processing device provided in this embodiment includes:

[0075] An input module 31, used for inputting data;

[0076] The hash calculation module 32 is used to calculate the hash of the input data to obtain the fingerprint of the input data;

[0077] The data container hash_map33 stores the convolution result, which corresponds to the hash value;

[0078] A query module 34 is used to query whether there is a convolution result of the same input data in the data container hash_map. If there is a convolution result of the same data, the convolution result is directly returned, otherwise the convolution calculation module is triggered;

[0079] A convolution calculation module 35, used for calculating the convolution of input data;

[0080] The saving module 36 saves the hash of the input data and the convolution result into the data container hash_map in the form of key-value, where the hash value is used as the key and the convolution result is used as the value, and then returns the convolution result.

[0081] If, by obtaining fingerprint information from the input data of the convolutional neural network, a solution is adopted to query the historical calculation results based on the fingerprint information, which can be used to reduce repeated calculations and improve the processing speed of the convolution and neural network; at the same time, the hash of the input data is calculated to improve the processing speed and the utilization rate of the IP.

[0082] In a specific embodiment, the data container hash_map33 is empty or contains historical data loaded from a file.

[0083] In a specific embodiment, if Figure 4 As shown, the query module 34 includes:

[0084] A comparison unit 341 is used to compare the difference of each bit in h and h'; h is the hash value of the input data, and h' is a hash value selected from hash_map;

[0085] The statistical unit 342 is used to count the number of difference bits. If the number of differences is less than the set threshold D, it is considered that the query result is found. Otherwise, the next hash value in the hash_map is compared.

[0086] Embodiment 3:

[0087] See also Figure 5 As shown, the data processing terminal provided in this embodiment includes a processor 51, a memory 52, and a computer program 53 stored in the memory 52 and executable on the processor 51, such as a data processing program. When the processor 51 executes the computer program 53, the steps of the above-mentioned embodiment 1 are implemented, such as Figure 1 Alternatively, the processor 51 implements the functions of each module / unit in the above-mentioned embodiment 2 when executing the computer program 53.

[0088] Exemplarily, the computer program 53 may be divided into one or more modules / units, which are stored in the memory 52 and executed by the processor 51 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 53 in the data processing terminal.

[0089] The data processing terminal may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The data processing terminal may include, but is not limited to, a processor 51 and a memory 52. ​​Those skilled in the art will appreciate that Figure 5 It is only an example of a data processing terminal and does not constitute a limitation of the data processing terminal. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the data processing terminal may also include input and output devices, network access devices, buses, etc.

[0090] The processor 51 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0091] The memory 52 may be an internal storage unit of the data processing terminal, such as a hard disk or memory of the data processing terminal. The memory 52 may also be an external storage device of the data processing terminal, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the data processing terminal. Further, the memory 52 may also include both an internal storage unit of the data processing terminal and an external storage device. The memory 52 is used to store the computer program and other programs and data required by the data processing terminal. The memory 52 may also be used to temporarily store data that has been output or is to be output.

[0092] Embodiment 4:

[0093] This embodiment provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method described in Embodiment 1 are implemented.

[0094] The computer-readable medium shown can be any device that can contain, store, communicate, propagate or transmit a program for use with an instruction execution system, device or apparatus or in conjunction with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium can even be paper or other suitable media on which the program can be printed, such as by optically scanning the paper or other medium, then editing, interpreting or processing in other suitable ways as necessary to obtain the program electronically, and then storing it in a computer memory.

[0095] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable ordinary technicians in the field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made based on the essence of the content of the present invention should be included in the protection scope of the present invention.

Claims

1. A data processing method, wherein the data is image data, characterized in that: include: Input data, calculate the hash of the input data, and obtain the fingerprint of the input data; According to the hash of the input data, query whether there is a convolution result of the same input data in the data container hash_map. If there is a convolution result of the same data, return the convolution result directly, otherwise proceed to the next step; Calculate the convolution of the input data; The hash of the input data and the convolution result are saved in the data container hash_map in the form of key-value, with the hash value as the key and the convolution result as the value, and then the convolution result is returned.

2. The data processing method according to claim 1, characterized in that: Also includes: During the first run, a data container hash_map corresponding to the hash value and the convolution result is created.

3. The data processing method according to claim 2, characterized in that: The data container hash_map is empty or contains historical data loaded from a file.

4. The data processing method according to claim 1, characterized in that: The query method for querying in the data container hash_map is: Compare and The difference of each bit in ; h is the hash value of the input data, It is a hash value selected from hash_map; Count the number of difference bits. If the number of differences is less than the set threshold D, it is considered that the query result is found. Otherwise, compare the next hash value in hash_map.

5. A data processing device, wherein the data is image data, characterized in that: include: Input module, used to input data; The hash calculation module is used to calculate the hash of the input data and obtain the fingerprint of the input data; The data container hash_map stores the convolution result, which corresponds to the hash value; The query module is used to query whether there is a convolution result of the same input data in the data container hash_map. If the query finds a convolution result of the same data, the convolution result is directly returned, otherwise the convolution calculation module is triggered; Convolution calculation module, used to calculate the convolution of input data; The saving module saves the hash of the input data and the convolution result into the data container hash_map in the form of key-value, where the hash value is used as the key and the convolution result is used as the value, and then returns the convolution result.

6. The data processing device according to claim 5, characterized in that The data container hash_map is empty or contains historical data loaded from a file.

7. The data processing device according to claim 5, characterized in that The query module includes: Comparison unit, used to compare and The difference of each bit in ; h is the hash value of the input data, It is a hash value selected from hash_map; The statistical unit is used to count the number of difference bits. If the number of differences is less than the set threshold D, it is considered that the query result is found, otherwise the next hash value in the hash_map is compared.

8. A data processing terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images