Image recognition method and related device

Abstract

The present application discloses an image recognition method and a related device, wherein the image recognition method aims at the problem that in the image recognition process, the original weight data and the original image data contain a large amount of zero data that is not helpful for the recognition process, and proposes a solution of sparse weight mask and sparse weight data. A large amount of zero data is eliminated through sparse weight mask and sparse weight data, which greatly simplifies the data volume of effective vectors and sparse weight data for deep convolution calculation, achieves the purpose of reducing the amount of data calculation in the image recognition process, and reduces the requirements of image recognition for hardware resources.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to an image recognition method and related devices. Background Art

[0002] Deep learning technology based on convolutional neural networks has been widely used in various fields. However, from the perspective of using convolutional neural network models to solve problems, people often tend to design deeper and larger convolutional neural networks (CNN) to collect more data in order to obtain better results.

[0003] However, as the complexity of the model increases, the model parameters become more and more, and the hardware resources required for specific applications based on the model also increase. How to reduce the amount of data calculation in the specific application process has become an urgent problem to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides an image recognition method and related devices to achieve the purpose of reducing the amount of data calculation in the image recognition process and reducing the requirements of image recognition for hardware resources.

[0005] To achieve the above technical objectives, the present application provides the following technical solutions:

[0006] An image recognition method, comprising:

[0007] Obtain original image data and original weight data;

[0008] The original weight data are grouped, and a sparse weight mask and sparse weight data are generated based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero;

[0009] Filtering the original image data using the sparse weight mask to obtain a valid vector;

[0010] A deep convolution calculation is performed using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

[0011] Optionally, grouping the original weight data includes:

[0012] According to the order of the original weight data, N adjacent original weight data are grouped into one group, where N is an integer greater than or equal to 2.

[0013] Optionally, generating sparse weight masks and sparse weight data based on each group of original weight data includes:

[0014] When all data in a group of the original weight data are zero, generating first state data as a sparse weight mask corresponding to the group of the original weight data, and deleting the group of the original weight data;

[0015] When the data in a group of the original weight data is not all zero, second state data is generated as a sparse weight mask corresponding to the group of the original weight data, and the group of the original weight data is retained as sparse weight data corresponding to the group of the original weight data.

[0016] Optionally, the using the sparse weight mask to filter the original image data to obtain a valid vector includes:

[0017] Grouping the original image data to obtain multiple groups of image data units;

[0018] When a sparse weight mask corresponding to a group of the image data units is first state data, discarding the image data units;

[0019] When a sparse weight mask corresponding to a group of the image data units is second state data, retaining the image data units;

[0020] All the reserved image data units are arranged in sequence to obtain the valid vector.

[0021] Optionally, the number of data in each group of image data units is the same as the number of data in each group of original weight data.

[0022] Optionally, the first state data is 0, and the second state data is 1.

[0023] Optionally, after grouping the original weight data and generating a sparse weight mask and sparse weight data based on each group of original weight data, before filtering the original image data using the sparse weight mask to obtain a valid vector, the method further includes:

[0024] The sparse weight mask is compressed using Huffman coding.

[0025] An image recognition system, comprising:

[0026] The original data module is used to obtain the original image data and the original weight data;

[0027] A weight grouping module, used to group the original weight data, generate a sparse weight mask and sparse weight data based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero;

[0028] A data screening module, used for screening the original image data using the sparse weight mask to obtain a valid vector;

[0029] A convolution calculation module is used to perform a deep convolution calculation using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

[0030] An image recognition system, comprising: a memory and a processor;

[0031] The memory stores a program code, the processor is used to call the program code, and the program code is used to execute any one of the above-mentioned image recognition methods.

[0032] A storage medium having program code stored thereon, wherein the program code implements any of the above-mentioned image recognition methods when executed.

[0033] It can be seen from the above technical scheme that the embodiment of the present application provides an image recognition method and a related device, wherein the image recognition method proposes a solution of sparse weight masks and sparse weight data to address the problem that in the image recognition process, the original weight data and the original image data contain a large amount of zero data that is not helpful for the recognition process. A large amount of zero data is eliminated through sparse weight masks and sparse weight data, which greatly simplifies the data volume of effective vectors and sparse weight data for deep convolution calculations, thereby achieving the purpose of reducing the amount of data calculation in the image recognition process and reducing the requirements of image recognition for hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 A schematic diagram of a convolution operation in an image recognition process in the prior art;

[0036] Figure 2 A schematic diagram of a flow chart of an image recognition method provided for one embodiment of the present application;

[0037] Figure 3 A schematic diagram of the process of performing convolution operation using the present method (right figure) and the image recognition method in the prior art (left figure) is shown;

[0038] Figure 4 A flowchart of an image recognition method provided for another embodiment of the present application;

[0039] Figure 5 A flowchart of an image recognition method provided for another embodiment of the present application;

[0040] Figure 6 The process of determining a sparse weight mask for original weight data in an embodiment of the present application is shown;

[0041] Figure 7 A flowchart of an image recognition method provided for yet another embodiment of the present application;

[0042] Figure 8 A schematic diagram showing how to use a sparse weight mask W to filter the original image data R to obtain a valid vector P;

[0043] Fig. 9 A flowchart of an image recognition method provided for an optional embodiment of the present application. DETAILED DESCRIPTION

[0044] As described in the background technology, as the complexity of the model increases, the model parameters increase, and the specific application based on the model requires more and more hardware resources. The inventors have found that, especially for image recognition applications, Figure 1 As shown, there are a large number of zero data (i.e., data with a value of zero) in the original image data. Accordingly, there are also a large number of zero data in the weight data used for original image data recognition. These zero data will not contribute to subsequent image recognition, but will lead to a large number of zero intermediate calculation results during the convolution operation. Specifically, since there are a large number of zero data in the weight data (such as weight 310), if these zero data also enter the deep convolution core module to participate in the operation, a large number of zero intermediate calculation results (such as intermediate results 320, 330, 340) will be generated. This will waste the computing resources of the deep convolution core module and prolong the convolution operation time.

[0045] In order to solve this problem, an embodiment of the present application provides an image recognition method, including:

[0046] Obtain original image data and original weight data;

[0047] The original weight data are grouped, and a sparse weight mask and sparse weight data are generated based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero;

[0048] Filtering the original image data using the sparse weight mask to obtain a valid vector;

[0049] A deep convolution calculation is performed using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

[0050] The image recognition method aims to solve the problem that in the image recognition process, the original weight data and the original image data contain a large amount of zero data that is not helpful for the recognition process. The solution of sparse weight mask and sparse weight data is proposed. A large amount of zero data is eliminated through sparse weight mask and sparse weight data, which greatly simplifies the data volume of effective vectors and sparse weight data for deep convolution calculation, achieves the purpose of reducing the amount of data calculation in the image recognition process, and reduces the requirements of image recognition for hardware resources.

[0051] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0052] The present application embodiment provides an image recognition method, such as Figure 2 As shown, including:

[0053] S101: Acquire original image data and original weight data.

[0054] In step S101, the original image data and the original weight data may be pre-stored or cached in a storage device, and the original image data and the original weight data may be obtained by reading.

[0055] In addition, the original image data and original weight data can also be input in real time based on an input device. The present application does not limit the specific method for obtaining the original image data and original weight data, which depends on the actual situation.

[0056] The original weight data may be general weight data applied to image recognition, or may be specifically set according to a specific application scenario.

[0057] S102: Grouping the original weight data, generating a sparse weight mask and sparse weight data based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero.

[0058] In step S102, the generated sparse weight mask is used to characterize whether each group of original weight data is all zero data. Therefore, the sparse weight mask has only two states, one state characterizes that a group of original weight data is all zero data, and the other state characterizes that a group of original weight data is not all zero data.

[0059] S103: Filter the original image data using the sparse weight mask to obtain a valid vector.

[0060] In step S103, the original image data is screened using the sparse weight mask, so as to remove a large amount of zero data in the original image data, thereby alleviating the amount of calculation in subsequent deep convolution calculations.

[0061] S104: Performing a deep convolution calculation using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

[0062] In step S104, since a large amount of zero data is eliminated based on the sparse weight mask in the process of generating the valid vector, and a large amount of zero data in the original weight data is also eliminated in the sparse weight data, the invalid calculation of zero data in the deep convolution calculation process is greatly eliminated, and the data amount of the valid vector and sparse weight data for the deep convolution calculation is greatly simplified, thereby achieving the purpose of reducing the amount of data calculation in the image recognition process and reducing the requirements of image recognition for hardware resources.

[0063] Specific reference Figure 3 , Figure 3 A schematic diagram of the process of performing convolution operations using this method (right figure) and the image recognition method in the prior art (left figure) is shown. In the unsparse convolutional neural network calculation process, since there is a large amount of zero data in the weight data (such as weight 310), if these zero data also enter the deep convolution core module to participate in the calculation, a large number of zero intermediate calculation results (such as intermediate results 320, 330, 340) will be generated. This will waste the computing resources of the deep convolution core module and extend the convolution operation time. The method proposed in the embodiment of the present application eliminates a large amount of repeated zero data (such as weight 360) and useless vectors (such as 350) in the weight during the convolution calculation process, improves the efficiency of the convolution operation, and reduces the convolution operation time.

[0064] pass Figure 3It is not difficult to find from the comparison that the image recognition method provided in the embodiment of the present application eliminates a large amount of zero data based on the characteristics of image recognition, simplifies the convolution operation process, and improves the algorithm operation efficiency.

[0065] The following describes the feasible execution process of each step of the image recognition method provided in the embodiment of the present application.

[0066] Optional, reference Figure 4 , the grouping of the original weight data comprises:

[0067] S1021: Grouping N adjacent original weight data into one group according to the order of the original weight data, where N is an integer greater than or equal to 2.

[0068] In this embodiment, a specific method for grouping original weight data is provided. Generally, the order of the original weight data corresponds to the order of convolution processing on the original image data. Therefore, in the grouping process, it is necessary to group them in sequence according to the order of the original weight data. The value of N can be a positive integer such as 2, 3, 4, etc. The specific value can be determined according to actual needs, and this application does not limit this.

[0069] Optional, reference Figure 5 and Figure 6 , the generating of sparse weight masks and sparse weight data based on each group of original weight data includes:

[0070] S1022: When all data in a group of the original weight data are zero, generate first state data as a sparse weight mask corresponding to the group of the original weight data, and delete the group of the original weight data;

[0071] S1023: When the data in a group of the original weight data are not all zero, generate second state data as a sparse weight mask corresponding to the group of the original weight data, and retain the group of the original weight data as the sparse weight data corresponding to the group of the original weight data.

[0072] Figure 6 A specific original weight data is taken as an example to illustrate the process of determining a sparse weight mask for the original weight data.

[0073] exist Figure 6In the original weight data groups 410 and 420, if the data in 410 is not all zero, the weight sparse mask W corresponds to the second state data (that is, it can be selected as 1), and if the data in 420 is all zero, the weight sparse mask W corresponds to the first state data (that is, it can be selected as 0, that is, the first state data can be selected as 0 and the second state data can be selected as 1). Each bit of the sparse weight mask represents the state of the corresponding original weight data group, such as the original weight group 410 corresponds to the sparse weight mask 430, and the original weight group 420 corresponds to the sparse weight mask 440.

[0074] Optional, reference Figure 7 , the using the sparse weight mask to filter the original image data to obtain a valid vector comprises:

[0075] S1031: Group the original image data to obtain multiple groups of image data units.

[0076] In order to ensure that each group of image data units corresponds to the data in each group of original weight data, the method of grouping the original image data needs to be the same as the method of grouping the original weight data, that is, the number of data in each group of image data units is the same as the number of data in each group of original weight data.

[0077] S1032: When a sparse weight mask corresponding to a group of the image data units is first state data, discard the image data units;

[0078] S1033: When a sparse weight mask corresponding to a group of the image data units is second state data, retain the image data units;

[0079] S1034: Arrange all the retained image data units in order to obtain the valid vector.

[0080] refer to Figure 8 , Figure 8 FIG. 1 shows a schematic diagram of using a sparse weight mask W to screen the original image data R to obtain a valid vector P. Figure 8 In the sparse weight mask W, each bit represents two states: first state data and second state data. If the state of the sparse weight mask 610 is the second state data, it means that the data of the corresponding original data group 630 is available data and needs to be output to the deep convolution core module and placed in the valid vector group 650; if the state of the sparse weight mask 620 is the first state data, it means that the data of the corresponding original data group 640 is unavailable data and does not need to be output to the deep convolution core module and is directly discarded. After being filtered by the weight sparse mask W, the original image data R is transposed into a valid vector P.

[0081] Based on the above embodiment, in an optional embodiment of the present application, as Fig. 9 As shown, after the original weight data is grouped and a sparse weight mask and sparse weight data are generated based on each group of original weight data, before the original image data is filtered using the sparse weight mask to obtain a valid vector, the method further includes:

[0082] S105: Compress the sparse weight mask using Huffman coding.

[0083] In this embodiment, the sparse weight mask is compressed using Huffman coding to compress the storage space of the sparse weight mask when there is a large amount of continuous data of the same state (first state data or second state data) in the sparse weight mask. The compressed sparse weight mask can be used to filter the original image data while decoding, that is, the sparse weight mask is used to filter the original image data to obtain a valid vector, including: decoding the compressed sparse weight mask, and filtering the original image data using the decoded sparse weight mask.

[0084] The image recognition system provided in the embodiment of the present application is described below. The image recognition system described below can be referenced in correspondence with the image recognition method described above.

[0085] Accordingly, the embodiment of the present application further provides an image recognition system, including:

[0086] The original data module is used to obtain the original image data and the original weight data;

[0087] A weight grouping module, used to group the original weight data, generate a sparse weight mask and sparse weight data based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero;

[0088] A data screening module, used for screening the original image data using the sparse weight mask to obtain a valid vector;

[0089] A convolution calculation module is used to perform a deep convolution calculation using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

[0090] Accordingly, an embodiment of the present application further provides an image recognition system, comprising: a memory and a processor;

[0091] The memory stores a program code, the processor is used to call the program code, and the program code is used to execute the image recognition method described in any of the above embodiments.

[0092] Correspondingly, an embodiment of the present application further provides a storage medium, on which program code is stored, and when the program code is executed, the image recognition method described in any of the above embodiments is implemented.

[0093] In summary, the embodiments of the present application provide an image recognition method and a related device thereof, wherein the image recognition method proposes a solution of sparse weight masks and sparse weight data to address the problem that in the image recognition process, the original weight data and the original image data contain a large amount of zero data that is not helpful for the recognition process. A large amount of zero data is eliminated through sparse weight masks and sparse weight data, which greatly simplifies the data volume of effective vectors and sparse weight data for deep convolution calculations, thereby achieving the purpose of reducing the amount of data calculation in the image recognition process and reducing the requirements of image recognition for hardware resources.

[0094] The features described in the embodiments of the present specification may be replaced or combined with each other. Each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments may be referenced to each other.

[0095] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image recognition method, characterized in that: include: Obtain original image data and original weight data; According to the order of the original weight data, N adjacent original weight data are grouped into one group, where N is an integer greater than or equal to 2; based on each group of original weight data, a sparse weight mask and sparse weight data are generated, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero; Compressing the sparse weight mask using Huffman coding; Decode the compressed sparse weight mask, and directly use part of the information of the decoded sparse weight mask to filter the original image data to obtain a valid vector; A deep convolution calculation is performed using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

2. The method according to claim 1, characterized in that The generating of sparse weight masks and sparse weight data based on each group of original weight data includes: When all data in a group of the original weight data are zero, generating first state data as a sparse weight mask corresponding to the group of the original weight data, and deleting the group of the original weight data; When the data in a group of the original weight data is not all zero, second state data is generated as a sparse weight mask corresponding to the group of the original weight data, and the group of the original weight data is retained as sparse weight data corresponding to the group of the original weight data.

3. The method according to claim 2, characterized in that The using the sparse weight mask to filter the original image data to obtain a valid vector comprises: Grouping the original image data to obtain multiple groups of image data units; When a sparse weight mask corresponding to a group of the image data units is first state data, discarding the image data units; When a sparse weight mask corresponding to a group of the image data units is second state data, retaining the image data units; All the reserved image data units are arranged in sequence to obtain the valid vector.

4. The method according to claim 3, characterized in that The number of data in each group of the image data units is the same as the number of data in each group of original weight data.

5. The method according to claim 4, characterized in that The first state data is 0, and the second state data is 1.

6. An image recognition system, characterized in that: include: The original data module is used to obtain the original image data and the original weight data; A weight grouping module, used to group N adjacent original weight data into one group according to the order of the original weight data, where N is an integer greater than or equal to 2; generate a sparse weight mask and sparse weight data based on each group of original weight data, wherein the sparse weight mask indicates whether each group of original weight data is all zero data, and the sparse weight data is generated based on each group of original weight data that is not all zero; A data screening module is used to compress the sparse weight mask using Huffman coding; decode the compressed sparse weight mask, and simultaneously directly use part of the information of the decoded sparse weight mask to screen the original image data to obtain a valid vector; A convolution calculation module is used to perform a deep convolution calculation using the effective vector and the sparse weight data to obtain a deep convolution result corresponding to the original image data.

7. An image recognition system, characterized in that: include: Memory and processor; The memory stores a program code, the processor is used to call the program code, and the program code is used to execute the image recognition method according to any one of claims 1 to 5.

8. A storage medium, characterized in that: The storage medium stores program codes, which, when executed, implement the image recognition method according to any one of claims 1 to 5.

Images