Image feature extraction method, device and system

Abstract

The present invention discloses an image feature extraction method, device and system, and relates to the field of image processing technology. The present invention expands the convolution kernel of the image convolution into multiple weight matrices, expands the input feature map into a total input matrix, extracts the sub-input matrix corresponding to each weight matrix from the total input matrix through a sliding value operation, multiplies each weight matrix with the corresponding sub-input matrix through a small matrix multiplication operation, obtains multiple intermediate result matrices, accumulates the multiple intermediate result matrices through an intermediate result accumulation operation, and finally obtains a feature matrix of an output feature map, and less additional storage space is required during the convolution calculation process, thereby reducing the memory resources occupied by the image convolution.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image feature extraction method, device and system. Background Art

[0002] Image convolution is a process of sliding a convolution kernel on an image, multiplying the grayscale value of an image pixel by the corresponding weight on the convolution kernel, and then adding all the multiplied values ​​as the grayscale value of the image pixel corresponding to the middle pixel of the convolution kernel, and finally sliding all the image calculations. Image convolution is mostly used for image classification and image detection. Image classification, such as face recognition and labeling in images taken by surveillance cameras, and image detection, such as product defect recognition based on product images in industrial production. At present, whether it is image convolution in the field of face recognition and labeling, or image convolution in the field of industrial product defect recognition, the existing image convolution methods have high spatial complexity and occupy a lot of memory resources. Summary of the invention

[0003] The present invention solves the technical problem of how to reduce the memory resources occupied by image convolution by providing an image feature extraction method, device and system.

[0004] On the one hand, the present invention provides the following technical solutions:

[0005] A method for extracting image features, comprising:

[0006] Get the convolution kernel of the image convolution and the input feature map to be processed;

[0007] Expand the convolution kernel to The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of the image convolution, is the number of input channels of the image convolution;

[0008] Expand the input feature map into a The total input matrix is is the height of the input feature map, is the width of the input feature map;

[0009] The size corresponding to each weight matrix extracted from the total input matrix is The sub-input matrix of is the height of the output feature map, is the width of the output feature map;

[0010] Multiply each weight matrix by the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0011] right The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

[0012] Optionally, the convolution kernel is expanded to The size is The weight matrix includes:

[0013] Based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different number of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

[0014] Optionally, the input feature map is expanded into a feature map of size The total input matrix includes:

[0015] Based on the principle that each element in the same input feature map is in the same row of the total input matrix and arranged in sequence, the input feature map is expanded into a matrix of size The total input matrix.

[0016] Optionally, the size corresponding to each weight matrix extracted from the total input matrix is The sub-input matrix includes:

[0017] Determine the number of rows i and columns j of the elements in the weight matrix in the convolution kernel;

[0018] Repeat the following process starting from n=1 until : Determine the target element with the number of rows [i+(n-1)s] and the number of columns j in the input feature map, where s is the convolution stride of the convolution kernel; starting from the column where the target element in the total input matrix is ​​located, continuously extract the target element in the total input matrix Column elements, get a single extraction matrix; let n=n+1;

[0019] Will Extracted The single extraction matrices are combined into the sub-input matrix corresponding to the weight matrix.

[0020] On the other hand, the present invention also provides the following technical solutions:

[0021] An image feature extraction device, comprising:

[0022] An acquisition module, used to obtain the convolution kernel of the image convolution and the input feature map to be processed;

[0023] An expansion module is used to expand the convolution kernel into The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of the image convolution, is the number of input channels of the image convolution;

[0024] And expand the input feature map into a size of The total input matrix is is the height of the input feature map, is the width of the input feature map;

[0025] An extraction module is used to extract the size corresponding to each weight matrix from the total input matrix. The sub-input matrix of is the height of the output feature map, is the width of the output feature map;

[0026] A calculation module is used to multiply each weight matrix with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0027] and The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

[0028] Optionally, the expansion module is specifically used to:

[0029] Based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different number of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

[0030] On the other hand, the present invention also provides the following technical solutions:

[0031] An image feature extraction system, comprising:

[0032] The first-level cache module is used to store the weight matrix, sub-input matrix and feature matrix of the output feature map in the image convolution process;

[0033] A multiply-accumulate array and an adder are used to multiply each weight matrix stored in the first-level cache module with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0034] A three-level cache module, used for storing the intermediate result matrix output by the adder;

[0035] An accumulator is used to store the data in the three-level cache module. The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map, and the feature matrix of the output feature map is output to the first-level cache module.

[0036] Optionally, the multiply-accumulate array is composed of a plurality of multiply-accumulate units and a secondary cache module corresponding to each of the multiply-accumulate units.

[0037] On the other hand, the present invention also provides the following technical solutions:

[0038] A computer device comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of any image feature extraction method.

[0039] On the other hand, the present invention also provides the following technical solutions:

[0040] A computer-readable storage medium stores a computer program, which implements the steps of any image feature extraction method when executed by a processor.

[0041] One or more technical solutions provided by the present invention have at least the following technical effects or advantages:

[0042] In the embodiment of the present invention, the convolution kernel of the image convolution is expanded as The size is The weight matrix of , expands the input feature map into a size of The total input matrix is ​​extracted from the total input matrix by sliding the value operation to obtain the size of each weight matrix. Sub-input matrices are multiplied by small matrix multiplication operations to obtain each weight matrix and the corresponding sub-input matrix. The size is The intermediate result matrix is ​​accumulated by the intermediate result operation. The intermediate result matrices are accumulated and finally a matrix of size The feature matrix of the output feature map has a spatial complexity of , less additional storage space is required, reducing the memory resources occupied by image convolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0044] Figure 1 This is a schematic diagram of the working principle of Img2Col in an embodiment of the present invention;

[0045] Figure 2 is a flow chart of an image feature extraction method according to an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of the principle of the image feature extraction method in an embodiment of the present invention;

[0047] Figure 4 Schematic diagram of the expansion principle of the convolution kernel and the input feature map in an embodiment of the present invention;

[0048] Figure 5 Schematic diagram of an image feature extraction system in an embodiment of the present invention;

[0049] Figure 6 Schematic diagram of an image feature extraction device in an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The embodiments of the present invention solve the technical problem of how to reduce the memory resources occupied by image convolution by providing an image feature extraction method, device and system.

[0051] In order to better understand the technical solution of the present invention, the technical solution of the present invention will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0052] Generally, image convolution can use the Im2Col (column sliding template) method, such as Figure 1 As shown, assuming the convolution kernel size for , input feature map size for , Number of input channels 2, the number of output channels is 1, and the Im2Col method can be used to obtain a The input matrix is The convolution matrix is ​​multiplied to output a It can be seen that the space complexity of Im2Col is , 40 (32+8) additional storage spaces are required to store the two-dimensional matrix, which occupies a lot of memory resources.

[0053] like Figure 2 As shown, the image feature extraction method of the embodiment of the present invention includes:

[0054] Step S1, obtaining a convolution kernel for image convolution and an input feature map to be processed;

[0055] Step S2, expand the convolution kernel to The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of image convolution, is the number of input channels of image convolution;

[0056] Step S3: Expand the input feature map into a The total input matrix is is the height of the input feature map, is the width of the input feature map;

[0057] Step S4, extract the size of each weight matrix from the total input matrix The sub-input matrix of is the height of the output feature map, is the width of the output feature map;

[0058] Step S5, multiply each weight matrix by the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0059] Step S6: The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

[0060] In step S1, after the convolutional neural network model is trained in the computer software, the convolution kernel of the convolutional neural network model is determined, so step S1 can obtain the convolution kernel from the model training software on the computer. The input feature map can be obtained from a public test set on the Internet, or it can be obtained after preliminary processing of the image taken by the camera.

[0061] Wherein, step S2 may specifically include: based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different number of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

[0062] Wherein, step S3 may specifically include: based on the principle that each element in the same input feature map is in the same row of the total input matrix and is arranged in sequence, the input feature map is expanded into a matrix of size The total input matrix.

[0063] Wherein, step S4 may specifically include: determining the number of rows i and the number of columns j of the elements in the weight matrix in the convolution kernel;

[0064] Repeat the following process starting from n=1 until : Determine the target element with the number of rows [i+(n-1)s] and the number of columns j in the input feature map, where s is the convolution stride of the convolution kernel; starting from the column where the target element in the total input matrix is ​​located, continuously extract the Column elements, get a single extraction matrix; let n=n+1;

[0065] Will Extracted The single extraction matrices are combined into the sub-input matrix corresponding to the weight matrix.

[0066] The process of steps S2-S6 is as follows Figure 3 shown. Figure 3 The dimension of the convolution kernel is , the dimension of the input feature map is , A11-A33 are the weights of the convolution kernel. Steps S4-S6 are Figure 3 The sliding multiplication and accumulation operation in step S4 includes the sliding value operation in step S4, the small matrix multiplication operation in step S5, and the intermediate result accumulation operation in step S6. After accumulating the sliding multiplication and accumulation operation, we can get intermediate result matrix, After accumulating the intermediate result matrices, we get the feature matrix of the output feature map. Reshaping the feature matrix can get a dimension of The output feature map of .

[0067] In the embodiment of the present invention, the convolution kernel size for , input feature map size for , Number of input channels 2, the number of output channels 1, output feature map size for To illustrate with an example, .

[0068] like Figure 4As shown in the figure, in the two convolution kernels, the number of rows and columns of elements A1 and A5 in the convolution kernel is the same, and A1 and A5 form a weight matrix; the number of rows and columns of elements A2 and A6 in the convolution kernel is the same, and A2 and A6 form a weight matrix; the number of rows and columns of elements A3 and A7 in the convolution kernel is the same, and A3 and A7 form a weight matrix; the number of rows and columns of elements A4 and A8 in the convolution kernel is the same, and A4 and A8 form a weight matrix; the two convolution kernels are expanded into 4 The weight matrix of .

[0069] In the two input feature maps, the elements B1-B9 in the first input feature map are in the first row of the total input matrix and are arranged in sequence, and the elements B10-B18 in the second input feature map are in the second row of the total input matrix and are arranged in sequence; the two input feature maps are expanded into one The total input matrix.

[0070] Set s=1. For the first weight matrix A1A5, the number of rows i and the number of columns j of elements A1 and A5 in the convolution kernel are both 1. First, n=1, and the first extraction is performed. [i+(n-1)s]=1. The target elements with a row number of 1 and a column number of 1 determined in the input feature map are B1 and B10. The first extraction starts from the column where the elements B1 and B10 in the total input matrix are located, that is, starting from the first column of the total input matrix, and continuously extracting 2 columns of elements B1B10 and B2B11 to obtain the first single extraction matrix. ; Then n=n+1=2, the second extraction is performed, [i+(n-1)s]=2, the target elements with 2 rows and 1 columns determined in the input feature map are B4 and B13, then the second extraction starts from the column where the elements B4 and B13 in the total input matrix are located, that is, starting from the 4th column of the total input matrix, and continuously extracts 2 columns of elements B4B13 and B5B14 to obtain the second single extraction matrix ; Then n=n+1=3, stop extraction. The two single extraction matrices of the two extractions are combined into the sub-input matrix corresponding to the first weight matrix ; Multiply the first weight matrix with the corresponding sub-input matrix to obtain the first intermediate result matrix m1: .

[0071] For the second weight matrix A2A6, the row number i of elements A2 and A6 in the convolution kernel is 1 and the column number j is 2. First, n=1, and the first extraction is performed, [i+(n-1)s]=1. The target elements with a row number of 1 and a column number of 2 determined in the input feature map are B2 and B11. The first extraction starts from the column where the elements B2 and B11 in the total input matrix are located, that is, starting from the second column of the total input matrix, and continuously extracting 2 columns of elements B2B11 and B3B12 to obtain the first single extraction matrix ; Then n=n+1=2, and the second extraction is performed. The target elements with 2 rows and 2 columns determined in the input feature map are elements B5 and B14. The second extraction starts from the columns where elements B5 and B14 are located in the total input matrix, that is, starting from the 5th column of the total input matrix, and continuously extracts 2 columns of elements B5B14 and B6B15 to obtain the second single extraction matrix ; Then n=n+1=3, stop extraction. The two single extraction matrices of the second extraction are combined into the sub-input matrix corresponding to the second weight matrix ; Multiply the second weight matrix with the corresponding sub-input matrix to obtain the second intermediate result matrix m2: .

[0072] Similarly, the third weight matrix A3A7 is multiplied with the corresponding sub-input matrix to obtain the third intermediate result matrix m3, and the fourth weight matrix A4A8 is multiplied with the corresponding sub-input matrix to obtain the fourth intermediate result matrix m4. Then, the four intermediate result matrices m1, m2, m3, and m4 are accumulated to finally obtain a matrix of size The feature matrix of the output feature map is a matrix of size The characteristic matrix of has a space complexity of , only need An additional storage space is used to store the intermediate result matrix, reducing the memory resources occupied by image convolution.

[0073] As can be seen from the above, the embodiment of the present invention expands the convolution kernel of the image convolution into The size is The weight matrix of , expands the input feature map into a size of The total input matrix is ​​extracted from the total input matrix by sliding the value operation to obtain the size of each weight matrix. Sub-input matrices are multiplied by small matrix multiplication operations to obtain each weight matrix and the corresponding sub-input matrix. The size is The intermediate result matrix is ​​accumulated by the intermediate result operation. The intermediate result matrices are accumulated and finally a matrix of size The feature matrix of the output feature map has a spatial complexity of , requiring less additional storage space, reducing the memory resources occupied by image convolution. The embodiments of the present invention are respectively based on Figure 2 The method and Im2Col method were used to conduct face recognition and labeling experiments in access control systems and strip steel product defect recognition experiments. The experimental results show that Figure 2 The method takes up less memory resources.

[0074] like Figure 5 As shown, the embodiment of the present invention also provides an image feature extraction system for implementing the calculations in steps S5-S6, including:

[0075] The first-level cache module is used to store the weight matrix, sub-input matrix and feature matrix of the output feature map in the image convolution process;

[0076] The multiply-accumulate array and adder are used to multiply each weight matrix stored in the first-level cache module with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0077] A three-level cache module for storing the intermediate result matrix output by the adder;

[0078] Accumulator, used to store the third-level cache module The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map, and the feature matrix of the output feature map is output to the first-level cache module.

[0079] The multiplication-accumulation array may be composed of a plurality of multiplication-accumulation units and a secondary cache module corresponding to each multiplication-accumulation unit. The secondary cache module is used to store the weight matrix and the sub-input matrix from the primary cache module. The multiplication-accumulation unit is used to perform the multiplication-accumulation operation on the weight matrix and the sub-input matrix (such as calculating the multiplication-accumulation operation in m1). and ), and input the operation result into the adder; the adder performs the addition operation of the output of each multiplication and accumulation unit (such as calculating the , , and ) and then stores the result in the third-level cache module; the accumulator is responsible for accumulating all intermediate result matrices stored in the third-level cache module (such as calculating m1+m2+m3+m4); after the convolution calculation is completed, the accumulator outputs the result to the first-level cache module, and the system master decides whether to output the result to external storage. When performing convolution calculations, when the amount of data (weights and inputs) exceeds the capacity of the accelerator, the master will fill the multiplication and accumulation array multiple times, and use "0" to fill the blank position in the last set of data to achieve the goal of fully utilizing computing resources.

[0080] To verify Figure 2The feasibility of the image feature extraction method and image feature extraction system thereof is investigated. The embodiment of the present invention uses LeNet convolutional neural network to perform MNIST (handwritten digit recognition) test, and uses Resnet18 (Resnet convolutional neural network with a depth of 18 layers) to perform Cifar10 (color image classification) test. The PL part of the Xilinx ZCU104 development board is used to implement the accelerator device, with the onboard ARM processor as the main control, and the onboard memory is used to implement the three-level cache. In the network, except for convolution, the remaining layers, quantization, and dequantization operations are all implemented using the ARM processor of Xilinx ZCU104. The MNIST data set is verified using Mnist_LeNet, and the network structure is shown in Table 1:

[0081] Table 1 Mnist_LeNet network structure table

[0082]

[0083] The number of parameters of the Mnist_LeNet network is 44.426 K, and the number of multiplication and accumulation calculations is 281.64 K.

[0084] The Cifar10 dataset is verified using ResNet18, and the network structure is shown in Table 2:

[0085] Table 2 Cifar10_ResNet18 network structure table

[0086]

[0087] The number of parameters of the Mnist_LeNet network is 11.181642 M, and the number of multiplication and accumulation calculations is 555.43 M.

[0088] Then follow Figure 2 The model is inferred by the method, and the test results are shown in Table 3:

[0089] Table 3 Network reasoning verification results

[0090]

[0091] like Figure 6 As shown, the embodiment of the present invention also provides an image feature extraction device, including:

[0092] An acquisition module, used to obtain the convolution kernel of the image convolution and the input feature map to be processed;

[0093] The expansion module is used to expand the convolution kernel into The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of image convolution, is the number of input channels of image convolution;

[0094] And expand the input feature map into a size of The total input matrix is is the height of the input feature map, is the width of the input feature map;

[0095] The extraction module is used to extract the size of each weight matrix from the total input matrix. The sub-input matrix of is the height of the output feature map, is the width of the output feature map;

[0096] The calculation module is used to multiply each weight matrix with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of

[0097] and The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

[0098] Furthermore, the expansion module can be used specifically for:

[0099] Based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different numbers of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

[0100] Furthermore, the expansion module can also be specifically used to expand the input feature map into a matrix of size The total input matrix.

[0101] Furthermore, the extraction module can be specifically used for:

[0102] Determine the number of rows i and columns j of the elements in the weight matrix in the convolution kernel;

[0103] Repeat the following process starting from n=1 until : Determine the target element with the number of rows [i+(n-1)s] and the number of columns j in the input feature map, where s is the convolution stride of the convolution kernel; starting from the column where the target element in the total input matrix is ​​located, continuously extract the Column elements, get a single extraction matrix; let n=n+1;

[0104] Will Extracted The single extraction matrices are combined into the sub-input matrix corresponding to the weight matrix.

[0105] Based on the same inventive concept as the above-mentioned image feature extraction method, an embodiment of the present invention also provides a computer device, including a memory, a processor and a computer program stored in the memory, and the processor executes the computer program to implement the steps of any one of the above-mentioned image feature extraction methods.

[0106] Among them, the bus architecture (represented by the bus), the bus can include any number of interconnected buses and bridges, and the bus links various circuits including one or more processors represented by the processor and memory represented by the memory. The bus can also link various other circuits such as peripherals, voltage regulators, and power management circuits together, which are all well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter can be the same element, i.e., a transceiver, which provides a unit for communicating with various other devices on a transmission medium. The processor is responsible for managing the bus and general processing, while the memory can be used to store data used by the processor when performing operations.

[0107] Since the computer device introduced in the embodiment of the present invention is a computer device used to implement the image feature extraction method in the embodiment of the present invention, based on the image feature extraction method introduced in the embodiment of the present invention, a person skilled in the art can understand the specific implementation of the computer device in the embodiment of the present invention and its various variations, so how the computer device implements the method in the embodiment of the present invention is not described in detail here. As long as a person skilled in the art implements the computer device used in the image feature extraction method in the embodiment of the present invention, it belongs to the scope of protection of the present invention.

[0108] Based on the same inventive concept as the above-mentioned image feature extraction method, the present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned image feature extraction methods are implemented.

[0109] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0114] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for extracting image features, characterized in that: include: Get the convolution kernel of the image convolution and the input feature map to be processed; Expand the convolution kernel to The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of the image convolution, is the number of input channels of the image convolution; Expand the input feature map into a The total input matrix of 𝑊, is the height of the input feature map, is the width of the input feature map; The size corresponding to each weight matrix extracted from the total input matrix is The sub-input matrix of is the height of the output feature map, is the width of the output feature map; Multiply each weight matrix by the corresponding sub-input matrix to obtain The size is The intermediate result matrix of right The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

2. The image feature extraction method according to claim 1, characterized in that: The convolution kernel is expanded into The size is The weight matrix includes: Based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different number of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

3. The image feature extraction method according to claim 1, characterized in that: The input feature map is expanded into a The total input matrix of 𝑊, including: Based on the principle that each element in the same input feature map is in the same row of the total input matrix and arranged in sequence, the input feature map is expanded into a matrix of size The total input matrix of 𝑊.

4. The image feature extraction method according to claim 1, characterized in that: The size corresponding to each weight matrix extracted from the total input matrix is The sub-input matrix includes: Determine the number of rows i and columns j of the elements in the weight matrix in the convolution kernel; Repeat the following process starting from n=1 until n= +1: Determine the target element with row number [i+(n-1)s] and column number j in the input feature map, where s is the convolution stride of the convolution kernel; start from the column where the target element in the total input matrix is ​​located, and continuously extract the target element in the total input matrix. Column elements, get a single extraction matrix; let n=n+1; Will Extracted The single extraction matrices are combined into the sub-input matrix corresponding to the weight matrix.

5. An image feature extraction device, characterized in that: include: An acquisition module, used to obtain the convolution kernel of the image convolution and the input feature map to be processed; An expansion module is used to expand the convolution kernel into The size is The weight matrix of is the height of the convolution kernel, is the width of the convolution kernel, is the number of output channels of the image convolution, is the number of input channels of the image convolution; And expand the input feature map into a size of The total input matrix of 𝑊, is the height of the input feature map, is the width of the input feature map; An extraction module is used to extract the size corresponding to each weight matrix from the total input matrix. The sub-input matrix of is the height of the output feature map, is the width of the output feature map; A calculation module is used to multiply each weight matrix with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of and The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map.

6. The image feature extraction device according to claim 5, characterized in that: The expansion module is specifically used for: Based on the principle that elements with the same number of rows and columns in different convolution kernels are in the same weight matrix, and elements with different number of rows and columns in the same convolution kernel are in different weight matrices, the convolution kernel is expanded into The size is The weight matrix of .

7. An image feature extraction system, characterized in that: include: The first-level cache module is used to store the weight matrix, sub-input matrix and feature matrix of the output feature map in the image convolution process; The size of the weight matrix is , is the number of output channels of the image convolution, is the number of input channels of the image convolution; The sub-input matrix is ​​extracted from the total input matrix and corresponds to the weight matrix, and the total input matrix is ​​obtained by expanding the input feature map; The total input matrix size is 𝑊, is the height of the input feature map, is the width of the input feature map; The size of the sub-input matrix is , is the height of the output feature map, is the width of the output feature map; A multiply-accumulate array and an adder are used to multiply each weight matrix stored in the first-level cache module with the corresponding sub-input matrix to obtain The size is The intermediate result matrix of is the height of the convolution kernel of the image convolution, is the width of the convolution kernel; A three-level cache module, used for storing the intermediate result matrix output by the adder; An accumulator is used to store the data in the three-level cache module. The intermediate result matrices are accumulated to obtain the feature matrix of the output feature map, and the feature matrix of the output feature map is output to the first-level cache module.

8. The image feature extraction system according to claim 7, characterized in that: The multiplication-accumulation array is composed of a plurality of multiplication-accumulation units and a secondary cache module corresponding to each of the multiplication-accumulation units.

9. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 4.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images