An efficient image feature extraction method based on deep learning

Abstract

The application discloses a kind of based on deep learning's image feature efficient extraction method, comprising: obtaining image data, the first feature map is obtained by convolution processing to image data, based on the first feature map obtains second feature map data, based on second feature map data, obtain first even data and second even data, to second even data is convoluted and adjacent channel separation, obtain first similar feature data and second similar feature data, to second similar feature data is depth separable convolution and with first similar feature data are spliced, obtain first feature data, first feature data and first even data are spliced, obtain second feature data, based on second feature data obtains third feature map data, based on third feature map data obtains target class number data, based on target class number data, obtain predicted image class.The above technical scheme can carry out fast inference to image data and extract feature more efficiently.

Description

Technical Field

[0001] This invention belongs to the field of computer image processing, and in particular relates to an efficient image feature extraction method based on deep learning. Background Technology

[0002] Deep learning-based inference models have wide applications in various terminal tasks based on computer vision, such as image recognition, object tracking, and instance segmentation. Early convolutional neural network-based methods, while achieving high accuracy, often involved large numbers of parameters and computational overhead. Existing lightweight inference networks have been improved for mobile devices with limited computing power, such as CPUs and ARM architectures, resulting in faster inference speeds. However, this sacrifices accuracy, and the inference speed on high-throughput GPUs remains unsatisfactory. Recent work has focused on introducing the self-attention mechanism of Transformers to achieve higher accuracy, but this increases network complexity while improving detection accuracy, resulting in unsatisfactory inference speed. It is clear that designing a network that maintains high inference speed on both low-end and high-performance devices is a challenge. Current deep learning models cannot simultaneously address the aforementioned problems and efficiently infer image data on both low-end and high-performance devices. Summary of the Invention

[0003] To address the problems of existing technologies that cannot perform high-speed inference on image data and cannot take into account device performance, the purpose of this invention is to provide a deep learning-based method for efficient image feature extraction. This method can efficiently and accurately extract features from image data, effectively extract image features and predict image categories, and is applicable to different device performance levels, thereby solving the problems of the aforementioned existing technologies.

[0004] To achieve the above objectives, this invention provides an efficient image feature extraction method based on deep learning, which can be used for image recognition. The method includes: acquiring image data; performing convolution processing on the image data to obtain first feature map data; performing depthwise separable convolution on the first feature map data to obtain second feature map data; performing feature channel equalization on the second feature map data to obtain first equalization data and second equalization data; performing convolution and adjacent channel separation on the second equalization data to obtain first similar feature data and second similar feature data; performing depthwise separable convolution on the second similar feature data and concatenating it with the first similar feature data to obtain first feature data; concatenating the first feature data and the first equalization data to obtain second feature data; acquiring third feature map data based on the second feature data; obtaining target category number data based on the third feature map data; and obtaining the image category corresponding to the image data based on the target category number data.

[0005] Optionally, the process of obtaining the first feature map data includes:

[0006] The image data is convolved using a convolution kernel with a stride of 2 and a kernel size of 3*3. The number of feature channels of the convolution kernel is increased, and the convolved image data is downsampled to obtain the first feature map data.

[0007] Optionally, the process of obtaining the second feature map data includes:

[0008] An activation function is constructed to perform batch normalization on the first feature map data. The normalized first feature map data is then activated using the activation function. The activated first feature map data is then subjected to depthwise separable convolution using a convolution kernel with a stride of 2 and a kernel size of 3*3. The convolved image data is then downsampled to obtain the second feature map data.

[0009] Optionally, the process of obtaining the first similarity feature data and the second similarity feature data includes:

[0010] The second feature map data is batch normalized. Then, the normalized second feature map data is convolved with a 1*1 kernel. The convolved second feature map data is activated with an activation function. The activated second feature map data is then divided into two equal feature channels to obtain first and second equal feature data. The second equal feature data is then convolved and separated into adjacent channels to obtain first and second similar feature data.

[0011] Optionally, the process of obtaining the second feature data includes:

[0012] The second similarity feature data is batch normalized. The normalized second similarity feature data is then convolved with a 1*1 convolution kernel. The convolutional second similarity feature data is then activated with an activation function. The activated second similarity feature data is then concatenated with the first similarity feature data to obtain the first feature data. Finally, the first feature data is concatenated with the first average data to obtain the second feature data.

[0013] Optionally, the process of obtaining the third feature map data includes:

[0014] The second feature data is shuffled through channels, and the features of the shuffled second feature data are recombined to obtain the third feature map data downsampled by 32 times.

[0015] Optionally, the process of obtaining the number of target categories includes:

[0016] A fully connected layer is constructed to perform global average pooling on the third feature map data. The number of features is then transformed on the average pooled third feature map data through the fully connected layer to obtain the number of target categories.

[0017] Optionally, the process of obtaining the image category corresponding to the image data includes:

[0018] The probability distribution of the target category is obtained by processing the target category number data using the softmax function. The image category corresponding to the image data is obtained by extracting the category corresponding to the maximum value of the probability distribution.

[0019] The technical effects of this invention are as follows:

[0020] (1) The present invention can perform fast inference on image data. Specifically, the feature extraction structure designed in the present invention uses fewer batch normalization and activation functions, and removes the max pooling layer, which makes the model inference faster.

[0021] (2) This invention can effectively infer images while taking into account the performance of different devices. Specifically, the feature extraction structure designed in this invention minimizes the memory access cost of the model, which can effectively improve the training and inference speed of the model. At the same time, the feature grouping method designed in this invention retains some of the feature information extracted above without generating more memory consumption and computation, making the model extract features more efficiently. Attached Figure Description

[0022] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0023] Figure 1 This is a network structure diagram of an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the feature grouping structure according to an embodiment of the present invention;

[0025] Figure 3 This is a flowchart illustrating the training and inference process of an image recognition model built using the structure of an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] Example 1

[0028] like Figures 1-3 As shown, this embodiment provides an efficient image feature extraction method based on deep learning, including:

[0029] Image data is acquired, and the image data is convolved to obtain first feature map data. The first feature map data is then subjected to depthwise separable convolution to obtain second feature map data. The second feature map data is then divided into feature channels to obtain first and second equally divided data. The second equally divided data is then convolved and separated into adjacent channels to obtain first and second similar feature data. The second similar feature data is then subjected to depthwise separable convolution and concatenated with the first similar feature data to obtain first feature data. The first feature data and the first equally divided data are then concatenated to obtain second feature data. Based on the second feature data, third feature map data is acquired. Based on the third feature map data, the number of target categories is obtained. Based on the number of target categories, the image category corresponding to the image data is obtained.

[0030] This invention can be effectively applied to car recognition scenarios. Specifically, it involves acquiring a car image, performing convolution processing on the image data to obtain first feature map data, performing depthwise separable convolution on the first feature map data to obtain second feature map data, performing equal feature channel division on the second feature map data to obtain first equal-divided data and second equal-divided data, performing convolution and adjacent channel separation on the second equal-divided data to obtain first similar feature data and second similar feature data, performing depthwise separable convolution on the second similar feature data and concatenating it with the first similar feature data to obtain first feature data, concatenating the first feature data and the first equal-divided data to obtain second feature data, acquiring third feature map data based on the second feature data, obtaining target category number data based on the third feature map data, processing the target category number data using the softmax function to obtain the probability distribution of the target categories, and then extracting the maximum value of the probability distribution to obtain the predicted car information such as brand, model, and year.

[0031] The network structure of this invention embodiment is as follows: Figure 1 As shown, a Block is a convolutional block structure with a stride of 1. The middle diagram shows a feature extraction structure of an S2 stage. An S2 stage contains N Blocks, where N is a hyperparameter, which is explained in detail in Tables 1 and 2.

[0032] The feature grouping structure of this invention embodiment is as follows: Figure 2 As shown, the left image is Figure 1 The feature grouping structure in a block, as shown in the diagram on the right, involves separating adjacent channels to form two sets of features.

[0033] The flowchart of model training and inference for using the structure of this invention in vehicle image recognition is as follows: Figure 3As shown, during training, the gradient is updated by calculating the error between the predicted result and the true label, and backpropagation is performed to each parameter, which is then updated again until the end. The model parameters of the best result during training are saved, and the weight parameters are loaded during inference. The probability distribution of the target class is obtained by applying the softmax function to the target class data, and the predicted car image class is obtained by extracting the maximum value.

[0034] The detailed parameter configurations of the models in this embodiment of the invention are shown in Tables 1 and 2. Table 1 shows the detailed structural configuration of the small model, which has 0.3M parameters and 57M Flops. Table 2 shows the detailed structural configuration of the larger model, which has 1.1M parameters and 185M Flops.

[0035] Table 1 shows the model structure details of the 0.3M parameters in Embodiment 1 of the present invention; Table 2 shows the model structure details of the 1.1M parameters in Embodiment 1 of the present invention.

[0036] Table 1

[0037] InputSize Operator Stride InputChannels OutputChannels Repeat <![CDATA[224 2 ]]> Conv2d 2 3 40 1 <![CDATA[112 2 ]]> DownsampleLayer 2 40 80 1 <![CDATA[56 2 ]]> Block 1 80 80 2 <![CDATA[56 2 ]]> DownsampleLayer 2 80 160 1 <![CDATA[28 2 ]]> Block 1 160 160 2 <![CDATA[28 2 ]]> DownsampleLayer 2 160 320 1 <![CDATA[14 2 ]]> Block 1 320 320 6 <![CDATA[14 2 ]]> DownsampleLayer 2 320 640 1 <![CDATA[7 2 ]]> Block 1 640 640 2 <![CDATA[7 2 ]]> GlobalPool - - - 1 <![CDATA[1 2 ]]> FC - 640 196 1

[0038] Table 2

[0039] InputSize Operator Stride InputChannels OutputChannels Repeat <![CDATA[224 2 ]]> Conv2d 2 3 80 1 <![CDATA[112 2 ]]> DownsampleLayer 2 80 160 1 <![CDATA[56 2 ]]> Block 1 160 160 2 <![CDATA[56 2 ]]> DownsampleLayer 2 160 320 1 <![CDATA[28 2 ]]> Block 1 320 320 2 <![CDATA[28 2 ]]> DownsampleLayer 2 320 640 1 <![CDATA[14 2 ]]> Block 1 640 640 6 <![CDATA[14 2 ]]> DownsampleLayer 2 640 1280 1 <![CDATA[7 2 ]]> Block 1 1280 1280 2 <![CDATA[7 2 ]]> GlobalPool - - - 1 <![CDATA[1 2 ]]> FC - 1280 196 1

[0040] S1: Perform a normal convolution with a stride of 2 and a kernel size of 3*3 on the input car image to expand the number of feature channels and obtain the first feature map data downsampled by half. Then perform a batch normalization operation and a ReLU function activation operation.

[0041] S2: Perform a depthwise separable convolution on the feature map of S1 with a stride of 2 and a kernel size of 3*3. Downsample the feature map by half, perform batch normalization, then perform a 1*1 convolution, and finally activate it with the ReLU function; second feature map data.

[0042] S2.1: Divide the feature channels of the second feature map data in S2 into equal parts to obtain the first equal part data and the second equal part data. Keep one part and feed the second part into N blocks for convolution operation, where N is a hyperparameter.

[0043] S2.1.1: A block is formed by first separating one of every two adjacent channels to obtain two sets of similar features, namely the first similar feature data and the second similar feature data. The first set is retained as the context feature, and the second set is further convolved.

[0044] S2.1.2: The second similarity feature data in S2.1.1 is first subjected to a 3*3 depthwise separable convolution, then batch normalization, then a 1*1 convolution, and finally activated by the ReLU function;

[0045] S2.1.3: Concatenate the features obtained in S2.1.2 with the features mentioned above in S2.1.1;

[0046] S2.2: Combine the first feature set in S2.1 with the concatenated feature set in S2.1.3 to form a new feature set;

[0047] S2.3: Perform channel shuffling on the spliced ​​features from S2.2 and recombine them;

[0048] S3: Repeat all steps of S2 three more times to obtain the third feature map data downsampled by 32 times;

[0049] S4: Perform global average pooling on the last two dimensions of the first similarity feature data in S3;

[0050] S5: The feature map obtained in S4 is passed through a fully connected layer to transform the number of features into the number of target categories.

[0051] S6: The target category number data is processed by the softmax function to obtain the probability distribution of the target category, and then the maximum value of the probability distribution is extracted to obtain the predicted car brand, model, year, etc.

[0052] Embodiment 1 of this invention studies the roles of different structures in convolutional neural networks during image feature extraction, explores more efficient feature extraction structures, and proposes a lightweight and fast image feature extraction network. First, it reduces intensive memory access by decreasing batch normalization operations and activation functions without weakening feature extraction capabilities. Second, it minimizes memory access costs by adjusting the number of input channels to equal the number of output channels in each convolutional kernel. Third, it reduces the use of 1x1 convolutional kernels, decreasing the number of parameters and computational cost, and compensates for the accuracy loss caused by reducing the number of layers through operations such as channel shuffling. Fourth, it alternately separates adjacent channels into two parts, skipping some convolutional layers in one part to accelerate gradient propagation and improve convergence speed, while preserving some early features to prevent performance degradation, while the other part continues normal convolutional operations.

[0053] The above description is merely a preferred embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for efficient image feature extraction based on deep learning, characterized in that, include: Image data is acquired, and the image data is convolved to obtain first feature map data. The first feature map data is then subjected to depthwise separable convolution to obtain second feature map data. The feature channels of the second feature map data are evenly divided to obtain first evenly divided data and second evenly divided data. The second evenly divided data is then convolved and adjacent channels are separated to obtain first similar feature data and second similar feature data. The second similar feature data is then subjected to depthwise separable convolution and concatenated with the first similar feature data to obtain first feature data. The first feature data and the first evenly divided data are then concatenated to obtain second feature data. Third feature map data is acquired based on the second feature data. The number of target categories is obtained based on the third feature map data. The image category corresponding to the image data is obtained based on the number of target categories. The process of obtaining the first feature map data includes: The image data is convolved using a convolution kernel with a stride of 2 and a kernel size of 3*3. The number of feature channels in the convolved image data is then increased, and the convolved image data is downsampled to obtain the first feature map data. The process of obtaining the second feature map data includes: An activation function is constructed to perform batch normalization on the first feature map data. The normalized first feature map data is then activated using the activation function. The activated first feature map data is then subjected to depthwise separable convolution using a convolution kernel with a stride of 2 and a kernel size of 3*3. The convolutional image data is then downsampled to obtain the second feature map data. The process of obtaining the first similarity feature data and the second similarity feature data includes: The second feature map data is batch normalized. The normalized second feature map data is then convolved with a 1*1 kernel. The convolved second feature map data is then activated with an activation function. The feature channels of the activated second feature map data are then evenly divided to obtain first and second evenly divided data. The second evenly divided data is then convolved and adjacent channel separation is performed to obtain first and second similar feature data. The process of obtaining the second feature data includes: The second similarity feature data is batch normalized. The normalized second similarity feature data is convolved with a convolution kernel of size 1*1. The convolutional second similarity feature data is activated with an activation function. The activated second similarity feature data is concatenated with the first similarity feature data to obtain the first feature data. The first feature data is concatenated with the first average data to obtain the second feature data. The process of obtaining the third feature map data includes: The second feature data is shuffled through channels, the features of the shuffled second feature data are recombined, and the recombined second feature data is downsampled to obtain the third feature map data.

2. The method according to claim 1, characterized in that, The process of obtaining the number of target categories includes: A fully connected layer is constructed to perform global average pooling on the third feature map data. The number of features is then transformed on the average pooled third feature map data through the fully connected layer to obtain the number of target categories.

3. The method according to claim 1, characterized in that, The process of obtaining the image category corresponding to the image data includes: The probability distribution of the target category is obtained by processing the target category number data using the softmax function. The image category corresponding to the image data is obtained by extracting the category corresponding to the maximum value of the probability distribution.

Images