An image classification method based on a high-precision and lightweight feature extraction network model

Abstract

The application provides an image classification method based on a high-precision and light-weight feature extraction network model. The method comprises image preprocessing and image feature extraction. For an image to be classified, the image is input into a front-end feature pre-extraction module PrM for pre-extraction of features, the extracted features are gradually input into a first-stage feature extraction module, a second-stage feature extraction module and a third-stage feature extraction module, and the features output by the third-stage feature extraction module are further input into a post-processing module to obtain finally extracted features. Meanwhile, after the network training is completed, the network pruning operation is performed on the third-stage feature extraction module and the post-processing module. The method can better serve the upper-layer tasks such as target classification and target detection, and provides another option for mobile end embedding.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of digital image processing, and particularly relates to an image classification method based on a high-precision and lightweight feature extraction network model. BACKGROUND

[0002] Machine vision technology, as a branch of artificial intelligence, mainly converts the information of a target into image information through a camera and transmits the image information to a corresponding image processing system. The image processing system realizes various middle-layer tasks, such as image segmentation, target detection and target tracking, by performing feature extraction on the input image, and then serves the upper-layer tasks, such as the unmanned driving technology. Therefore, the bottom-layer image feature extraction task is particularly important for the subsequent middle and upper-layer tasks.

[0003] With the gradual improvement of the hardware level in recent years, the feature extraction model based on the neural network has developed rapidly. The powerful feature extraction ability brought by the object independence and model independence is favored by more and more researchers. At present, according to the time main line, the feature extractors based on the neural network mainly include AlexNet, VGG, GoogLeNet, ResNet, DenseNet, ResNeXt and SE-Net. The development characteristics of the network are to deepen or widen the network structure to gradually improve the feature extraction ability, and at the same time, to solve the problem of unfriendly training caused by the deepening of the network layer, such as gradient disappearance and gradient explosion. However, blindly deepening or widening the network ignores the actual application scenarios, such as the requirement of the network model for the hardware storage capacity and the running efficiency of the network in the actual application end. Among them, the model size of the above-mentioned ResNet and DenseNet after training can even reach the level of several hundred MB or even GB, which is extremely unfriendly to the deployment of the mobile end. SUMMARY

[0004] The application aims to solve the problem that the existing feature extractor network model based on the neural network is too large and is not friendly to training and deployment, thereby providing a new choice for the deployment of the end-side platform, and proposes an image classification method based on a high-precision and lightweight feature extraction network model.

[0005] The application is implemented by the following technical solutions. The application proposes an image classification method based on a high-precision and lightweight feature extraction network model, which specifically includes the following steps.

[0006] S1: Preprocessing all input pictures;

[0007] S2: Inputting the preprocessed pictures, i.e., feature maps, into a front-end feature pre-extraction module PrM to pre-extract features;

[0008] S3: input the pre-extracted features obtained in the manner of S2 into the first-stage feature extraction module MS1 to obtain features q1 extracted by MS1;

[0009] S4: input the features q1 obtained in the manner of S3 into the second-stage feature extraction module MS2 to obtain features q2 extracted by MS2;

[0010] S5: input the features q2 obtained in the manner of S4 into the third-stage feature extraction module MS3 to obtain features q3 extracted by MS3;

[0011] S6: input the features q3 obtained in the manner of S5 into the post-processing module PoM to obtain the final extracted features;

[0012] S7: perform a final transformation of the final extracted features by a combined operation composed of average pooling and full connection, and convert the final extracted features into class information recognizable by the output end, thereby completing the classification task for the input picture.

[0013] Further, network pruning operations are performed on the MS3 module in S5 and the PoM module in S6.

[0014] Further, the preprocessing of all input pictures specifically includes size resetting, center cropping, horizontal flipping and normalization processing.

[0015] Further, in S2, after the feature map is input into the PrM module, the PrM module performs a convolution operation and a maximum pooling operation on the feature map respectively to extract features with preliminary global information.

[0016] Further, the first-stage feature extraction module MS1, the second-stage feature extraction module MS2 and the third-stage feature extraction module MS3 are all composed of one γ-WSB module and a plurality of ASB or SB modules in structure, wherein MS1 contains three ASB-S modules, MS2 contains two ASB-L and five ASB-S modules, and MS3 contains three SB modules; in MS2, the first and the sixth are ASB-L, and the rest are ASB-S.

[0017] Further, the γ-WSB module is composed of a feature extraction part, a γ-weight shuffling module γ-WSM and a shuffling module OSM. First, the input feature map is input into the feature extraction part of the γ-WSB module for feature extraction, then the extracted features are input into the γ-WSM and the OSM at the back end of the γ-WSB for feature map re-shuffling operation, and then the shuffled feature map is fused to prepare for the following ASB-S;

[0018] The ASB-S is composed of a series of convolution, regularization and activation operations, and completes the re-feature extraction of the extracted features of the front-end gamma-WSB.

[0019] Further, the PoM in S6 is composed of a combination operation of convolution-regularization-activation (C-B-R) once, and adjusts the input features in the channel.

[0020] Further, the network pruning operation is specifically:

[0021] Step 1: preliminary pruning preparation;

[0022] Step 2: channel adjustment;

[0023] Step 3: pruning at the block coupling position;

[0024] Step 4: pruning;

[0025] Step 5: fine-tuning.

[0026] The present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the steps of the image classification method based on the high-precision and lightweight feature extraction network model when executing the computer program.

[0027] The present application provides a computer readable storage medium for storing computer instructions, which are executed by a processor to implement the steps of the image classification method based on the high-precision and lightweight feature extraction network model.

[0028] The present application has the following advantages:

[0029] The present application introduces gamma-Weighted Shuffle Block and Attentioned Shuffle Block on the basis of ShuffleNetv2, and simultaneously performs network pruning operation on the third stage feature extraction module and post-processing module, thereby forming a high-precision and lightweight feature extraction network ShuffleNetv2-Pro, which can better serve the upper-layer tasks such as target classification and image segmentation, and provides a new choice for mobile embedded. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 It is the overall structure diagram of the high-precision and lightweight feature extraction network ShuffleNetv2-Pro of the present application;

[0031] Figure 2 It is a bilinear interpolation diagram;

[0032] Figure 3 is a structure diagram of a gamma-weighted shuffle module;

[0033] Figure 4 is a structure diagram of an original shuffle module;

[0034] Figure 5 is a structure diagram of a depthwise separable convolution;

[0035] Figure 6 is a structure diagram of a dual-group convolution (small) with an attention head;

[0036] Figure 7 is a structure diagram of a dual-group convolution (large) with an attention head;

[0037] Figure 8 is an inter-group schematic diagram of an ASB module. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0039] Embodiment one:

[0040] Referring to Figure 1 , a high-precision and lightweight feature extraction network is described using a target classification task, including the following steps:

[0041] The application is based on ShuffleNetv2, for the image to be classified, first input it into the front-end feature pre-extraction module Pre-processing Module (PrM) for pre-extraction of features, then input the extracted features into the first stage feature extraction module Module Stage 1 (MS1), further input the features output by the first stage feature extraction module into the second stage feature extraction module Module Stage 2 (MS2), further input the features output by the second stage feature extraction module into the third stage feature extraction module, further input the features output by the third stage feature extraction module Module Stage 3 (MS3) into the post-processing module Post-processing Module (PoM), obtain the finally extracted features, perform the final transformation of the features by once combination operation of average pooling and full connection (Avg-FC), convert it into class information recognizable by the output end, so as to complete the classification task of the input picture. At the same time, after the network training is completed, the network pruning (Slimming) operation is performed on the third stage feature extraction module and the post-processing module, and the layers with small weights in the sparse model are pruned, further compressing the model volume.

[0042] Before training the network, all input images need to be preprocessed.

[0043] The preprocessing operation includes size resetting, center cropping, horizontal flipping and normalization processing.

[0044] The size resetting is completed by means of bilinear interpolation method, and the specific principle is as follows:

[0045] In combination with Figure 2 , the solving process of bilinear interpolation method is that the feature value of any point P (x3, y3) in the rectangular region composed of four points R 11 , R 12 , R 21 , R 22 is solved. First, the positions of R4 and R5 in Figure 2 are located according to the position of P point, and the feature values of R4 and R5 are solved according to the following formula:

[0046]

[0047] The feature value of the final P point is calculated according to the feature values of R4 and R5 according to the following formula:

[0048]

[0049] Center crop is to diffuse the picture after resetting the size according to the fixed size required by the network to the center of the picture and then crop the required picture size.

[0050] Horizontal flip is to flip the input image in the horizontal direction according to the following formula:

[0051] I'(m,n) = I(W-m,n) (3)

[0052] In the formula, I represents the input image, I' represents the output image, W represents the width of the input image I, and (m, n) represents the coordinate value of any point in the image;

[0053] Normalization processing is to normalize the pixel value of the input image to the [0, 1] interval in the following way:

[0054] I''(m,n) = I'(m,n)-min(I') (4)

[0055]

[0056] In the formula, I' represents the output image after the horizontal flip, I'' represents the output image after the normalization, and min(·) and max(·) respectively represent the minimum and maximum pixel value operations in the image matrix.

[0057] The picture after the preprocessing operation is input into the front-end feature pre-extraction module PrM for feature pre-extraction. The PrM module is mainly composed of convolution and maximum pooling operations. The module performs one convolution operation and one maximum pooling operation on the feature map respectively, extracts the features with preliminary global information, and the specific structure parameters are shown in Table 1.

[0058] Table 1

[0059]

[0060] As can be seen from Table 1, after the input image passes through the PrM module, the output channel number is expanded from 3 to 24, and the size of the output feature map is reduced to 1 / 4 of the size of the input image.

[0061] The output features of the PrM module are input into a first stage feature extraction module Module Stage 1 (MS1). The first stage feature extraction module Module Stage 1 (MS1), the second stage feature extraction module Module Stage 2 (MS2) and the third stage feature extraction module Module Stage 3 (MS3) are all composed of a gamma-Weighted Shuffle Block (gamma-WSB) module and a plurality of Attentioned Shuffle Block (ASB) or SB modules. The MS1 contains three ASB-S modules, the MS2 contains two ASB-L and five ASB-S modules (of which the first and the sixth are ASB-L and the rest are ASB-S), and the MS3 contains three SB modules, so that the model extracts features while compressing the size of the feature map. The specific structure data is shown in Table 2.

[0062] Table 2

[0063]

[0064] As can be seen from Table 2, after the feature map passes through the MS3, its size is further reduced (1 / 2) 3 = 1 / 8. Since the MS1, MS2 and MS3 are similar in structure, the present application only describes the MS1 process, and the differences between the MS2 and MS3 and the MS1 are described.

[0065] The specific structure of the gamma-WSB is shown in Figure 1 As can be seen from the figure, the feature map output by the PrM is input into the gamma-WSB of the MS1 in different branches, i.e. branch ① and branch ②. The branch ① performs a GC-B-C-B-R combination operation (GC, B, C and R respectively represent Group Convolution, Batch Normalization, Convolution and ReLU) on the input feature map, and the branch ② performs a C-B-R-GC-B-C-B-R combination operation on the input feature map. The branch ① and the branch ② further extract features from the input feature map. Since the combination operation is a Group Convolution operation, in order to enable the exchange of feature information between the channels of the feature map, the features output by the branch ① and the branch ② are respectively input into the subsequent gamma-Weighted Shuffle Module / gamma-WSM module and the Original Shuffle Module / OSM module for channel shuffling operation of the feature map, so that the subsequent feature extraction process is not limited to the local features within the group.

[0066] The specific structure of the gamma-WSM module is described in Figure 3 , and the specific steps are as follows:

[0067] For the input feature map of the gamma-WSM, the branch ① and the branch ② are subjected to the corresponding batch normalization processing, that is:

[0068]

[0069] wherein, is the input feature map, and y is the output feature map. Therefore, after the batch normalization processing, each feature map channel will learn a corresponding hyperparameter pair (γ, β), that is, a scaling factor γ and a translation factor β, wherein γ represents the feature expression ability of the corresponding feature channel. The greater γ is, the more important the corresponding feature map is. Therefore, for the gamma-WSM module, not only the feature map is input, but also the corresponding γ vector is input into the gamma-WSM module.

[0070] Referring to Figure 3 , for the input feature maps G1 and G2, the channels are spliced to obtain the spliced feature map G3, and the input corresponding scaling factor vectors γ1 and γ2 are also spliced by channel to obtain the spliced scaling factor vector γ3. γ3 is sorted in descending order to obtain the sorted vector γ4, then γ4 is truncated at the center (center truncation) to obtain vectors γ5 (begin) and γ6 (end), γ6 is reversed to obtain γ7, and finally the scaling factor values of γ5 and γ7 vectors are taken respectively to form the final scaling factor vector γ8, so as to realize mutual splicing. The feature map G 41 is reordered by adjusting the channels of the feature map G3 according to the scaling factor vector γ8. Specifically, assuming that γ1 = [0.1, 0.2, 0.4], γ2 = [0.5, 0.3, 0.6], γ3 = [0.1, 0.2, 0.4, 0.5, 0.3, 0.6] at this time, γ4 = [0.6, 0.5, 0.4, 0.3, 0.2, 0.1], γ5 = [0.6, 0.5, 0.4], γ6 = [0.3, 0.2, 0.1], γ7 = [0.1, 0.2, 0.3], and γ8 = [0.6, 0.1, 0.5, 0.2, 0.4, 0.3]. Therefore, the feature map is reordered in this way, and in the subsequent grouped convolution operation, the feature maps in each group have similar importance, thereby reducing the difference between the groups and avoiding uneven distribution of features in some groups. At the same time, in order to make the subsequent network extract more compact features, the “important” feature map is reserved twice, as described in Figure 3This involves a secondary preservation of highly important factors and their corresponding feature maps. Specifically, γ5 is self-stitched to obtain a stitched scaling factor vector γ9. Feature map G3 is then subjected to channel adjustment based on this scaling factor vector γ9 to obtain a disordered feature map G. 42 The feature volume G adjusted according to two different scaling factor vector channels 41 G 42 The features are fused to obtain the fused feature body G5. In this invention, feature fusion is performed element-wise by addition. Therefore, the feature value at each feature position in G5 includes features of high importance factors and features of equally important factors. To ensure that G5 contains better features more suitable for subsequent feature extraction, this invention further refines the feature fusion process. The features at each location were mapped N times (N=10 in the specific implementation of this invention). See also Figure 3 G5 and mapping weights Calculate the Hadamard product and rearrange the features to obtain the mapped feature volume. For each 2-D feature map in G6, the optimal mapping feature is obtained by calculating the row-wise optimal mapping feature matrix. Finally, the features of G7 are rearranged again to obtain the final output feature body of the γ-WSM module.

[0071] For the specific structure of the OSM module, please refer to [link / reference]. Figure 4 The specific steps are as follows:

[0072] The input feature map f mainly underwent grouping-recombination-concatenation operations. The grouping involved dividing the input feature map into equal subgroups. Figure 4 The data is grouped into four groups according to index order. Then, the feature maps of each group are reorganized. This reorganization involves extracting feature maps from each group sequentially according to their index, thus forming... Figure 4 The three groups of recombined features shown are then concatenated in sequence to form the final disordered feature map f. s .

[0073] The outputs of the γ-WSM module and the OSM module are fused. This invention employs the following feature fusion method:

[0074] f out =η·G8+(1-η)·f s (7)

[0075] At this time, the output feature f out This is a linear combination of the two input features, which maps the features to a new space that the original features could not cover, thus improving the network's generalization ability.

[0076] The fused feature f out is input into a subsequent feature extraction network.

[0077] Referring to Figure 1 , the part (GC-B-C) in the dashed box in the MS1 gamma-WSB, i.e., in branch ① and branch ②, is also called a depthwise separable convolution (DSC), and the specific structure thereof is shown in Figure 5 . For the input feature map First, group convolution is performed on it, Figure 5 as shown in the figure, which is divided into 3 groups, i.e., group = 3, and 3 different convolution kernels are used to perform convolution on each input feature map respectively, to obtain the convolution feature map Since such group convolution cannot exchange feature information between channels, the feature maps of g2 are then spliced and a 1x1 convolution kernel is used for regular dense convolution, so that the entire Figure 5 The parameter amount of the DSC process shown in the figure is 3x3x3+3x3x1x1 = 36, compared with the use of dense convolution throughout the process, the parameter amount is reduced from 3x3x3x3 = 81 to 36, so that the network model can be compressed.

[0078] The output feature f out of the gamma-WSB is input into three ASB-S modules. The specific structure of the ASB-S module is shown in Figure 1 , and the main steps are as follows: f out is divided into a front half (half front / h f ) and a back half (half back / h b ). For the h f alf back, in order to further reduce the volume of the model, a combination operation of C-B-R-(A-DGCs)-B-R is performed, where A-DGC is an attentioned dual group convolution (small) (A-DGCs) proposed in the present application, and the specific structure of A-DGCs is shown in Figure 6 , and the specific steps are as follows:

[0079] For the input feature map They are input into two different flow layers respectively to perform different task operations. The solid arrows correspond to the feature flow layer, which is mainly used to generate feature maps of different levels; and the dashed arrows correspond to the attention flow layer, which is used to generate an attention map that can fuse channel information. Specifically, first, g'1 is input into the feature flow layer (solid line) to perform a first re-grouping convolution operation, to obtain the convolutional feature map Then, g'2 is flowed into the attention flow layer (dashed line). First, all the feature maps of g'2 are spliced, and then the spliced feature map information is aggregated. For convenience, the embodiment performs feature aggregation in the channel-wise mean (C-Mean) manner, and thus obtains a preliminary attention map The p s1 is subjected to a 1x1 convolution operation to increase the nonlinearity of the network and make the network fit the optimal attention map, and thus the attention map p is obtained Then, p s2 is fused with all the feature maps in g'2 obtained from the feature flow layer, and the fusion manner is as follows:

[0080] g"2(k) = g'2(k) O p s2 (8)

[0081] where k represents the index of the feature map in g'2, and O is the Hadamard product between matrices. Then, the fused feature map is subjected to a second re-grouping convolution (using a 1x1 convolution kernel) in the manner shown above, to obtain the final output feature g'3. Due to the presence of the attention map, the channel feature information is fused into the input feature map of the second re-grouping convolution, so that the dense convolution in the back end is omitted, and thus the model volume is further compressed. For Figure 6 , the parameter amount is 3x3x3+1x1+1x1x3=31, which is less than that of DSC. Assuming that the input channel number and the output channel number are both 100, the parameter amount of DSC will be 3x3x100+1x1x100x100=10900, and the parameter amount of A-DGCs is 3x3x100+1x1+1x1x100=1001, which is reduced by 90.8% compared with DSC. Finally, the feature map subjected to the second re-grouping convolution is spliced with the feature map of the branch h b in the channel order, and is subjected to the out-of-sequence operation of the OSM module, that is, the C-OS operation in Figure 1 .

[0082] The output of MS1 is input into MS2, and the output of MS2 is input into MS3 for further extraction of features, gradually converting low-level features to high-level features, and the specific extraction steps are consistent with MS1. The present application only describes the differences between MS1 and MS2:

[0083] Wherein, MS2 is different from MS1 in that, referring to Figure 1 and Table 2, two ASB-Ls are used in MS2, and the difference between ASB-S in MS1 is that A-DGCs are no longer used, but A-DGCl is used, and the specific structure is referred to Figure 7 , and the specific steps are: for the input feature map First, a first re-grouping convolution is performed to obtain the convolution feature map Then, all feature maps in the attention flow layer in g′2 are divided into three groups and convolved using 1x1 and different hole rates of convolution kernel to improve the network's perception ability of the image region of interest, thereby obtaining three groups of convolution feature maps. Then, the three groups of feature maps are spliced, and the spliced feature maps are again convolved using a 1x1 convolution for convolution compression operation, compressing the channel number to 1 / 3 of the convolution before. The compressed feature maps are also aggregated between channels in the form of channel-wise mean (CMean) to obtain a preliminary attention map p l1 The subsequent operations are consistent with A-DGCs and will not be described again.

[0084] The difference between MS3 and MS1 is that three SB modules are used in MS3 as shown in Table 2, referring to Figure 1 Compared with ASB-S / L module, SB module only removes A-DGCs or A-DGCl, i.e. SB is a module without attention head, and the rest of the process is consistent with ASB-S / L module.

[0085] Finally, the output of MS3 is input into the post-processing module PoM in the backend, wherein PoM is composed of C-B-R combination operations. In order to complete the task requirement of image classification, an average pooling AvgPooling operation and a fully connected Fully Connected operation are used in PoM, so that the final output is understandable category information.

[0086] In the third stage feature extraction module and the post-processing module, a network slimming operation is performed. The main principle of the slimming operation is that since the hyperparameter γ in the BN layer represents the expression ability of the channel, the higher the γ value, the stronger the feature expression ability of the channel, and therefore the higher the importance of the layer. Therefore, the network channel is pruned by the size of the γ value. The slimming operation mainly consists of the following 5 steps:

[0087] Step 1: preliminary pruning preparation.

[0088] Given the pruning rate λ (the pruning rate used in this embodiment is 0.4), count the γ values of all BN layers and sort them in ascending order, locate the position of the 40th γ value and record its corresponding value γ T , which is used as the threshold for pruning. Recount all layers in the network, and the layers with γ values greater than the threshold γ T are retained, and their positions are written into the position index list V, waiting for subsequent pruning;

[0089] Step 2: channel adjustment.

[0090] Since grouped convolution is used in the ASB module, and the number of input and output channels of the grouped convolution is consistent in this embodiment, the number of channels of the BN layers before and after the grouped convolution of the pruned model should also be consistent. Therefore, the aforementioned position index list V is further adjusted, specifically:

[0091] Locate BN1 and BN2 before and after the grouped convolution, compare the number of remaining channels in BN1 and BN2 after the preliminary pruning by the aforementioned position index list V, and fill in the one with fewer remaining channels. Assuming that the number of remaining channels in BN1 is v1, and the number of remaining channels in BN2 is v2, and v1 < v2, at this time, re-locate all scaling factor γ values in the current BN1 layer, arrange them in descending order, and take out the index positions corresponding to the first v2 values to replace the corresponding positions in the position index list V. The channel adjustment of the remaining BN layers before and after the grouped convolution is carried out in the same way.

[0092] Step 3: pruning at block coupling.

[0093] According to the values in the position index list V, the corresponding layers from the unpruned network are taken to form a new pruned network. Among them, for the pruning between ASB coupling modules, as follows:

[0094] Referring to Figure 8 , assuming that the number of channels after pruning at BN2 of ASB-S1 is c1, and the number of channels at h b of ASB-S1 is c2, therefore the number of channels after pruning at h′ f of ASB-S2 is c3 = (c1 + c2) / 2, during the pruning implementation, directly take the first c3 channels of the unpruned h f of ASB-S2 to construct the new pruned network, and the pruning between the remaining ASB-S / L coupling modules is carried out in the same way.

[0095] Step 4: pruning.

[0096] Pruning the model according to the position index list V.

[0097] Step 5: Finetune.

[0098] The pruned network is retrained until the loss value no longer decreases, and the training is stopped.

[0099] The overall network structure is shown in Table 3.

[0100] Table 3

[0101]

[0102]

[0103] Example Two:

[0104] This embodiment is specifically performed according to the following steps:

[0105] 1. Data preparation stage

[0106] All data used for training, verification and testing in this embodiment 1 are derived from Cifar100 (this dataset is well known to those skilled in the art, and the embodiment of the present application does not repeat it). The Cifar100 dataset contains 20 large categories and 100 small categories, covering various aspects of life such as animals, plants, furniture, appliances and vehicles. Each small category contains 600 images, of which 500 are training images and 100 are test images, and the resolution of the images is 32x32, which is expanded to 160x160 in this embodiment. The division details of the Cifar100 dataset in the implementation process of this embodiment are shown in Table 4.

[0107] Table 4

[0108]

[0109] 2. Training stage

[0110] The training process of the training network repeats steps b1 and b2.

[0111] b1: initialize network training parameters and hyperparameters. The embodiment sets the maximum number of iterations Iteration = 550; the batch training size is set to BatchSize = 128; in order to have a faster convergence speed in training, the learning rate lr = 0.01 is set; the Stochastic Gradient Descent (Stochastic Gradient Descent) optimization algorithm is introduced as the optimizer, the weight decay coefficient L2 = 0.0005, and the momentum = 0.9; all network parameters are initialized in the Xavier manner; the loss function selected is the Cross Entropy Loss. Among them, for the algorithm of the application, due to the existence of pruning operation, after pre-training, the network is pruned according to the manner described in embodiment I, and the pruned network is retrained for 550 iterations under the above parameters.

[0112] b2: the image obtained in step S1 is used as the input of the training network, and the training of the network is started. The training is an iterative process as a whole, and each iteration first calculates the loss value of the forward propagation, and then updates all the parameters of the network in the reverse direction with the goal of minimizing the loss value.

[0113] 3. Test phase

[0114] The process of the test phase is repeated from step I to step II.

[0115] Step I: repeat step S1 for the test picture;

[0116] Step II: input the test image after step I into the trained network after step b1 and b2 to output the classification result of the test picture.

[0117] The experimental results are shown in Table 5, where A and B represent the addition of ASB and γ-WSB modules proposed in the application, respectively. ShuffleNetv2+A indicates that the ASB module is added to the baseline algorithm ShuffleNetv2 according to the form of reference Figure 1 , and the rest of the models are similarly added. In group 1, “_1.0x” represents the corresponding algorithm model volume code, i.e., the model 1.0 version of the corresponding algorithm, and the rest are similarly added. At the same time, Table 5 also shows the comparison results with the remaining mainstream lightweight networks with excellent performance. In order to ensure fairness, all algorithms are divided into different groups for comparison according to the model volume. As can be seen from Table 5:

[0118] 1. In group 1, the algorithm ShuffleNetV2Pro_1.0x proposed in the application achieves the best in terms of accuracy (Acc) and model size (K-Params). The comparative algorithm MobileNetV3-Small_1.0x achieves the best in terms of complexity (M-FLOPs);

[0119] 2. In group 1, the algorithms ShuffleNetV2_1.0x+A, ShuffleNetV2_1.0x+B and ShuffleNetV2Pro_1.0x proposed in the application all improve the accuracy (Acc) compared with the original algorithm ShuffleNetV2_1.0x. At the same time, the accuracy of ShuffleNetV2Pro_1.0x reaches the best after adding the ASB and γ-WSB modules proposed in the application. Quantitatively, the accuracy of ShuffleNetV2Pro_1.0x is improved by 1.64% compared with ShuffleNetV2_1.0x, and the model size of ShuffleNetV2Pro_1.0x is reduced by 7.5% compared with the original algorithm ShuffleNetV2_1.0x;

[0120] 3. In group 2, when the ShuffleNetV2Pro_1.0x is further pruned according to the pruning strategy proposed in the application (pruned by ~5% and ~20% respectively), the model size is further reduced, and the accuracy is still better than the baseline algorithm ShuffleNetV2_1.0x. Quantitatively, when the network layer of the ShuffleNetV2Pro_1.0x proposed in the application is pruned by ~5%, the accuracy is improved by 0.89% compared with ShuffleNetV2_1.0x, and the network size is further reduced by 11.5% compared with ShuffleNetV2_1.0x; when pruned by ~20%, the accuracy is improved by 0.3% compared with ShuffleNetV2_1.0x, and the network size is further reduced by 27.9% compared with ShuffleNetV2_1.0x, so the ShuffleNetV2Pro proposed in the application can make it possible to reduce the model size without losing accuracy;

[0121] 4. In group 3, when compared with the baseline algorithm model 1.5 version, the experimental results show that the ShuffleNetV2Pro_1.5x proposed in the application significantly improves the accuracy compared with the original algorithm ShuffleNetV2_1.5x, and further reduces the model size compared with ShuffleNetV2_1.5x. Quantitatively, the accuracy is improved by 3.48%, and the model size is reduced by 9.2%.

[0122] Therefore, the ShuffleNetv2Pro has greater advantages in both accuracy and model volume, indicating that the ShuffleNetv2Pro has stronger feature extraction capability, and in the process of improving the model accuracy, the model is more lightweight, and the ShuffleNetV2Pro_1.0x proposed in the application still reaches ~104FPS on the CPU (Intel Core i5, 2.90GHz) end, providing a new choice for mobile end embedding.

[0123] Table 5

[0124]

[0125] The application provides an electronic device, including a memory and a processor, the memory stores a computer program, and the processor implements the steps of the image classification method based on the high-precision and lightweight feature extraction network model when executing the computer program.

[0126] The application provides a computer readable storage medium for storing computer instructions, and the computer instructions implement the steps of the image classification method based on the high-precision and lightweight feature extraction network model when executed by a processor.

[0127] The memory in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a read only memory (ROM), a programmable ROM (PROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM). It is to be noted that the memory described with the methods of the present application is intended to include, but not be limited to, these and any other suitable types of memory.

[0128] In the above embodiments, all or part of the methods can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the methods can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes one or more available media sets. The available media can be magnetic media (such as floppy disk, hard disk, magnetic tape), optical media (such as high-density digital video disc (DVD)), or semiconductor media (such as solid state disc (SSD)), etc.

[0129] In the implementation process, each step of the above method can be completed by integrated logic circuit of hardware in the processor or instruction in the form of software. The steps of the method disclosed in the embodiments of the present application can be directly embodied as hardware processor execution completion, or executed by combination of hardware and software modules in the processor. The software module can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0130] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with a signal processing capability. In the implementation process, each step of the method embodiments can be completed by the integrated logic circuit or the instruction in the form of software of the hardware in the processor. The processor mentioned above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method.

[0131] The above describes in detail the image classification method based on the high-precision and light-weight feature extraction network model. The principles and implementation manners of the present application are described by using specific examples. The above embodiment is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. An image classification method based on a high-precision and lightweight feature extraction network model, characterized in that, The method specifically comprises: S1: preprocessing all input pictures; S2: inputting the preprocessed picture, i.e., a feature map, into a front-end feature pre-extraction module PrM to perform pre-extraction of features; S3: input the pre-extracted features obtained in the manner of S2 into the first-stage feature extraction module MS1 to obtain features extracted by MS1 q 1; S4: features obtained in the manner according to S3 are input into a 2nd stage feature extraction module MS2, obtaining features extracted by MS2 q 1 input into a 2nd stage feature extraction module MS2, obtaining features extracted by MS2 q 2; S5: features obtained in the manner according to S4 are input into a 3rd stage feature extraction module MS3, obtaining features extracted by MS3 q 2 into a 3rd stage feature extraction module MS3, obtaining features extracted by MS3 q 3; S6: Features obtained in the same way as in S5 q 3. The input is fed into the post-processing module PoM to obtain the final extracted features; S7: performing final conversion of the finally extracted features through a combination operation composed of average pooling and full connection once to convert the features into class information recognizable by an output end, thereby completing a classification task for the input picture; In S2, after the feature map is input into the PrM module, the PrM module performs a convolution operation and a maximum pooling operation on the feature map once to extract features with preliminary global information; The first stage feature extraction module MS1, the second stage feature extraction module MS2 and the third stage feature extraction module MS3 are all composed of one module and several ASB or SB modules, wherein MS1 contains three ASB-S modules, MS2 contains two ASB-L and five ASB-S modules and MS3 contains three SB modules; in MS2, the first and the sixth are ASB-L, and the rest are ASB-S; The The module is composed of a feature extraction part, a weight shuffling module , a WSM and a shuffling module OSM. First, the input feature map is input to the feature extraction part of the WSB module for feature extraction, and then the extracted features are input to the WSM and OSM of the WSB back-end for feature map re-shuffling operation, and then the shuffled feature map is fused, thereby preparing for the following ASB-S. The ASB, ASB-S, ASB-L, SB are all composed of a series of convolution, regularization and activation operations, complete the front-end -WSB has extracted features again feature extraction; The PoM in S6 is composed of a combination operation of convolution-regularization-activation C-B-R to adjust the input features in the channel; The The WSM module specifically performs the steps as follows: For input - Feature map of WSM, both branch ① and branch ② are processed by corresponding Batch Normalization, namely: （6） wherein, is the input feature map, y is the output feature map; after batch normalization processing, each feature map channel will learn a corresponding hyperparameter pair ( , ), i.e., a scaling factor and a translation factor , wherein characterizes the feature expression capability of the corresponding feature channel; For the input feature map G 1 and G 2. Perform splicing by channel to obtain the spliced ​​feature map. G 3. Simultaneously, input the corresponding scaling factor vector. The vectors are also concatenated by channel to obtain the concatenated scaling factor vector. ,right Sort in descending order to obtain the sorted vector. Next Cut off at the center to obtain the vector. and ,right Reverse the order to get Finally, element by element and The vector is then processed by taking the scaling factor value, thus forming the final scaling factor vector. To achieve inter-swapping; according to the scaling factor vector For feature maps G 3. Re-adjust the channels to obtain the disordered feature map. G 41 The "important" feature maps were preserved twice, that is, the high-importance factors and their corresponding feature maps were preserved twice; specifically, for Self-concatenation was performed to obtain the concatenated scaling factor vector. According to the scaling factor vector For feature maps G 3. Perform channel adjustment again to obtain the disordered feature map. G 42 The feature volume adjusted according to two different scaling factor vector channels G 41 , G 42 The fusion process is performed to obtain the fused feature body. G 5; Feature fusion is performed here by adding elements one by one; The features at each location are mapped N times, that is... G 5. Mapping weights Calculate the Hadamard product and rearrange the features to obtain the mapped feature volume. ,right G For each 2-D feature map in step 6, the optimal mapping feature is obtained by calculating the optimal mapping feature matrix row by row. Ultimately G 7. Perform feature rearrangement again to obtain -The final output feature body of the WSM module .

2. The image classification method of claim 1, wherein, The network pruning operation is performed on the MS3 module in S5 and the PoM module in S6.

3. The image classification method of claim 1, wherein, The preprocessing of all input pictures specifically comprises size resetting, center cropping, horizontal flipping, and normalization processing.

4. The image classification method of claim 2, wherein, The network pruning operation specifically comprises: Step 1: preliminary pruning preparation; Step 2: channel adjustment; Step 3: pruning at block coupling positions; Step 4: pruning; Step 5: fine-tuning. 5.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the method in any one of claims 1-4.

6. A computer readable storage medium for storing computer instructions, characterized in that, The computer instructions are executed by the processor to implement the steps of the method in any one of claims 1-4.

Images