An image classification method based on a pruning-based lightweight network structure

Abstract

The application discloses a kind of image classification methods of lightweight network structure based on pruning, the method makes the sparse value of each filter combined with BN layer scaling coefficient, to discriminate the importance of overall channel, i.e. First, the importance function of filter is obtained using the sparsity formula and the BN layer scaling coefficient, using the importance function as the basis for judgment, deleting redundant filter, then using L1 regularization to sparsely train the model, and improving the network model.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of image classification, and particularly relates to an image classification method based on a light-weight network structure of pruning. BACKGROUND

[0002] In recent years, with the increasing number of layers of deep learning networks, the network performance is continuously improved, but at the same time, the complexity of the network also increases exponentially, which limits the use of neural networks to a certain extent. Due to the rapid development of deep neural networks and intelligent mobile devices, the light-weight design of network structure has gradually become a front and popular research direction. The core goal of light-weight design is to optimize the storage space and improve the running speed on the premise of maintaining the accuracy of deep neural networks.

[0003] Image classification has always been a hot topic in the field of machine learning. The convolutional neural network (CNN) algorithm was proposed as early as last century, but it was not until the 21st century that neural networks began to gradually replace traditional machine learning methods with the emergence of deep learning algorithms and the significant improvement of computing device performance. Among them, AlexNet is a pioneering work that first surpassed human image classification accuracy on the ImageNet dataset. With the rapid growth of model size and data volume, neural network algorithms have achieved breakthrough results in image, speech and video fields, far exceeding traditional algorithms, and gradually penetrating into daily life, industrial production and scientific research. However, at the same time, it also brings higher storage and computing costs, which makes the deployment of the network more challenging.

[0004] In order to use neural network models on mobile devices, some special structures and light-weight models have appeared in recent years, which have the characteristics of small storage, low computation and excellent network performance. For example, Google's MobileNetV1 uses a depth separable convolution structure instead of traditional convolution operations, which greatly reduces the parameter amount and computation amount while still achieving comparable classification accuracy to VGG neural networks on the ImageNet dataset, thereby attracting widespread attention. In order to further improve the accuracy, MobileNet gradually evolved into MobileNetV3, but its computing resource requirements also increased, and the resource requirements for mobile devices became higher and higher. Compared with traditional deep neural networks, light-weight network design is more ingenious, the structure is more compact, and the calculation method is more complex, but the compressible part becomes less and less, which brings new challenges to the research of deep learning model compression. In addition, different light-weight models have different network structure characteristics, and how to accurately select redundant structures becomes a key problem for compressing light-weight network models.

[0005] Therefore, in order to further reduce the hardware resource requirements of MobileNetV3 for mobile devices, so that it can be deployed to low-power and low-latency application scenarios, a new model compression pruning method is proposed based on the network structure characteristics of MobileNetV3.

[0006] Pruning is a commonly used technical method in the field of algorithm lightweight processing and network acceleration. By pruning the neural network, a network structure with a higher pruning rate can be obtained, thereby reducing the volume and computational consumption of the network, so that the network can run on devices with limited storage and computing capacity. The network with high pruning rate also requires less time in the forward propagation process, which provides help for some tasks that require real-time performance, such as real-time target detection. Practice has proved that network pruning is an effective technology that can compress and accelerate convolutional neural networks, so that deep networks can adapt to hardware devices with limited storage or computing resources. SUMMARY

[0007] 1. Technical problems to be solved:

[0008] In view of the above technical problems, the present application provides an image classification method based on a lightweight network structure of pruning, which adopts a lightweight model MobileNetV3 and a structured pruning to combine the two, and proposes a new pruning criterion to compress the network. The method combines the sparsity value of each filter with the BN layer scaling coefficient to determine the importance of the overall channel, i.e. the weight W.

[0009] 2. Technical solutions:

[0010] An image classification method based on a lightweight network structure of pruning, characterized in that it comprises the following steps:

[0011] Step 1: load a pre-trained MobileNetV3-based model for image classification;

[0012] Step 2: use the filter sparsity formula and the BN layer scaling coefficient to calculate the importance index of the filters contained in each convolutional layer; based on the importance index of the filters, prune the filters in each convolutional layer, and delete the filters in the convolutional layer that are lower than the preset importance index; and prune each convolutional layer in the model to obtain a pruned overall model;

[0013] Step 3: use L1 regularization to perform sparse training on the pruned overall model network to generate a lightweight model;

[0014] Step 4: update the weight of the lightweight model to obtain an image classification model of a lightweight network structure.

[0015] Further, step two specifically comprises:

[0016] S21: calculating the sparsity value of each filter of the deep convolutional neural network using a sparsity formula, wherein the sparsity formula is:

[0017]

[0018] (1)

[0019] In the above formula, denotes the sparsity value of the nth filter in the lth convolutional layer; is the weight of the convolution kernel; n, c, h, w are respectively the 4-dimensional tensor of the convolution kernel, and n, c, h, w are positive integers and [1, W], N is the number of output filters corresponding to the convolutional layer, C is the number of input filter layers corresponding to the convolutional layer, H is the height of the corresponding convolution kernel; W represents the width of the corresponding convolution kernel; denotes the standard deviation of the activation value of the filter at the corresponding position nchw in the lth layer;

[0020] wherein denotes the average value of the convolution kernel weight in the lth layer, which is calculated according to the following formula

[0021] (2)

[0022] S22: calculating the importance index of the filter based on the scaling coefficient γ of the BN layer and the importance determination function of the filter; the BN layer is placed in the next layer of the convolutional layer, and the output of the convolutional layer is normalized;

[0023] wherein the importance determination function of the filter is:

[0024] (3)

[0025] In the above formula, is the importance score of the ith filter, which can measure the importance or contribution of the filter in the network; is the sparsity value of the ith filter calculated based on formula (1); denotes the sum of the γ values of all filters, and each γ value is squared; γ is a parameter of the Batch Normalization (BN) layer, which represents the scaling coefficient of the corresponding filter.

[0026] In summary, the formula squares and sums the BN parameters γ of each filter, and adjusts the result through to obtain the importance The greater the importance value, the greater the influence of the corresponding filter on the model.

[0027] S23: According to the filter importance index Pruning the l-th layer of convolutional layer; the filters and the corresponding BN layer corresponding to the preset importance index are pruned, and the remaining filters E' and the BN layer are obtained, and the scaling coefficient of the BN layer at this time is ′;

[0028] S24: The importance set of the remaining m filters E' is obtained According to the preset pruning rate The threshold value of filter pruning is obtained :

[0029] (4)

[0030] Sort all the remaining filters E' in the network according to ascending order of importance, and obtain the pruning rate The importance index of the filter corresponding to the index position is taken as the threshold value of filter pruning; after obtaining the threshold value, all filters with importance lower than the threshold value in the convolutional layer are pruned from the convolutional layer;

[0031] S25: Continue steps S21-S24 until all convolutional layers are pruned.

[0032] Further, the step three reduces the parameter amount of the network by adding a sparsity constraint, so as to make the model lightweight; specifically including:

[0033] S31: Defining a loss function with an L1 regularization term to perform sparse training;

[0034] The loss function adopted is a cross-entropy loss function, and an L1 regularization term is added behind it; specifically:

[0035] (5)

[0036] In the formula: is the cross-entropy loss function; for the BN layer feature scaling coefficient γ, , The role of is to regularize the feature scaling coefficient in the BN layer ; β is a hyperparameter, used to control the weight size of the regularization term of the weight coefficient in the full connection layer or the convolutional layer, the greater the β, the more the parameters are punished, the greater the β, the greater the influence of the regularization term, and the closer the parameters in the BN layer to 0;

[0037] S32: sparse training is performed by using a proximal gradient optimization method APG; a regularization parameter λ is selected to control the influence degree of the regularization term on the total loss; specifically, the following steps are included:

[0038] S321: the objective function is defined as: (6)

[0039] In the above formula, x1 is a sparse coefficient vector, A is a basis vector matrix, b is an observation vector, and λ is a regularization parameter;

[0040] S322: the gradient is solved: the partial derivative of the objective function is obtained:

[0041]

[0042]

[0043] (7)

[0044] wherein, is the gradient of the data fitting term is the derivative of the L1 norm of , that is, the sign function;

[0045] S323: the subgradient of the sparse term is solved by using a subgradient method: that is,

[0046] The subgradient of the sparse term is

[0047] (8)

[0048] S324: the gradient of the data fitting term is solved: the gradient of the data fitting term is:

[0049]

[0050] S325: iterative optimization is performed; the minimum value of the objective function is solved by using a gradient descent method, and the parameter x 11 is updated by iteration to gradually approach the minimum value, wherein the update formula of each iteration is:

[0051] (9)

[0052] wherein, represents the parameter vector after the (k+1)th iteration, represents the parameter vector of the kth iteration; η is a learning rate, and a suitable value is selected by trial and error; the shrink function is a soft threshold function, and is used to realize sparsification by using the following formula:

[0053] ​​ (10)

[0054] In the above formula, y is a real number or vector to be processed, and a is a threshold value;

[0055] S326: updating the regularization parameter λ; specifically comprising:

[0056] S3261: calculating the L1 norm of the current sparse coefficient vector , that is, the sum of the absolute values of all elements in the sparse coefficient vector;

[0057] S3262: estimating a new value of λ according to the current L1 norm and a predefined parameter c, and the formula is as follows:

[0058] (11)

[0059] wherein, is the L1 norm of the current sparse coefficient vector;

[0060] S3263: repeating the above S3262 until the change of λ is less than a preset threshold value or a preset maximum number of iterations is reached.

[0061] Further, the weight of the lightweight model in step four is updated, and the initial weight is gradually optimized and pruned through iterative training and pruning; specifically comprising the following steps:

[0062] S41: initializing a network using He , and initializing a pruning mask ; He initialization is an initialization method for ReLU activation function, which is suitable for deep neural networks and corresponds to the network structure of MobileNet V3;

[0063] S42: training Wo using hyperparameter β for a preset number of times to obtain updated filter weight W1;

[0064] S43: calculating the importance index m i of the weight W1 according to the importance determination function of formula (3), and sorting the filters in ascending order;

[0065] S44: based on the preset threshold value a, retaining all weights whose m i is greater than or equal to a, and pruning the filters corresponding to the remaining weights; sorting the retained filters according to the weight values to obtain a list of retained filter weight values; selecting a percentage p, and finding the corresponding weight value in the sorted list according to the percentage, which is used as a new threshold value a';

[0066] S45: all weights satisfying the weight value greater than or equal to the new threshold a' are retained, and the rest of the weights are pruned, that is, the pruning mask theta corresponding to the pruning not required is set to 1, and the pruning required is set to 0;

[0067] S46: the loss function is recalculated according to formula (5), if the loss function of the pruned network is greater than the loss function calculated before step S43, it means that the deleted weight is very important, return to step S44 to select a new threshold a', and prune the network until the loss function of the pruned network is less than or equal to the loss function of the original network;

[0068] S47: if the loss function of the pruned network is less than or equal to the loss function of the original network, a compact weight parameter W3 is obtained, corresponding to the new network structure;

[0069] S48: the compact weight parameter W3 and the corresponding new network are retrained n times to obtain the updated weight W n ;

[0070] S49: repeat steps S43-S48 until the change range of the loss function is within the preset range, and obtain the final weight W n .

[0071] 3. Beneficial effects:

[0072] (1) The pruning in the method is to evaluate the importance of the filter by combining the sparse value parameter of the convolution layer filter and the parameter γ of the BN layer, and to prune the network according to the importance. Specifically, the method uses two factors to calculate the importance of the filter, and prunes the network based on the importance. From the following two factors: first, the sparse value parameter of the convolution layer filter represents the proportion of non-zero elements in the filter, which can be used as an indicator to measure the importance of the filter. When the sparse value of a filter is low, it means that the filter extracts more useful feature information from the input data, and therefore has higher importance; second, the parameter γ of the BN layer is the scaling coefficient of the filter, which represents the scaling adjustment of the filter to the output feature map. A larger γ value means that the filter contributes more to the output, and therefore has higher importance. By combining these two factors, the importance of the filter can be evaluated comprehensively. Then, according to the importance, the network is pruned, and the filters and corresponding scaling coefficients that meet the pruning requirements are pruned, thereby reducing the size of the network parameters and feature maps. Finally, a more compact network model is obtained, which has higher running efficiency and lower storage requirements.

[0073] (2) In this method, after pruning the model to a large extent, L1 regularization is used to sparsely train the model to improve the network model. Further sparsification of the model: pruning may reduce the weight to a very small or even zero level, but there may still be some remaining redundant weights. By applying L1 regularization, the sparsification of model parameters can be further promoted, i.e. more weights are set to 0, further reducing the redundant information in the model and making it more compact. Improve the generalization ability of the model: sparse training can be regarded as a regularization method, by increasing the sparsity constraint, the complexity of the model can be reduced, and it can help to prevent the model from overfitting. L1 regularization sparsifies the weights during training, making the model more robust and improving its generalization ability, so that it performs better on unseen samples. Reduce the complexity and computational cost of the model: the combination of pruning and L1 regularization can reduce the parameter quantity and computational cost of the model. Pruning can remove unnecessary weights and corresponding connections, reducing the computational load and storage requirements of the model. And L1 regularization further reduces the parameter quantity of the model by setting some weights to 0. In this way, the complexity and computational cost of the model can be reduced while maintaining its performance. Therefore, through sparse training combined with L1 regularization after pruning, the efficiency and generalization ability of the model can be further improved, while reducing the complexity and computational cost of the model.

[0074] (3) In this method, the accuracy of the model after pruning and regularization may decrease, so the model after pruning is fine-tuned to restore the accuracy. A portion of the training data is selected for further training to adjust the model to a higher performance level.

[0075] In summary, the method is based on deep learning and uses a lightweight network structure to improve the network architecture while maintaining high detection accuracy, resulting in smaller size, less computational load and faster running speed, thereby meeting the deployment requirements on mobile devices and embedded devices. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 The overall flowchart of the present application is shown in the figure;

[0077] Figure 2 The structured pruning method combined with S1(n) and scaling factor γ in the present application is shown in the figure;

[0078] Figure 3 The flowchart of updating the convolution layer filter weights in step four of the present application is shown in the figure;

[0079] Figure 4 The visual effect diagram after pruning and compression processing using the method is shown in the figure. DETAILED DESCRIPTION

[0080] The application will be described in detail below with reference to the accompanying drawings.

[0081] As shown in the accompanying Figure 1 The image classification method based on the pruning lightweight network structure has the characteristics that it comprises the following steps.

[0082] Step one: loading a pre-trained model based on MobileNetV3 for image classification;

[0083] Step two: using the filter sparsity formula and the BN layer scaling coefficient to calculate the importance index of the filter contained in each convolutional layer; taking each convolutional layer as a unit, pruning the filter contained in the convolutional layer based on the importance index of the filter, and deleting the filter in the convolutional layer below the preset importance index; pruning each convolutional layer in the model to obtain a pruned overall model;

[0084] Step three: using L1 regularization to perform sparse training on the pruned overall model network to generate a lightweight model;

[0085] Step four: updating the weight of the lightweight model, i.e. obtaining an image classification model of the lightweight network structure.

[0086] Further, step two specifically comprises:

[0087] S21: calculating the sparsity value of each filter of the deep convolutional neural network using the sparsity formula, wherein the sparsity formula is:

[0088]

[0089] (1)

[0090] In the above formula, denotes the sparsity value of the nth filter in the lth convolutional layer; is the weight of the convolution kernel; n, c, h, and w are respectively the 4-dimensional tensor of the convolution kernel, and n, c, h, and w are positive integers and [1,W],N is the number of output filters corresponding to the convolutional layer, C is the number of input filter layers corresponding to the convolutional layer, H is the height of the corresponding convolution kernel, and W represents the width of the corresponding convolution kernel; denotes the standard deviation of the activation value of the filter at the corresponding position nchw in the lth layer;

[0091] wherein denotes the average value of the convolution kernel weight in the lth layer, which is calculated according to the following formula

[0092] (2)

[0093] S22: Based on the scaling coefficient γ of the BN layer and the importance determination function of the filter, the importance index of the filter is calculated; the BN layer is placed in the next layer of the convolutional layer, and the output of the convolutional layer is normalized;

[0094] wherein the importance determination function of the filter is:

[0095] (3)

[0096] In the above formula, is the importance score of the i-th filter, which can measure the importance or contribution of the filter in the network; is the sparsity value of the i-th filter calculated based on formula (1); denotes the sum of all γ values of the filters, and each γ value is squared; γ is a parameter of the Batch Normalization (BN) layer, representing the scaling coefficient of the corresponding filter.

[0097] In summary, the formula squares and sums the BN parameters γ of each filter, and adjusts the result to obtain the importance of the filter. The larger the importance value is, the greater the influence of the corresponding filter on the model is.

[0098] S23: According to the filter importance index , the l-th convolutional layer is pruned; the filters corresponding to the pre-set importance index and the corresponding BN layer are pruned, and the remaining filters E' and their BN layers are obtained, and the scaling coefficient of the BN layer is ′;

[0099] S24: The importance set of the remaining m filters E' is , and the threshold value of filter pruning is obtained according to the pre-set pruning rate :

[0100] (4)

[0101] The importance of all remaining filters E' in the network is sorted in ascending order, and the index position of the filter corresponding to the pruning rate is obtained as the threshold value of filter pruning; after obtaining the threshold value, all filters with importance lower than the threshold value in the convolutional layer are pruned from the convolutional layer;

[0102] S25: Continue steps S21-S24 until all convolutional layers are pruned.

[0103] ​Further, the step three reduces the parameter amount of the network by adding sparsity constraint, and makes the model lightweight; as shown in the accompanying drawings, specifically comprising: Figure 2

[0104] S31: define a loss function with an L1 regularization term to perform sparse training;

[0105] The loss function adopted is a cross-entropy loss function, and an L1 regularization term is added behind it; specifically:

[0106] (5)

[0107] In the formula: is the cross-entropy loss function; for the BN layer feature scaling coefficient γ, , The role of is to regularize the feature scaling coefficient γ in the BN layer; β is a hyperparameter, used to control the weight size of the regularization term of the weight coefficient in the full connection layer or the convolution layer, the more the parameters are punished, the larger β is, the greater the influence of the regularization term is, and the closer the parameters in the BN layer are to 0;

[0108] S32: adopt the proximal gradient optimization method APG to perform sparse training; select the regularization parameter λ to control the influence degree of the regularization term on the total loss; specifically comprising the following steps:

[0109] S321: define the objective function as: (6)

[0110] In the formula, x1 is a sparse coefficient vector, A is a basis vector matrix, b is an observation vector, and λ is a regularization parameter;

[0111] S322: solve the gradient: take the partial derivative of the objective function to get:

[0112] =

[0113] =

[0114] (7)

[0115] Among them, is the gradient of the data fitting term is the derivative of the L1 norm of , that is, the sign function;

[0116] S323: adopt the subgradient method to solve the subgradient of the sparse term: that is,

[0117] The subgradient of the sparse term is ​

[0118] (8);

[0119] S324: Solving for the gradient of the data fitting term: The gradient of the data fitting term is:

[0120]

[0121] S325: Perform iterative optimization; use the gradient descent method to find the minimum value of the objective function, and update the parameter x iteratively. 11 To gradually approach the minimum value, the update formula for each iteration is:

[0122] (9)

[0123] in, This represents the parameter vector after the (k+1)th iteration. represents the parameter vector for the k-th iteration; η is the learning rate, which is selected using a trial-and-error method; the shrink function is a soft thresholding function, which is used to achieve sparsity using the following formula;

[0124] (10)

[0125] In the above formula, y is the real number or vector to be processed, and α is the threshold.

[0126] S326: Update the regularization parameter λ; specifically including:

[0127] S3261: Calculate the current sparse coefficient vector The L1 norm is the sum of the absolute values ​​of all elements in the sparse coefficient vector;

[0128] S3262: Estimate the new λ value based on the current L1 norm and a predefined parameter c, using the following formula:

[0129] (11)

[0130] in, It is the L1 norm of the current sparse coefficient vector;

[0131] S3263: Repeat S3262 above until the change in λ is less than the preset threshold or the preset maximum number of iterations is reached.

[0132] Furthermore, in step four, the weights of the lightweight model are updated, as shown in the attached diagram. Figure 3 As shown, by adjusting the initial weights The results are obtained through iterative training and pruning; specifically, the following steps are included:

[0133] S41: initializing a network using He , initializing a pruning mask ; He initialization is an initialization method for ReLU activation function, which is suitable for deep neural networks, and corresponds to the network structure of MobileNet V3;

[0134] S42: training Wo using the hyperparameter β for a preset number of times to obtain updated filter weights W1;

[0135] S43: calculating the importance index m of the weight W1 according to the importance determination function of formula (3) i , and sorting the filters in ascending order of the weight values;

[0136] S44: based on the preset threshold α, retaining all weights satisfying m i greater than or equal to α, and pruning the rest of the weights corresponding to the filters; sorting the retained filters according to the weight values to obtain a list of retained filter weight values; selecting a percentage p, and finding the corresponding weight value in the sorted list according to the percentage, which is used as a new threshold α';

[0137] S45: retaining all weights satisfying the weight value greater than or equal to the new threshold α', and pruning the rest of the weights, i.e. setting the pruning mask θ corresponding to the weights that do not need to be pruned to 1, and setting the pruning mask θ corresponding to the weights that need to be pruned to 0;

[0138] S46: recalculating the loss function according to formula (5), if the loss function of the pruned network is greater than the loss function calculated before step S43, it means that the deleted weight is very important, go back to step S44 to select a new threshold α', and prune the network until the loss function of the pruned network is less than or equal to the loss function of the original network;

[0139] S47: if the loss function of the pruned network is less than or equal to the loss function of the original network, a simplified weight parameter W3 corresponding to the new network structure is obtained;

[0140] S48: retraining the simplified weight parameter W3 and the new network corresponding thereto for n times to obtain the updated weight W n ;

[0141] S49: repeating steps S43-S48 until the change range of the loss function is within the preset range, to obtain the final weight W n .

[0142] In the special network structure of MobileNetV3, the inverted residual module accounts for a large part of the computational load and memory consumption. To reduce the model's parameters and computational load, a pruning scheme based on the basic network unit of MobileNetV3 is proposed. This method prunes the MobileNetV3 module to reduce the operation and memory requirements. MobileNetV3 module is an algorithm for target detection tasks that uses a lightweight convolutional neural network structure. By using the new Block structure, applying NAS (Neural Architecture Search) parameters, and redesigning the structure of the time-consuming layer, the network innovation of MobileNetV3 is achieved. These innovations enable the model to significantly reduce the burden of model parameters and computational load while maintaining high accuracy.

[0143] In depth separable convolution, the number of input and output filters of depth convolution must be consistent. However, direct pruning will cause the number of input and output filters to be inconsistent, making the network unable to run normally. In addition, in depth separable convolution, the main computational load and parameter quantity come from 1x1 point convolution, while depth convolution only occupies very little resource. Therefore, we can indirectly affect depth convolution and subsequent calculations by pruning the 1x1 convolution filter in the network unit, thereby minimizing the parameter quantity and computational load of the network. This can achieve network compression while maintaining the continuity of the network structure. DETAILED EMBODIMENT

[0145] The overall algorithm steps of the method are as follows:

[0146] (1) First, use neural network structure search technology and data and task-driven methods to train an image classification model with relatively high precision but small parameter quantity and computational load. Usually, a network with variable channel number and layer size is used, and the selection of channel number is learned by minimizing the loss of the model after pruning.

[0147] The images processed in the above steps usually need to go through an image preprocessing process, which usually includes grayscale conversion, binary processing and a series of conventional processes. This process is not described in detail as it is a conventional technique.

[0148] (2) Based on the model obtained in the previous step as the baseline model, give each group, block, channel level structure in the network a parameter factor related to its importance, i.e. the scaling factor in the BN layer; then prune each convolutional layer in the model to obtain the pruned overall model.

[0149] (3) Then, L1 norm regularization is applied to these parameter factors, and the accelerated proximal gradient optimization algorithm APG is used for sparse training. L1 regularization is used to sparsely train the model, and the L1 norm penalty term is added to the loss function to encourage the model's weights to become sparse. The effect of L1 regularization is to make some weights gradually approach zero, so that the model only retains features that contribute significantly to prediction. The APG algorithm, as an optimization algorithm, can be used to solve the least squares problem with convex regularization terms. In the case of L1 norm regularization, the APG algorithm can update the regularization parameter λ by repeatedly applying the soft threshold function to find the sparse solution that minimizes the objective function, and the value of the regularization parameter λ is updated during the iteration process to find the sparse solution that minimizes the objective function. Sparse training through L1 regularization can effectively reduce the complexity of the model, improve the interpretability of the model, and to some extent prevent overfitting.

[0150] (4) After the sparse training is completed, we trim the corresponding channels, groups, and blocks according to the parameters close to 0, thereby generating a lightweight model.

[0151] (5) Fine-tune the pruned model to restore the precision performance of the lightweight image classification model. If the fine-tuned model can achieve or approach the precision of the baseline model, usually set to within 5% decline, we return to step (2) and continue the next round of model compression and pruning. Otherwise, the process ends.

[0152] Through the above iterative compression and pruning process, we can gradually reduce the parameter quantity and computational complexity of the model while trying to keep its precision within an acceptable range.

[0153] Model testing:

[0154] There are many commonly used performance indicators in target detection algorithm testing, and the main indicators used in this paper are: precision (P), recall (R), average precision (AP), and average precision (mAP). The meanings of each evaluation indicator are as follows: P is the number of correctly classified positive samples in the algorithm; FP is the number of incorrectly classified positive samples; FN is the number of incorrectly classified negative samples; n is the number of pictures, and NC is the target type.

[0155] Precision (P): measures the percentage of correctly predicted positive samples among all positive predictions:

[0156]

[0157] Recall (R): measures the percentage of correctly predicted positive samples among all positive samples:

[0158]

[0159] Average precision (AP): average of the precision of the predicted samples;

[0160]

[0161] Average precision (mAP): average of the precision of all classes:

[0162]

[0163] The improved algorithm detection effect is verified by the above indexes.

[0164] The test set is used for model testing, a part of the processed data is randomly selected as the test set, the rest is used as the training set, the error between the result trained by the training set and the actual value is calculated, and the correlation analysis is performed to verify the excellence of the model; the results are as shown in Table 1 and Figure 4 .

[0165]

[0166] Table 1 comparison of several pruning criteria (cutting 50%)

[0167] Although the present application has been disclosed in the preferred embodiments as above, they are not intended to limit the present application, and any person skilled in the art can make various changes or modifications without departing from the spirit and scope of the present application, therefore the protection scope of the present application should be defined by the protection scope of the claims of the present application.

Claims

1. An image classification method based on a pruning-based lightweight network structure, characterized in that: Includes the following steps: Step 1: Load the pre-trained MobileNetV3-based model for image classification; Step 2: Calculate the importance index of the filters contained in each convolutional layer using the filter sparsity formula and the BN layer scaling factor; Each convolutional layer is pruned based on its importance index, and filters in that convolutional layer with an importance index lower than a preset value are removed. Pruning is performed on each convolutional layer in the model to obtain the pruned overall model. Step 3: Use L1 regularization to perform sparse training on the pruned overall model network to generate a lightweight model; Step 4: Update the weights of the lightweight model to obtain the image classification model with a lightweight network structure; Step two specifically includes: S21: Calculate the sparse values ​​of each filter in the deep convolutional neural network using the sparsity formula, where the sparsity formula is: ； （1）； In the above formula, This represents the sparse value of the nth filter in the l-th convolutional layer; Let n be the weights of the convolution kernel; n, c, h, w are the 4-dimensional tensors of the convolution kernel, where n, c, h, w are positive integers and ... [1,W], where N is the number of output filters of the corresponding convolutional layer, C is the number of input filters of the corresponding convolutional layer, H is the height of the corresponding convolutional kernel, and W represents the width of the corresponding convolutional kernel; This represents the standard deviation of the activation values ​​of the filter at position nchw in layer l; in The average weight of the l-th layer convolutional kernel is calculated according to the following formula. （2）； S22: Calculate the importance index of the filter based on the scaling factor γ of the BN layer and the importance determination function of the filter; the BN layer is placed after the convolutional layer, and the output of the convolutional layer is normalized. The importance determination function for the filter is: （3）； In the above formula, The importance score for the i-th filter can measure the importance or contribution of the filter in the network. Let be the sparse value of the i-th filter calculated based on formula (1); This represents summing the γ values ​​of all filters and squaring each γ value; γ is a parameter of the Batch Normalization (BN) layer, representing the scaling factor of the corresponding filter; S23: Based on the filter importance index The l-th convolutional layer is pruned; filters with importance levels below a preset threshold and their corresponding BN layers are removed, resulting in the remaining filter E′ and its BN layer. The scaling factor of the BN layer is then set to... ′; S24: The set of importance of the remaining m filters E′ According to the preset pruning rate Obtain the threshold for filter pruning : （4）； To obtain the pruning rate, the importance of all remaining filters E′ in the network is sorted in ascending order. The importance index of the corresponding index position filter is used as the threshold for filter pruning; after obtaining the threshold, all filters with an importance lower than the threshold in the convolutional layer are pruned from the convolutional layer. S25: Continue with steps S21-S24 until pruning is achieved in all convolutional layers; Step three reduces the number of network parameters by adding sparsity constraints, thus making the model lightweight; specifically, it includes: S31: Define a loss function with L1 regularization for sparse training; The loss function used is the cross-entropy loss function, with an L1 regularization term added after it; specifically: （5）； In the formula: Let be the cross-entropy loss function; for the feature scaling factor γ of the BN layer, , Its function is to adjust the feature scaling factor in the BN layer. Regularization is performed; β is a hyperparameter used to control the weight of the regularization term in the weight coefficients of the fully connected layer or convolutional layer. The larger β is, the more parameters are penalized. The larger β is, the greater the influence of the regularization term, and the closer the parameters in the BN layer will be to 0. S32: Sparsity training is performed using the proximal gradient optimization method (APG); a regularization parameter λ is selected to control the impact of the regularization term on the total loss; specifically, the following steps are included: S321: Define the objective function as: (6); In the above formula, x1 is the sparse coefficient vector, A is the basis vector matrix, b is the observation vector, and λ is the regularization parameter. S322: Solving for the gradient: Taking the partial derivative with respect to the objective function, we get: = = （7）； in, It is the gradient of the data fitting term. yes The derivative of the L1 norm, which is the sign function; S323: Solve for the subgradient of sparse terms using the subgradient method: that is... The subgradient of the sparse term is （8）； S324: Solving for the gradient of the data fitting term: The gradient of the data fitting term is: S325: Perform iterative optimization; use the gradient descent method to find the minimum value of the objective function, and update the parameter x iteratively. 11 To gradually approach the minimum value, the update formula for each iteration is: （9）； in, This represents the parameter vector after the (k+1)th iteration. represents the parameter vector for the k-th iteration; η is the learning rate, which is selected using a trial-and-error method; the shrink function is a soft thresholding function, which is used to achieve sparsity using the following formula; （10）； In the above formula, y is the real number or vector to be processed, and α is the threshold. S326: Update the regularization parameter λ; specifically including: S3261: Calculate the current sparse coefficient vector The L1 norm is the sum of the absolute values ​​of all elements in the sparse coefficient vector; S3262: Estimate the new λ value based on the current L1 norm and a predefined parameter c, using the following formula: （11）； in, It is the L1 norm of the current sparse coefficient vector; S3263: Repeat S3262 above until the change in λ is less than the preset threshold or the preset maximum number of iterations is reached.

2. The image classification method based on a pruning-based lightweight network structure according to claim 1, characterized in that: Step four involves updating the weights of the lightweight model by adjusting the initial weights. The results are obtained through iterative training and pruning; specifically, the following steps are included: S41: Initialize a network using He Initialize the pruning mask He initialization is an initialization method for the ReLU activation function, applicable to deep neural networks, corresponding to the network structure of MobileNet V3; S42: Train Wo using hyperparameter β for a preset number of times to obtain the updated filter weights W1; S43: Calculate the importance index m of weight W1 according to the importance judgment function of equation (3). i And sort the filters in ascending order; S44: Based on a preset threshold α, all conditions satisfying m i Weights greater than or equal to α are retained, and the filters corresponding to the remaining weights are pruned; the retained filters are sorted according to their weight values ​​to obtain a list of retained filter weight values; a percentage p is selected, and the corresponding weight value is found in the sorted list according to the percentage, and it is used as the new threshold α'; S45: Retain all weights that satisfy the new threshold α', and prune the remaining weights, that is, set the pruning mask θ corresponding to the weights that do not need to be pruned to 1, and set the weights that need to be pruned to 0. S46: Recalculate the loss function according to equation (5). If the loss function of the pruned network is greater than the loss function calculated before step S43, it means that the deleted weight is very important. Go back to step S44 to select a new threshold α' and prune the network until the loss function of the pruned network is less than or equal to the loss function of the original network. S47: If the loss function of the pruned network is less than or equal to the loss function of the original network, then a weight parameter W3 is obtained that is simplified compared to the original network, corresponding to the new network structure. S48: Retrain the simplified weight parameters W3 and the corresponding new network n times to obtain the updated weights W. n ; S49: Repeat steps S43-S48 until the loss function changes within a preset range, and obtain the final weight W. n .

Images