Lightweight pruning method and system for image recognition task

Abstract

The application relates to the technical field of artificial intelligence, and discloses a light-weight pruning method and system for an image recognition task, which can efficiently and quickly reduce model complexity while ensuring that the accuracy meets the requirements and saving computing resources and storage space. The method comprises the following steps: collecting a training image dataset; performing sparse training on an original YOLOv5s network based on the training image dataset to obtain the weight values of a sparse YOLOv5s model and scaling factors of various convolution kernels; performing fine-tuning training on the model after joint screening and pruning of the sparse model according to the weight values of the model and the scaling factors of the various convolution kernels to obtain a light-weight YOLOv5s model; and inputting image or video data to be detected into the pruned light-weight YOLOv5s model to obtain a target detection result.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a lightweight pruning method and system for image recognition tasks. Background Technology

[0002] Intelligent security and smart mining have become hot research areas, and real-time video monitoring and analysis technology has been widely applied to all aspects of mine production. First, the demand for edge computing in smart mining industrial scenarios is increasing, making it crucial to reduce power consumption at the edge and conserve computing resources and storage space. Second, lightweight modeling is a core technology for achieving edge intelligence, aiming to efficiently compress models while maintaining detection accuracy.

[0003] Currently, lightweight model methods can be broadly categorized into four types: low-precision weight quantization, compact network design, low-rank decomposition, and network pruning. Network pruning is further divided into unstructured pruning and structured pruning. Weight quantization primarily involves directly quantizing 32-bit weights into 8-bit values ​​for storage, reducing the model's storage space. However, during model inference, shared weights still need to be restored to 32 bits, thus not reducing energy consumption or model complexity. Compact network design involves rebuilding the convolutional neural network model, such as ShuffleNets, MobileNets, and GhostNet, but it's not suitable for compressing and lightweighting existing models. Low-rank decomposition utilizes techniques like Singular Value Decomposition (SVD) to approximate the weight matrix in the neural network with a low-rank matrix. This method performs well in fully connected layers, but since computational operations in CNNs mainly originate from convolutional layers, it also cannot effectively reduce model complexity. Network pruning is the most widely researched aspect of model lightweighting. Unstructured pruning is... Weight-based fine-grained pruning allows for the removal of any weight, resulting in typically high model compression rates. However, this disrupts the model's structure, leading to sparse connections and significantly degrading the performance of highly parallel inference models like those on GPUs. Without dedicated hardware or libraries, model compression and acceleration cannot be achieved. Structured pruning, on the other hand, involves channel-level convolutional kernel or layer pruning, preserving the original convolutional structure. This eliminates the need for additional dedicated hardware / libraries for model compression and acceleration. Among structured pruning methods, channel-level convolutional kernel pruning is the most popular, offering finer-grained pruning that remains compatible with traditional deep learning frameworks. However, there is no unified standard for selecting importance metrics and using them to retain important convolutional kernels while discarding redundant ones.

[0004] In summary, there is a need to provide a channel-level convolutional kernel selection and pruning algorithm that can effectively reduce the computational complexity of YOLOv5s models and save computational resources and storage space. Summary of the Invention

[0005] The purpose of this invention is to disclose a lightweight pruning method and system for image recognition tasks, so as to efficiently and quickly reduce model complexity while ensuring that the accuracy requirements are met, thereby saving computing resources and storage space.

[0006] To achieve the above objectives, the method disclosed in this invention includes:

[0007] Step S1: Collect the training image dataset.

[0008] Step S2: Based on the training image dataset, the original YOLOv5s network is trained in a sparsified manner to obtain the weights of the sparse YOLOv5s model and the scaling factors of each convolutional kernel.

[0009] Step S3: Assign an importance coefficient to each weight of the convolutional layer. Obtain a threshold for the weight importance coefficient based on the weight importance coefficient and the global pruning rate. Then, obtain the pruning rate of each convolutional layer and the number of convolutional kernels to be retained in each convolutional layer based on the weight importance coefficient threshold. Finally, obtain a comprehensive importance similarity matrix by fusing the similarity matrix in the direct Euclidean space of the convolutional kernels with the scaling factor of each convolutional kernel to select the convolutional kernels to be retained. Fine-tune the model after pruning the remaining convolutional kernels to obtain the lightweight YOLOv5s model.

[0010] Step S4: Input the image or video data to be detected into the pruned lightweight YOLOv5s model to obtain the target detection result.

[0011] Preferably, step S2 includes the following steps:

[0012] Step S21: Determine the sparsity factor of the channels available for integration in the convolutional network structure.

[0013] Step S22: Use the L1 sparse regularization method to determine the form of the constrained optimization problem.

[0014] Step S23: Determine the iterative form of the regularization penalty coefficient λ.

[0015] Preferably, step S21 includes:

[0016] Step S211: Introduce a trainable scaling factor for each channel of each feature map. The scaling factor adopts the scaling coefficient γ in the batch normalization layer of the convolutional network. The formula for the batch normalization layer is as follows:

[0017]

[0018] In the formula, z in and z outThese are the input and output feature map matrices of the batch normalization layer, respectively, where B is the current minimum batch size during training, and μ is the input and output feature map matrices. B and σ B γ and β are the mean and standard deviation of batch B, and γ and β are trainable scale and displacement transformation parameters.

[0019] Preferably, step S22 includes:

[0020] Step S221, jointly training the YOLOv5s network weights and the scaling factor γ introduced in step S21, the specific form of the constrained optimization problem is as follows:

[0021]

[0022] In the formula, L(·) is the loss function between the predicted value f(x,W) and the true value y, g(γ) is the regularization function of the parameter γ, x is the input image, y is the actual category of the object in the image, and W represents the network weights.

[0023] Preferably, step S221 includes:

[0024] Step S2211: Introduce the Lagrange operator λ and select the L1 norm as the regularization function g(γ) = γ to obtain the sparse regularization expression based on the L1 norm:

[0025]

[0026] Right now:

[0027]

[0028] In the formula, λ is the penalty coefficient.

[0029] Preferably, step S23 includes:

[0030] Step S231: Set the penalty coefficient λ according to the training situation of the dataset, and the penalty coefficient λ decreases adaptively as the number of iterations increases. After training, a sparse YOLOv5s model is obtained.

[0031] Preferably, step S3 includes:

[0032] Step S31: Determine the pruning rate of the convolutional layers that can be pruned and the number of convolutional kernels to be retained based on the overall model pruning rate required.

[0033] Step S32: Filter the number of convolutional kernels to be retained in the pruning convolutional layers determined in step S31, retain the number of convolutional kernels with the highest overall importance, and delete unimportant convolutional kernels and their corresponding output feature map channels and the corresponding channels of the next layer convolutional kernels.

[0034] Step S33: Fine-tune the lightweight YOLOv5s model after joint screening and pruning to restore accuracy.

[0035] Preferably, step S31 includes:

[0036] Step S311, for each weight of the convolutional layer The importance coefficient is assigned using the following formula:

[0037]

[0038] In the formula, i represents the i-th convolutional layer, j represents the j-th convolutional kernel in the corresponding convolutional layer, and q represents the q-th weight in the convolutional kernel. It is the absolute value of the weight, %FLOPs i η is the ratio of the number of floating-point operations per second of the i-th pruning layer to the number of floating-point operations per second of the first convolutional layer, and η is a hyperparameter shared across the network that is greater than or equal to zero.

[0039] Step S312: Globally rank all importance coefficients of all values, and determine the minimum threshold θ of the importance of the weights based on the overall pruning rate P of the demand model. thred Below the threshold θ thred The weights are discarded, and the expression for P is as follows:

[0040]

[0041] In the formula, n i n is the total number of convolutional layers. j n is the number of convolutional kernels in this convolutional layer. q Let Q be the number of weights in the convolution kernel, and let Q be the total number of all weights.

[0042] Step S313, the minimum threshold θ of the pruning importance coefficient determined in step S312. thred Calculate the specific pruning rate p in each convolutional layer. i Rounding down, the number of convolutional kernels that need to be retained in the i-th convolutional layer is Num(K). i ) = n j -int(n j ·p i ), p i The specific expression is as follows:

[0043]

[0044] In the formula, σ(·) is a switching function; the output is 1 when the input is true, and 0 otherwise. Num(K i ) represents the number of convolutional kernels retained in the i-th convolutional layer.

[0045] Preferably, step S32 includes:

[0046] Step S321: Perform k-nearest neighbor recommendation on the convolutional layer to obtain n, which can measure the compactness between the kernels of the i-th convolutional layer. i ×n i The similarity matrix S contains elements that, with the j-th convolutional kernel as the center, yield the similarity coefficients between the j-th convolutional kernel and the h-th convolutional kernel. Its expression is as follows:

[0047]

[0048] In the formula, 1≤j, h≤n i , This represents the distance between the h-th and j-th convolutional kernels in the i-th convolutional layer.

[0049] Step S322: Merge the scaling factor γ and the similarity matrix S of the nearest neighbors of k for joint screening to obtain n i ×n i The joint selection matrix JS is formed by multiplying the similarity coefficient between the h-th and j-th convolutional kernels in the i-th convolutional layer by the scaling factor of the h-th convolutional kernel. Its expression is as follows:

[0050]

[0051] In the formula This is the scaling factor for the h-th convolutional kernel of the i-th convolutional layer after sparsification training in step S20.

[0052] Step S323, from step S313, the number of convolutional kernels to be filtered in the i-th convolutional layer is Num(K) i Iterate through all the convolutional kernels of the convolutional layer and sort the joint screening matrix in descending row order to obtain the screening importance ranking result for each convolutional kernel.

[0053] Step S324: For each convolutional kernel in the i-th convolutional layer, the number of convolutional kernels selected is k, and its initial value is set to Num(K). i The i-th convolutional layer will n i n convolutional kernels i Take the intersection of the group screening results Ω ki Increase the value of k successively until the intersection Ω is reached. ki The number of elements is Num(K) i ).

[0054] Step S325: Repeat steps S321-S324 until the last convolutional layer that can be pruned, and end the convolutional kernel joint selection algorithm.

[0055] Step S326: Traverse each convolutional layer and delete the output feature map channels corresponding to the discarded convolutional kernels of each convolutional layer and the corresponding convolutional kernel channels of the next convolutional layer. Reintegrate the channels and adjust the connection relationship of the layers to obtain the lightweight YOLOv5s model after joint screening and pruning.

[0056] To achieve the above objectives, the present invention also discloses a lightweight pruning system for image recognition tasks, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method.

[0057] The present invention has the following beneficial effects:

[0058] By constructing a comprehensive importance similarity matrix for fusion screening to determine the final convolution kernel, the importance evaluation is more comprehensive. Under the same fine-tuning training, the accuracy of the pruned model is higher. Moreover, it can efficiently and quickly reduce model complexity while ensuring that the accuracy requirements are met, saving computing resources and storage space. It is especially suitable for application scenarios such as rapid recognition of pedestrians and vehicles in mines.

[0059] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0060] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0061] Figure 1 This is a flowchart of a lightweight YOLOv5s algorithm for rapid identification of pedestrians and vehicles underground, according to an embodiment of the present invention.

[0062] Figure 2 This is a graph showing the iterative training changes of the scaling factor γ during the sparsification training in an embodiment of the present invention.

[0063] Figure 3 This is a flowchart of the joint screening and pruning algorithm according to an embodiment of the present invention. (Not shown in the embodiment)

[0064] Figure 4 This is a performance evaluation comparison chart of the original model and the lightweight model after joint screening and pruning in the embodiments of the present invention.

[0065] Figure 5 This is a graph showing the changes in accuracy and loss function during the fine-tuning training of the lightweight model in this embodiment of the invention.

[0066] Figure 6 This is a diagram of pedestrians and vehicles in normal underground conditions according to an embodiment of the present invention.

[0067] Figure 7 This is a diagram showing the results of pedestrian and vehicle detection in normal downhole conditions using the lightweight YOLOv5s model in this embodiment of the invention.

[0068] Figure 8 This is a diagram of pedestrians and vehicles in a dangerous underground state according to an embodiment of the present invention.

[0069] Figure 9 This is a diagram showing the results of pedestrian and vehicle detection in dangerous downhole conditions using the lightweight YOLOv5s model according to an embodiment of the present invention. Detailed Implementation

[0070] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings, but the present invention can be implemented in many different ways as defined and covered by the claims.

[0071] Example 1

[0072] This embodiment discloses a lightweight YOLOv5s algorithm for rapid pedestrian and vehicle identification tasks in mines, such as... Figures 1 to 9 As shown, it includes the following steps:

[0073] Step S1: Collect underground video data and construct a training image dataset of underground miners and mine carts.

[0074] In this step, LabelImg can be used to manually label the miners and mine carts in each image of the underground training dataset, using the smallest bounding rectangle of the target as the ground truth bounding box.

[0075] Step S2: Perform sparse training on the original YOLOv5s network based on the training image dataset.

[0076] This step may specifically include the following steps:

[0077] Step S21 determines the sparsity factor of the channels available for integration in the convolutional network structure. This consists of the following sub-steps:

[0078] Step S211: A trainable scaling factor is introduced for each channel of each feature map. Since one convolutional kernel of this convolutional layer corresponds to one channel in the output feature map, this scaling factor can indirectly select the convolutional kernel. Because each convolutional layer in the YOLOv5 network structure is followed by a batch normalization layer to fix the data distribution on a standard normal distribution with a mean of 0 and a standard deviation of 1, thereby achieving fast convergence and better generalization performance, the scaling factor γ in the normalization method of the batch normalization layer in the convolutional network is used as the channel selection factor in this invention. The formula for the batch normalization layer is as follows:

[0079]

[0080]

[0081] In equation (2-1), z in and z out These are the input and output feature map matrices of the batch normalization layer, respectively, where B is the current minimum batch size during training, and μ is the input and output feature map matrices. B and σ B γ and β are the mean and standard deviation of batch B, and γ and β are trainable scale and shift transformation parameters. The scaling factor γ in equation (2-2) is used to integrate the channels in the feature map. The larger γ is, the more important the channel is, the more important the corresponding convolution kernel is, and the more worthy it is to be selected and inherited.

[0082] Step S22, determine the form of the constrained optimization problem, and select the L1 sparse regularization method, including the following steps:

[0083] Step S221: Jointly train the YOLOv5s network weights and the scaling factor γ introduced in step S21. This problem is abstracted into a constrained optimization problem as follows:

[0084]

[0085] In the formula, L(·) is the loss function between the predicted and the true values, g(γ) is the regularization function for the parameter γ, also known as the regularization term or penalty term for γ, x is the input image, y is the actual category of the object in the image, and W represents the network weights. Generally, the regularization function g(γ) is the norm of the parameter vector, and can be L0 norm, L1 norm, L2 norm, kernel norm, etc.

[0086] Step S2211: Introducing the Lagrange operator λ and selecting the L1 norm as the regularization function g(γ) = γ, we can obtain the sparse regularization expression based on the L1 norm:

[0087]

[0088] Right now:

[0089]

[0090] In the formula, λ is the penalty coefficient. The larger λ is, the more γ is penalized to near the value of 0.

[0091] Step S23, determine the iterative form of the regularization penalty coefficient λ, including the following steps:

[0092] Step S231: Set the penalty coefficient λ based on the training results of the dataset; the order of magnitude is typically 10. -4 To prevent excessive sparsity, the iteration method for λ in this embodiment is set as follows:

[0093] λ=λ·(1 - 0.9·epoch / epochs) (2-6)

[0094] Here, epoch is the current training epoch, and epochs is the total number of training epochs. As the number of training epochs increases, λ decreases linearly to prevent excessive sparsity caused by λ being too large.

[0095] like Figure 2 As shown, after 50 training rounds, 80% of the scaling factor γ is constrained to near 0, and after 150 training rounds, a sparse YOLOv5s model is obtained.

[0096] Step S3: After model sparsification, this embodiment performs pruning and fine-tuning training on the sparsified model. The specific joint selection and pruning algorithm flow is as follows: Figure 4 As shown, firstly, the importance ranking criteria for each weight in the convolutional layer are determined based on the floating-point operations of each convolutional layer. Then, the pruning rate is determined to filter out the importance metric threshold of the weights, thereby determining the number of convolutional kernels to be retained in each convolutional layer. Next, the convolutional kernels to be retained are filtered in the convolutional layers. For each original convolutional kernel, k-nearest neighbor recommendation is performed. Then, the scaling factor γ of each convolutional kernel after sparse training is combined with the similarity matrix to perform a second filtering of the convolutional kernels, resulting in the pruned YOLOv5s model. The steps include:

[0097] Step S31, determining the pruning rate of convolutional layers that can be pruned and the number of convolutional kernels to be retained based on the overall model pruning rate required, includes the following steps:

[0098] Step S311, for each weight of the convolutional layer The importance coefficient is assigned using the following formula:

[0099]

[0100] In the formula, i represents the i-th convolutional layer, j represents the j-th convolutional kernel in the corresponding convolutional layer, and q represents the q-th weight in the convolutional kernel. It is the absolute value of the weight, %FLOPs i η is the ratio of the number of floating-point operations per second of the i-th pruning layer to the number of floating-point operations per second of the first convolutional layer. η≥0 is a hyperparameter shared across the network, which is set to 1 in this embodiment.

[0101] Step S312: Globally rank all importance coefficients of all values, and determine the minimum threshold θ of the importance of the weights based on the overall pruning rate P of the demand model. thred Below the threshold θ thred The weights are discarded, and the expression for P is as follows:

[0102]

[0103] In the formula n i n is the total number of convolutional layers. j n is the number of convolutional kernels in this convolutional layer. q Let Q be the number of weights in the convolution kernel, and let Q be the total number of all weights.

[0104] Step S313, the minimum threshold θ of the pruning importance coefficient determined in step S312. thred Calculate the specific pruning rate p in each convolutional layer. i Rounding down, the number of convolutional kernels that need to be retained in the i-th convolutional layer is Num(K). i ) = n j -int(n j ·p i ), p i The specific expression is as follows:

[0105]

[0106] In the formula, σ(·) is a switching function; the output is 1 when the input is true, and 0 otherwise. Num(K i Let be the number of convolutional kernels retained in the i-th convolutional layer. To avoid discarding all kernels in an entire convolutional layer, a maximum pruning rate threshold P is set for the overall model. thred The maximum pruning threshold P in this invention cannot exceed this threshold. thred It is 0.7.

[0107] Step S32 involves using a filtering algorithm to select the number of convolutional kernels to be retained in the pruning convolutional layers determined in step S31. The algorithm retains the convolutional kernels with the highest overall importance and deletes unimportant convolutional kernels and their corresponding output feature maps and the corresponding channels of the next layer's convolutional kernels. Specifically, this includes the following steps:

[0108] Step S321: Perform k-nearest neighbor recommendation on the convolutional layer to obtain n, which can measure the compactness between the kernels of the i-th convolutional layer. i ×n i The similarity matrix S contains elements that, with the j-th convolutional kernel as the center, yield the similarity coefficients between the j-th convolutional kernel and the h-th convolutional kernel. Its expression is as follows:

[0109]

[0110] In the formula, 1≤j, h≤n i , This represents the distance between the h-th and j-th convolutional kernels in the i-th convolutional layer. In this embodiment, the L2 norm is used. The larger the value, the smaller the distance between the h-th convolutional kernel and the j-th convolutional kernel, the higher the similarity, and the more worthwhile it is for the j-th convolutional kernel to be recommended for inheritance.

[0111] Step S322: Merge the scaling factor γ and the similarity matrix S of the nearest neighbors of k for joint screening to obtain n i ×n i The joint selection matrix JS is formed by multiplying the similarity coefficient between the h-th and j-th convolutional kernels in the i-th convolutional layer by the scaling factor of the h-th convolutional kernel. Its expression is as follows:

[0112]

[0113] In the formula Let h be the scaling factor of the h-th convolutional kernel of the i-th convolutional layer after sparsification training in step S20. The larger the kernel size, the more worthwhile it is to be selected and inherited.

[0114] Step S323, from step S313, the number of convolutional kernels to be filtered in the i-th convolutional layer is Num(K) i ), traverse all convolutional kernels of this convolutional layer, that is Each row of the matrix is ​​sorted in descending order to obtain the recommendation importance ranking result corresponding to each convolutional kernel.

[0115] Step S324: For each convolutional kernel in the i-th convolutional layer, the number of convolutional kernels selected is k, and its initial value is set to Num(K). i The i-th convolutional layer will n i n convolutional kernels i Take the intersection of the group screening results Ω ki Increase the value of k successively until the intersection Ω is reached. ki The number of elements is Num(K) i ).

[0116] Step S325: Repeat steps S321-S324 until the last convolutional layer that can be pruned, and end the convolutional kernel joint selection algorithm.

[0117] Step S326: Traverse each convolutional layer and delete the output feature map channels corresponding to the discarded convolutional kernels of each convolutional layer and the corresponding convolutional kernel channels of the next convolutional layer. Reintegrate the channels and adjust the connection relationship of the layers to obtain the lightweight YOLOv5s model after joint screening and pruning.

[0118] Step S33 involves fine-tuning the lightweight YOLOv5s model after joint screening and pruning to quickly restore accuracy, including the following steps:

[0119] In step S331, the PyTorch 1.10.0 deep learning framework was used, the training image size was 640*640, and the learning rate, batch size, number of iterations and number of classes were set to 0.001, 16, 100 and 2 respectively. At the same time, the Mosaic data augmentation method was used to enhance the images.

[0120] Based on the underground miners and mining truck datasets, this embodiment evaluates the lightweight model after pruning and the original YOLOv5s model in terms of model size, floating-point operations per second (FLOPs), total number of parameters, and mean average precision (mAP@0.5). The evaluation results are shown in Table 1.

[0121] Table 1: Comparison of Performance Parameters between Combined Pruning and the Original Model

[0122] Model storage space / MB 13.8 3.4 FLOPs / GMacs 1.984 0.637 params / M 7.03 1.54 mAP@0.5 / % 81.8 83.1

[0123] From Table 1 and Figure 5 It is evident that the lightweight model after joint screening and pruning significantly reduces the original model in terms of storage space, computational complexity, and number of parameters. It occupies only 24.63% of the storage space, has only 32.1% of the computational complexity, reduces the number of parameters by 78.1%, and improves the average accuracy by 1.3%. Figure 6 It can also be seen that the pruned lightweight model is more stable in fine-tuning training, converges faster, and has higher accuracy.

[0124] Step S4: Input the image or video data to be detected into the pruned lightweight YOLOv5s model to obtain the target detection results for underground vehicle and pedestrian data. For example... Figure 7 , Figure 8 As shown, the lightweight model has a higher recognition rate for underground miners and mine cars, such as... Figure 9 As shown, when the lightweight model detects that the distance between the miner and the moving mining truck is too close, it can issue a warning in time.

[0125] Example 2

[0126] Corresponding to the above embodiments, this embodiment discloses a lightweight pruning system for image recognition tasks, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-described method.

[0127] In summary, the lightweight pruning method and system for image recognition tasks disclosed in the above embodiments of the present invention determine the final convolution kernel by constructing a comprehensive importance similarity matrix for fusion screening. This results in a more comprehensive importance evaluation, and under the same fine-tuning training, the accuracy of the pruned model is higher. Furthermore, it can efficiently and quickly reduce model complexity while ensuring that the accuracy requirements are met, saving computing resources and storage space. It is particularly suitable for application scenarios such as rapid recognition of pedestrians and vehicles underground.

[0128] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A lightweight pruning method for image recognition tasks, characterized in that, Includes the following steps: Step S1: Collect the training image dataset; Step S2: Sparsify the original YOLOv5s network based on the training image dataset to obtain the weights of the sparse YOLOv5s model and the scaling factor of each convolution kernel. Step S3: Assign an importance coefficient to each weight of the convolutional layer. Obtain a weight importance coefficient threshold based on the weight importance coefficient and the global pruning rate. Then, obtain the pruning rate of each convolutional layer and the number of convolutional kernels to be retained in each convolutional layer based on the weight importance coefficient threshold. Finally, obtain a comprehensive importance similarity matrix by fusing the similarity matrix in the direct Euclidean space of the convolutional kernels with the scaling factor of each convolutional kernel to select the convolutional kernels to be retained. Fine-tune the model after pruning the remaining convolutional kernels to obtain the lightweight YOLOv5s model. Step S4: Input the image or video data to be detected into the pruned lightweight YOLOv5s model to obtain the target detection result.

2. The lightweight pruning method for image recognition tasks according to claim 1, characterized in that, Step S2 includes the following steps: Step S21: Determine the sparsity factor of the channels available for integration in the convolutional network structure; Step S22: Use the L1 sparse regularization method to determine the form of the constrained optimization problem; Step S23: Determine the iterative form of the regularization penalty coefficient λ.

3. The lightweight pruning method for image recognition tasks according to claim 2, characterized in that, Step S21 includes: Step S211: Introduce a trainable scaling factor for each channel of each feature map. The scaling factor adopts the scaling coefficient γ in the batch normalization layer of the convolutional network. The formula for the batch normalization layer is as follows: In the formula, z in and z out These are the input and output feature map matrices of the batch normalization layer, respectively, where B is the current minimum batch size during training, and μ is the input and output feature map matrices. B and σ B γ and β are the mean and standard deviation of batch B, and γ and β are trainable scale and displacement transformation parameters.

4. The lightweight pruning method for image recognition tasks according to claim 3, characterized in that, Step S22 includes: Step S221, jointly training the YOLOv5s network weights and the scaling factor γ introduced in step S21, the specific form of the constrained optimization problem is as follows: In the formula, L(·) is the loss function between the predicted value f(x,W) and the true value y, g(γ) is the regularization function of the parameter γ, x is the input image, y is the actual category of the object in the image, and W represents the network weights.

5. The lightweight pruning method for image recognition tasks according to claim 4, characterized in that, Step S221 includes: Step S2211: Introduce the Lagrange operator λ and select the L1 norm as the regularization function g(γ)=|γ| to obtain the sparse regularization expression based on the L1 norm: Right now: In the formula, λ is the penalty coefficient.

6. The lightweight pruning method for image recognition tasks according to claim 2, characterized in that, Step S23 includes: Step S231: Set the penalty coefficient λ according to the training situation of the dataset, and the penalty coefficient λ decreases adaptively as the number of iterations increases. After training, a sparse YOLOv5s model is obtained.

7. The lightweight pruning method for image recognition tasks according to claim 1, characterized in that, Step S3 includes: Step S31: Determine the pruning rate of the convolutional layers that can be pruned and the number of convolutional kernels to be retained based on the overall model pruning rate required; Step S32: Filter the number of convolutional kernels to be retained in the pruning convolutional layers determined in step S31, retain the number of convolutional kernels with the highest overall importance, and delete unimportant convolutional kernels and their corresponding output feature map channels and the corresponding channels of the next layer convolutional kernels. Step S33: Fine-tune the lightweight YOLOv5s model after joint screening and pruning to restore accuracy.

8. The lightweight pruning method for image recognition tasks according to claim 7, characterized in that, Step S31 includes: Step S311, for each weight of the convolutional layer The importance coefficient is assigned using the following formula: In the formula, i represents the i-th convolutional layer, j represents the j-th convolutional kernel in the corresponding convolutional layer, and q represents the q-th weight in the convolutional kernel. It is the absolute value of the weight, %FLOPs i η is the ratio of the number of floating-point operations per second of the i-th pruning layer to the number of floating-point operations per second of the first convolutional layer, and η is a hyperparameter shared across the network that is greater than or equal to zero. Step S312: Globally rank all importance coefficients of all values, and determine the minimum threshold θ of the importance of the weights based on the overall pruning rate P of the demand model. thred Below the threshold θ thred The weights are discarded, and the expression for P is as follows: In the formula, n i n is the total number of convolutional layers. j n is the number of convolutional kernels in this convolutional layer. q Let Q be the number of weights in the convolution kernel, and let Q be the total number of all weights. Step S313, the minimum threshold θ of the pruning importance coefficient determined in step S312. thred Calculate the specific pruning rate p in each convolutional layer. i Rounding down, the number of convolutional kernels that need to be retained in the i-th convolutional layer is Num(K). i ) = n j -int(n j ·p i ), p i The specific expression is as follows: In the formula, σ(·) is a switching function; the output is 1 when the input is true, and 0 otherwise. Num(K i ) represents the number of convolutional kernels retained in the i-th convolutional layer.

9. The lightweight pruning method for image recognition tasks according to claim 7, characterized in that, Step S32 includes: Step S321: Perform k-nearest neighbor recommendation on the convolutional layer to obtain n, which can measure the compactness between the kernels of the i-th convolutional layer. i ×n i The similarity matrix S contains elements that, with the j-th convolutional kernel as the center, yield the similarity coefficients between the j-th convolutional kernel and the h-th convolutional kernel. Its expression is as follows: In the formula, 1≤j, h≤n i , This represents the distance between the h-th convolutional kernel and the j-th convolutional kernel in the i-th convolutional layer; Step S322: Merge the scaling factor γ and the similarity matrix S of the nearest neighbors of k for joint screening to obtain n i ×n i The joint selection matrix JS is formed by multiplying the similarity coefficient between the h-th and j-th convolutional kernels in the i-th convolutional layer by the scaling factor of the h-th convolutional kernel. Its expression is as follows: In the formula is the scaling factor of the h-th convolutional kernel of the i-th convolutional layer after sparsification training in step S20; Step S323, from step S313, the number of convolutional kernels to be filtered in the i-th convolutional layer is Num(K) i ), traverse all convolutional kernels of the convolutional layer, and sort the joint screening matrix in descending row order to obtain the screening importance ranking result for each convolutional kernel; Step S324: For each convolutional kernel in the i-th convolutional layer, the number of convolutional kernels selected is k, and its initial value is set to Num(K). i The i-th convolutional layer will n i n convolutional kernels i Take the intersection of the group screening results Ω ki Increase the value of k successively until the intersection Ω is reached. ki The number of elements is Num(K) i ); Step S325: Repeat steps S321-S324 until the last convolutional layer that can be pruned, and end the convolutional kernel joint selection algorithm. Step S326: Traverse each convolutional layer and delete the output feature map channels corresponding to the discarded convolutional kernels of each convolutional layer and the corresponding convolutional kernel channels of the next convolutional layer. Reintegrate the channels and adjust the connection relationship of the layers to obtain the lightweight YOLOv5s model after joint screening and pruning.

10. A lightweight pruning system for image recognition tasks, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1 to 9.

Images