Model compression method for progressive pruning of YOLOv8 target detection network

Abstract

The invention belongs to the technical field of automatic driving, and particularly relates to a model compression method for progressive pruning of a YOLOv8 target detection network. The method comprises the following steps: selecting pictures from a traffic video, marking a data set of categories, and training a YOLOv8 target detection network to obtain a picture detection model; multiple rounds of continuous pruning are carried out, and L1 regular constraint training is carried out on the model in each round, so that weights are sparse; the weight of each layer of the model is arranged in a descending order in each round of pruning, unimportant weights are screened out, the optimal pruning proportion is found through multiple times of attempts, and the size of the model is gradually reduced; fine adjustment is carried out on the model through retraining after each round of pruning, so that the model detection accuracy is ensured; and during each round of pruning, the weight number of each layer is ensured to be a multiple of a quantized basic unit, so that the model is further quantized and compressed. The method can reduce the model scale under the condition of ensuring the precision, saves the calculation time, adapts to the quantification step, and is suitable for embedded platform development in an automatic driving scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving, and in particular relates to a model compression method for progressive pruning of a YOLOv8 target detection network. Background Art

[0002] In autonomous driving, perception algorithms use sensors such as cameras and radars to identify the type, state, distance, and other information of objects around the car. Common techniques use deep learning of neural networks to extract features from images or point cloud data, and use detection heads such as classifiers and bounding box regression to achieve final information judgment. Therefore, the deeper the network layers and the more complex the structure, the more data features can be extracted, achieving more accurate judgments.

[0003] However, complex deep networks often require greater computing power and model inference time. For scenarios such as autonomous driving that place great emphasis on real-time detection, the model needs to be further optimized to meet the requirements of real-world scenarios before the algorithm can be engineered and implemented. A common optimization method is model compression, including pruning, quantization, distillation and other technologies. These technologies are often used together to achieve optimal performance. Model compression can effectively reduce the model size and computational complexity while ensuring the accuracy of model detection, thereby achieving faster reasoning in embedded devices, which is a key link in the field of autonomous driving.

[0004] Among the three types of model compression mentioned above, pruning refers to reducing the size of the model by removing the number of neurons that contribute less to the calculation in the neural network model, thereby reducing the amount of calculation and memory usage, while improving the inference speed. Pruning can be divided into structured pruning and unstructured pruning. Structured pruning usually refers to removing the entire weight or neuron group according to a certain rule, while unstructured pruning refers to arbitrarily removing individual weights or neurons. Pruning is usually achieved by combining training and fine-tuning. During the training process, by setting the pruning criteria (such as comparing weights and a certain threshold), the pruned model is fine-tuned to compensate for the accuracy loss caused by pruning.

[0005] If pruning is performed only once to reduce the original model size to the target size, it will cause a loss of accuracy. In other words, large-scale pruning once alone may cause a significant decrease in model accuracy, because important weights or neurons may be deleted during the pruning process, affecting the model's learning ability and breaking the model's original balance and complexity. There is even a high probability that the accuracy and performance cannot be restored at all during the fine-tuning phase. Once the model performance is severely degraded after pruning, restoring it to the original performance may require the introduction of other extremely complex adjustment strategies, which causes inconvenience to the algorithm process development.

[0006] After training, in order to be applied to autonomous driving, the algorithm needs to be transplanted to the embedded development board on the vehicle, and it cannot be limited to the training server. Therefore, in the deployment phase, in order to adapt to embedded devices such as GPU or NPU commonly used in autonomous driving, quantization conversion to the corresponding format must be performed; and ordinary pruning does not check whether the pruned model parameters meet the quantization standard, which causes great inconvenience to deployment reasoning. Summary of the invention

[0007] The purpose of the present invention is to propose a model compression method for progressive pruning of a YOLOv8 target detection network with high recognition accuracy, significantly reduced model size, and convenient embedded quantization deployment.

[0008] The model compression method of the YOLOv8 target detection network progressive pruning proposed by the present invention comprises the following steps: selecting a data set of images and annotating m categories from a traffic video, and using it to train a YOLOv8 target detection network model (hereinafter referred to as the model) to obtain a picture detection model; performing multiple rounds of progressive pruning, and performing L1 regular constraint training on the model in each round to make the weight sparse; arranging the weights of each layer of the model in descending order in each round of pruning, filtering out unimportant weights, and finding the optimal pruning ratio through cyclic testing to gradually reduce the model size; fine-tuning the model through retraining after each round of pruning to ensure the prediction accuracy of the model; ensuring that the number of weights of each layer is a multiple of a quantization basic unit in each round of pruning, so that the model can be further quantized and compressed;

[0009] YOLOv8 (You Only Look Once version 8) is one of the latest versions of the YOLO series, and its prediction speed and accuracy are improved compared to previous versions. Its topological structure includes convolutional layers, activation layers, batch normalization layers, pooling layers, upsampling layers, fusion layers, etc. Each layer has its own weight parameter w, also known as weight, which is the parameter value learned by the model during training. These weights will continuously adjust themselves during the training process, making the network's prediction results for input data more accurate.

[0010] The specific steps of the model compression method of the present invention are:

[0011] (1) Construction and marking of the model training data set; First, select appropriate frames from the traffic video and use them as the images of the data set. Then, label these images with labels, involving a total of m different categories: such as red light, yellow light, green light, colorless light, pedestrians, cars, trucks and non-motor vehicles. For each image, its label information includes the category of the object and the location anchor frame of the object. The anchor frame consists of the coordinates of four points: upper left, upper right, lower left, and lower right. Assuming that there are n objects that need to be detected in a picture, the corresponding label information will contain 4n anchor frame points to accurately describe the locations of these objects. Let the jth anchor frame point of the ith object in the label be The image width and height are (p w ,p h ), let the format of the converted contour point be The conversion formula is as follows:

[0012]

[0013] The above dataset is divided into training set, validation set and test set in a ratio of 4:2:1.

[0014] (2) Conventional model training; that is, training the yolov8 target detection model, setting the batch size used in each training to 16, and the pixel size of the model's input image to 640x640. And define epoch (training round). Through this process, a .pt model that can detect m categories is trained. The .pt model is a file format used to save and load models in the PyTorch framework of the training model. .pt is the file extension of PyTorch, which is usually used to store trained model parameters and network structures. The .pt file includes the structure and weight information of the network model and can be loaded and used directly. During the detection process, a series of 2D anchor boxes describing the contours of the detected objects and their category information are obtained, which are specifically expressed as follows:

[0015] bbox 2d ={{(x 0 ,y 0 , w 0 ,h 0 ), (x 1 ,y 1 , w 1 ,h 1 ), (x 2 ,y 2 , w 2 ,h 2 ),...,(x n ,y n , w n ,h n)}, (2) where, suppose there are n detected segmented targets, each target’s bbox (bounding box) has four position parameters: x, y, w (width), and h (height), and the set of detected targets is denoted as bbox 2d ;

[0016] (3) Constrained training to make the model weights sparse; After the training in step (2) is completed, add the L1 regularization term to the loss function in training and perform constrained training. Constrained training refers to imposing some additional constraints on the model parameters during the model training process to meet certain specific requirements or optimize certain characteristics of the model. These constraints are usually implemented by modifying the loss function.

[0017] The purpose of constrained training is to further optimize the model so that unimportant weights in the model are more likely to become 0, while keeping important weights unchanged. Unimportant weights refer to those weights that have little impact on the model's prediction results. These weights may correspond to uncritical features or redundant connections in the model. In L1 regularization, smaller absolute values ​​of weights are considered unimportant because they have less impact on the model output. This design can effectively reduce the impact of the pruning process on the prediction accuracy of the original model;

[0018] Next, we will explain the specific steps. First, we add the L1 regularization term N to the loss function of the model. The regularization term is also called the penalty term or penalty value, and its expression is:

[0019] N=λ·|w|, (3)

[0020] Among them, λ is the penalty coefficient, w represents the weight vector of the model, and |w| represents the absolute value of the weight vector. A decay coefficient δ is also added to the regularization term so that the penalty value gradually decreases as the training cycle increases. In the initial stage, a larger penalty term will force the model's weight vector w to tend to be sparse, that is, many weights are close to zero, thereby reducing the complexity of the model. As the training progresses, the model will gradually converge to an appropriate state. "Appropriate state" usually means that the model gradually converges to a state that can balance the generalization ability and model complexity during the training process. This means that when the model reaches this state, its prediction accuracy reaches a certain level and does not overfit the training data. At this time, if a larger penalty term is continued to be used, it may limit the expressiveness of the model and lead to performance degradation. Therefore, by gradually reducing the penalty term, the model can be forced to be simplified in the early stage of training, while the model is allowed to learn and express more freely in the later stage. The formula for the penalty coefficient is as follows:

[0021]

[0022] Among them, λ i represents the penalty coefficient for the i-th iteration, λ0 represents the penalty coefficient at the initial iteration, T is the total number of training cycles, and i represents the i-th training. It can be seen that λ gradually decreases as the training cycle progresses.

[0023] The torch.sign() function is used to calculate the sign of the weight (+1, -1, or 0), and then multiplied by the penalty coefficient to obtain a regularized gradient. The expression of the torch.sign function is:

[0024]

[0025] Apply the calculated regularized gradient to the original weight gradient. Specifically, the loss function of the original yolov8 model is:

[0026] L=L cls +L reg , (6)

[0027] The original gradient is After adding regularization, the loss function and gradient update are:

[0028] L′=L+λ i ·|w|, (7)

[0029]

[0030] Among them, L is the total loss of the original model, L cls is the classification loss term, L reg is the regression loss term. L′ is the total loss after adding the regularization term. and They are the gradients before and after adding the regularization term.

[0031] During back propagation, the optimizer uses this updated gradient to update the weights, implementing L1 regularization to make the weights sparse. The weight values ​​are updated to:

[0032]

[0033] Among them, w and w′ are the weight values ​​before and after the update, and α is the learning rate parameter. Through this step, some weights with smaller absolute values ​​will tend to become 0, achieving weight sparseness.

[0034] (4) Pruning: Arrange the weight values ​​of each layer of the trained model in descending order, set the pruning ratio parameter (factor), and calculate the corresponding threshold (threshold). By comparing the weights and thresholds, unimportant weights are filtered out and only relatively large weights are retained to achieve pruning. The pruning ratio set each time needs to be verified multiple times, because the performance of the pruned model on the validation set will decrease. Each pruning should not exceed 10%, otherwise the model's map (mean average precision, an indicator for evaluating the performance of classification or detection models) will degenerate to 0, and subsequent fine-tuning operations will not be possible. The pruning ratio should be tried multiple times until a ratio that can minimize the model size and maintain the accuracy of the model prediction to a certain extent is found; the specific ratio can be set based on the performance of the experiment and the validation set. Initially, you can start with a smaller ratio, such as 5% or 10%, and then gradually increase it to observe the performance of the model on the validation set until a balance point is found.

[0035] After pruning, the total number of weights retained in each layer is checked. If it is not an integer multiple of the target quantization base value, it is rounded to the nearest multiple of this base value. If the number of weights retained after pruning is too small, it must be required to be aligned to at least 1 times the target quantization base value after pruning;

[0036] The following is a detailed explanation of the steps. Because the batch normalization layer summarizes the weights of each convolutional layer it is connected to, we first traverse each layer of the model, collect the weights w in each batch normalization layer, combine all the collected weights into a tensor, and sort the absolute values ​​of all weights in descending order;

[0037] Calculate the threshold value based on the given ratio parameter. Here, factor determines the ratio of the retained weights, that is, calculate an index position, which determines which value is selected as the threshold from the sorted weight list. The index calculation is:

[0038] index=floor(|W|·f),

[0039] threshold=W index , (10)

[0040] Among them, f is the scale parameter. When f is larger, the index value will be larger, which means that the elements at the end of the sorted list are selected. Since the sorting is in descending order, the elements at the end are relatively small. Therefore, the threshold will be a smaller value. For the target network layer, find the weights of the layer whose absolute value is greater than the current threshold and retain them, and prune the weights less than the threshold. The main target layers for pruning are convolutional layers and batch normalization layers.

[0041] Assuming the quantization base value is 8, the number of remaining channels must be a multiple of 8. If the number of channels screened out under the threshold is not equal to 8, it will be rounded up to a multiple of 8. If the number of retained weights is <8, the threshold is halved and the next round of screening is entered. The loop is exited when the number of retained weights is >= 8. For an example of pruning with a weight of 8, see Figure 3 .

[0042] (5) Fine-tune, that is, delete the regularization term in the loss function, that is, restore the loss function to the loss function of the original model (Formula (6)), and then train the pruned model again. Because the pruning step causes the model to lose part of its structure, the detection accuracy will decrease. Through re-training, the model performance can be improved again until it is consistent with the model accuracy before pruning, ensuring the accuracy of the model prediction. Note the detection accuracy map of the model after step (2) is executed 0 , the detection accuracy after executing step (4) is map p After step (5), the accuracy is map f , which should satisfy the relationship:

[0043] map f ≥map 0 >map p , (11).

[0044] By comparing the performance on the test set and validation set, we can perform multiple pruning and fine-tuning in small amounts. Assuming the target pruning ratio is 10%, compared to pruning 10% once, using multiple pruning to gradually reach a total pruning ratio of 10% can more accurately control the pruning process and reduce the risk of excessive pruning, which is especially effective for small data sets.

[0045] (6) Repeat pruning multiple times; use the fine-tuned model as the starting model for the next round of pruning, and repeat steps (3), (4), and (5) in sequence, that is, add an L1 regularization term to the loss function in step (3) and perform constrained training; then prune the constrained trained model in step (4); then remove the regularization term of the loss function in step (5) and perform fine-tuning training. The prediction accuracy of the model must be ensured after each pruning. During multiple consecutive pruning processes, the pruning ratio should not exceed 10% each time. To ensure the performance of the pruned model, it is necessary to find the best pruning ratio for each round and perform fine-tuning. The best pruning ratio is a compromise between performance loss and model compression effect. It is necessary to find a pruning ratio that achieves the best balance between the accuracy of the model and the compression effect (the ratio of model size reduction) brought about by pruning. A series of experiments and verifications should be conducted to compare the performance of the model under different pruning ratios, and the best ratio should be determined based on actual application requirements. If the detection accuracy of the pre-pruned model cannot be achieved after fine-tuning training, it means that too much pruning has had an irreversible negative impact on the model. In this case, it is necessary to go back to step (4), reduce the pruning ratio of the current round, and then perform fine-tuning training again.

[0046] Assume that the initial model size is a 0 , each pruning ratio is p i , the total number of pruning is i, and the target model size is a target , then the relationship satisfied is:

[0047]

[0048] Output the final trained model and convert the format from .pt to .onnx to facilitate deployment and reasoning on embedded platforms. .onnx is the file extension of Open Neural Network Exchange. ONNX is an open format used to represent deep learning models. Compared with the .pt format that can only run in the pytorch environment, it has better portability and interoperability and can run in multiple software environments such as python and c++. The onnx model is significantly smaller than the normally trained model and ensures similar prediction accuracy. At the same time, it aligns the target quantization base value, and can directly perform quantization operations to further speed up reasoning in the target device.

[0049] The model compression method proposed in the present invention has high accuracy, significant model size reduction, strong transparency and traceability of the execution process, and is particularly suitable for embedded scenarios in the field of autonomous driving. Compared with the prior art, the present invention has the following advantages:

[0050] First, compared with the commonly used YOLOv3 or YOLOv5 image detection and pruning methods, the present invention adopts a newer generation of YOLOv8 network as a baseline, which improves the prediction accuracy and reasoning speed of the algorithm model itself from the perspective of baseline performance;

[0051] Second, compared with the commonly used model compression method of directly pruning the model once, the present invention adopts a method of continuous multiple small-scale pruning, which can more finely control the changes in the model structure and more effectively trace the results of each pruning, which is conducive to better maintaining the robustness and accuracy of the model and preventing excessive changes in the model structure caused by excessive pruning ratios, which affects the performance of the final detection of the model;

[0052] Third, compared with the previous method of using pruning alone to compress the model, the present invention also takes into account the ability to match the model quantization on the embedded deployment side, and considers model training and model deployment at the same time. It is particularly suitable for scenarios such as autonomous driving that require algorithm models to be embedded and deployed on edge devices such as development boards. It can combine a variety of different model compression methods to improve the real-time performance of the algorithm operation, which is particularly suitable for the needs of algorithm engineering implementation. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a flow chart of the model compression method for progressive pruning of the YOLOv8 target detection network of the present invention.

[0054] Figure 2 Schematic diagram of L1 regularization.

[0055] Figure 3 Schematic diagram of the pruning process.

[0056] Figure 4 This is a diagram for fine-tuning.

[0057] Figure 5 A comparison chart of single pruning and continuous multiple pruning.

[0058] Figure 6 Schematic diagram of the original pre-trained test image of the YOLOv8 network.

[0059] Figure 7 Schematic diagram of the test image after YOLOv8 network pruning. DETAILED DESCRIPTION

[0060] The present invention is further described below through embodiments in conjunction with the accompanying drawings.

[0061] like Figure 1As shown, the model compression method based on progressive pruning of the YOLOv8 target detection network provided by the present invention comprises: selecting pictures from traffic videos and annotating a data set of m categories, training the target detection network of YOLOv8, and obtaining a picture detection model; performing multiple rounds of progressive pruning, and performing L1 regular constraint training on the model in each round to make the weight sparse; in each round of pruning, the weights of each layer of the model are arranged in descending order, unimportant weights are filtered out, and the optimal solution pruning ratio is found through cyclic testing to gradually reduce the model size; after each round of pruning, the model is fine-tuned through retraining to ensure the prediction accuracy of the model; in each round of pruning, the number of weights of each layer is ensured to be a multiple of the quantization basic unit, so that the model can be further quantized and compressed; the specific steps are:

[0062] Step 1: Labeling of original video data

[0063] First, select appropriate frames from the traffic video and use them as images in the dataset. Then annotate these images, involving a total of m different categories: such as red lights, yellow lights, green lights, colorless lights, pedestrians, cars, freight trucks, and non-motor vehicles. For each image, its label information includes the category of the object and the location anchor frame of the object. The anchor frame consists of the coordinates of four points: upper left, upper right, lower left, and lower right. Assuming that there are n objects that need to be detected in a picture, the corresponding label information will contain 4n anchor frame points to accurately describe the locations of these objects. Let the jth anchor frame point of the ith object in the label be The image width and height are (p w ,p h ), let the format of the converted contour point be The conversion formula is as follows:

[0064]

[0065] The above dataset is divided into training set, validation set and test set in a ratio of 4:2:1.

[0066] Step 2: Conventional model training

[0067] When training the yolov8 object detection model, the batch size used in each training is set to 16, and the pixel size of the model's input image is set to 640x640. The number of training epochs is defined. Through this process, a .pt model that can detect m categories is trained. During the detection process, a series of 2D anchor boxes describing the contours of the detected objects and their category information are obtained, which are specifically expressed as follows:

[0068] bbox 2d ={{(x 0 ,y 0 , w 0 ,h0 ), (x 1 ,y 1 , w 1 ,h 1 ), (x 2 ,y 2 , w 2 ,h 2 ),...,(x n ,y n , w n ,h n )}, (2)

[0069] Among them, suppose there are n segmented targets detected, and the bbox (bounding box) of each target has four position parameters: x, y, w (width), and h (height). The set of detected targets is recorded as bbox 2d ;

[0070] Step 3: Constraint Training

[0071] After the training in step 2 is completed, add L1 regularization term to the loss function in training and perform constrained training. Constrained training refers to imposing some additional constraints on model parameters during model training to meet certain specific requirements or optimize certain characteristics of the model. These constraints are usually implemented by modifying the loss function.

[0072] The purpose of constrained training is to further optimize the model so that unimportant weights in the model are more likely to become 0, while keeping important weights unchanged. Unimportant weights refer to those weights that have little impact on the model's prediction results. These weights may correspond to uncritical features or redundant connections in the model. In L1 regularization, smaller absolute values ​​of weights are considered unimportant because they have less impact on the model output. This design can effectively reduce the impact of the pruning process on the prediction accuracy of the original model;

[0073] like Figure 2, using the 2-dimensional case for illustration, the red curve is the original loss function, and the green diamond is the L1 regularization function. If there is no regularization, the maximum value point on each red contour line will appear at the intersection of the red dotted line and the red curve; if regularization is added, it is equivalent to the maximum value point must appear at the intersection of the green diamond boundary and the red curve. It can be seen that compared with the absence of green diamond constraints, when the area of ​​the green diamond is small, the maximum value point will deviate to the vertical axis. The area of ​​this green diamond can be determined by the regularization coefficient. The larger the regularization coefficient, the smaller the area of ​​the green diamond and the larger the area of ​​the red curve. Therefore, when the regularization coefficient is relatively large, the maximum value points are all on the vertical axis, that is, the weight of a certain dimension is 0, which achieves sparseness, which is equivalent to filtering some weights. Of course, it should also be noted that when the dimension is large, the larger the regularization coefficient, the more weights equal to 0, and there is also the reverse harm of excessive filtering, resulting in underfitting.

[0074] Specifically, first add the L1 regularization term N to the loss function of the model. The regularization term is also called the penalty term or penalty value, and its expression is:

[0075] N=λ·|w|, (3)

[0076] Among them, λ is the penalty coefficient, w represents the weight vector of the model, and |w| represents the absolute value of the weight vector. A decay coefficient δ is also added to the regularization term so that the penalty value gradually decreases as the training cycle increases. In the initial stage, a larger penalty term will force the model's weight vector w to tend to be sparse, that is, many weights are close to zero, thereby reducing the complexity of the model. As the training progresses, the model will gradually converge to an appropriate state. "Appropriate state" usually means that the model gradually converges to a state that can balance the generalization ability and model complexity during the training process. This means that when the model reaches this state, its prediction accuracy reaches a certain level and does not overfit the training data. At this time, if a larger penalty term is continued to be used, it may limit the expressiveness of the model and lead to performance degradation. Therefore, by gradually reducing the penalty term, the model can be forced to be simplified in the early stage of training, while the model is allowed to learn and express more freely in the later stage. The formula for the penalty coefficient is as follows:

[0077]

[0078] Among them, λ i represents the penalty coefficient for the i-th iteration, λ 0 represents the penalty coefficient at the initial iteration, T is the total number of training cycles, and i represents the i-th training. It can be seen that λ gradually decreases as the training cycle progresses.

[0079] The torch.sign() function is used to calculate the sign of the weight (+1, -1, or 0), and then multiplied by the penalty coefficient to obtain a regularized gradient. The expression of the torch.sign function is:

[0080]

[0081] Apply the calculated regularized gradient to the original weight gradient. Specifically, the loss function of the original yolov8 model is:

[0082] L=L cls +L reg , (6)

[0083] The original gradient is After adding regularization, the loss function and gradient update are:

[0084] L′=L+λ i ·|w|, (7)

[0085]

[0086] Among them, L is the total loss of the original model, L cls is the classification loss term, L reg is the regression loss term. L′ is the total loss after adding the regularization term. and They are the gradients before and after adding the regularization term.

[0087] During back propagation, the optimizer uses this updated gradient to update the weights, implementing L1 regularization to make the weights sparse. The weight values ​​are updated to:

[0088]

[0089] Where w and w′ are the weight values ​​before and after the update, and α is the learning rate parameter. Through this step, some weights with smaller absolute values ​​will tend to become 0, achieving weight sparsification.

[0090] Step 4: Pruning

[0091] Arrange the weight values ​​of each layer of the trained model in descending order, set the pruning ratio parameter (factor), and calculate the corresponding threshold (threshold). By comparing the weights and thresholds, unimportant weights are filtered out, and only relatively large weights are retained to achieve pruning. The pruning ratio set each time needs to be verified multiple times, because the performance of the pruned model on the validation set will decrease. Each pruning should not exceed 10%, otherwise the model's map (mean average precision, an indicator for evaluating the performance of classification or detection models) will degenerate to 0, and subsequent fine-tuning operations will not be possible. The pruning ratio should be tried multiple times until it can be found to minimize the model size and maintain the accuracy of the model prediction to a certain extent; the specific ratio can be set according to the performance of the experiment and the validation set. Initially, you can start with a smaller ratio, such as 5% or 10%, and then gradually increase it to observe the performance of the model on the validation set until a balance point is found.

[0092] After pruning, the total number of weights retained in each layer is checked. If it is not an integer multiple of the target quantization base value, it is rounded to the nearest multiple of this base value. If the number of weights retained after pruning is too small, it must be required to be aligned to at least 1 times the target quantization base value after pruning;

[0093] The following is a detailed explanation of the steps. Because the batch normalization layer summarizes the weights of each convolutional layer it is connected to, we first traverse each layer of the model, collect the weights w in each batch normalization layer, combine all the collected weights into a tensor, and sort the absolute values ​​of all weights in descending order;

[0094] Calculate the threshold value based on the given ratio parameter. Here, factor determines the ratio of the retained weights, that is, calculate an index position, which determines which value is selected as the threshold from the sorted weight list. The index calculation is:

[0095] index=floor(|W|·f),

[0096] threshold=W index , (10)

[0097] Among them, f is the scale parameter, W is the weight array, |W| is the number of elements in the weight array, and floor represents the floor rounding function. Figure 3 , the calculated |W|·f is 6.4, so the final index is rounded down to 6.

[0098] When f is larger, the index value will be larger, which means that the later elements in the sorted list are selected. Since the sorting is in descending order, the later elements are relatively small. Therefore, the threshold will be a smaller value. For the target network layer, find the weights of the layer whose absolute value is greater than the current threshold and retain them, and prune the weights less than the threshold. The main target layers for pruning are convolutional layers and batch normalization layers.

[0099] Assuming the quantization base value is 8, the number of remaining channels must be a multiple of 8. If the number of channels screened out under the threshold is not equal to 8, it will be rounded up to a multiple of 8. If the number of retained weights is <8, the threshold is halved and the next round of screening is entered. When the number of retained weights is >= 8, the loop is exited. Figure 3 This is an example of pruning with a weight of 8. The number corresponding to index 6 is 2, but there are only 7 numbers from 8 to 2, which is less than an integer multiple of 8, so it is rounded up to 8, that is, 8 to 1 is retained as the weight.

[0100] Step 5: Fine-tune

[0101] After each round of pruning, the fine-tuning step is performed, that is, the regularization term in the loss function is deleted, that is, the loss function is restored to formula (6), and then the pruned model is trained again. Because the pruning step causes the model to lose part of its structure, the detection accuracy will decrease. Through re-training, the model performance is improved again until it is consistent with the model accuracy before pruning, ensuring the accuracy of model prediction. The detection accuracy map of the model after step 2 is executed is recorded as 0 , the detection accuracy after executing step 4 pruning is map p After step 5, the accuracy is map f , which should satisfy the relationship:

[0102] map f ≥map 0 >map p , (11)

[0103] like Figure 4 , by retraining the pruned model, the performance that has decreased after pruning can be restored to the state before pruning.

[0104] By comparing the performance on the test set and validation set, we can perform multiple pruning and fine-tuning in small amounts. Assuming the target pruning ratio is 10%, compared to pruning 10% once, using multiple pruning to gradually reach a total pruning ratio of 10% can more accurately control the pruning process and reduce the risk of excessive pruning, which is especially effective for small data sets.

[0105] Step 6: Repeat pruning multiple times

[0106] The fine-tuned model is used as the starting model for the next round of pruning, and steps (3), (4), and (5) are executed in a loop. That is, in step (3), an L1 regularization term is added to the loss function and constrained training is performed; in step (4), the constrained trained model is pruned; in step (5), the regularization term of the loss function is removed and fine-tuning training is performed. The prediction accuracy of the model must be ensured after each pruning. During multiple consecutive pruning processes, the pruning ratio should not exceed 10% each time. To ensure the performance of the pruned model, it is necessary to find the best pruning ratio for each round and perform fine-tuning. The best pruning ratio is a compromise between performance loss and model compression effect. It is necessary to find a pruning ratio that achieves the best balance between the accuracy of the model and the compression effect (the ratio of model size reduction) brought about by pruning. A series of experiments and verifications should be conducted to compare the performance of the model under different pruning ratios, and the best ratio should be determined based on actual application requirements. If the detection accuracy of the pre-pruned model cannot be achieved after fine-tuning training, it means that too much pruning has had an irreversible negative impact on the model. In this case, it is necessary to go back to step (4), reduce the pruning ratio of the current round, and then perform fine-tuning training again.

[0107] like Figure 5 , assuming the initial model size is a 0 , each pruning ratio is p i , the total number of pruning is i, and the target model size is a target , then the relationship satisfied is:

[0108]

[0109] like Figure 6 and Figure 7 , respectively, are the prediction results of the model in the test set after normal training in step 2 and after multiple pruning and fine-tuning. It can be seen that on the test set images, the pruned model can still detect the correct target, and the detection results are exactly the same as before pruning. In addition, in terms of other model indicators, map has increased slightly from 0.637 to 0.652, and the model size has been significantly reduced from 42.7mb to 38.3mb, a reduction of more than 10% of the original model.

[0110] Output the final trained model and convert the format from .pt to .onnx to facilitate deployment and reasoning on embedded platforms. The onnx model is significantly smaller than the normally trained model and guarantees similar prediction accuracy. It also aligns the target quantization base value, allowing for direct quantization operations, further speeding up reasoning in the target device.

[0111] The above description is a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A model compression method for progressive pruning of YOLOv8 target detection network, It is characterized in that The topological structure of the YOLOv8 target detection network model includes convolutional layer, activation layer, batch normalization layer, pooling layer, upsampling layer, and fusion layer. Each layer has its own weight parameter w, also known as weight; the model compression method includes selecting pictures from traffic videos and annotating m categories of data sets for training the YOLOv8 target detection network to obtain an image detection model; performing multiple rounds of progressive pruning, and each round of pruning first performs L1 regular constraint training on the model to make the weight sparse; and each round of pruning arranges the weights of each layer of the model in descending order, filters out unimportant weights, and finds the optimal pruning ratio through cyclic testing to gradually reduce the model size; after each round of pruning, the model is fine-tuned through retraining to ensure the accuracy of model prediction; in each round of pruning, the number of weights of each layer is ensured to be a multiple of the quantization basic unit, so that the model can be further quantized and compressed; the specific steps are: (1) Construction and labeling of the model training dataset. First, select appropriate frames from the traffic video and use them as the images of the dataset. Then, label these images with tags. For each image, its label information includes the category of the object and the location anchor frame of the object. Let the number of categories be m, and the anchor frame is composed of the coordinates of the upper left, upper right, lower left, and lower right points. Assuming that there are n objects to be detected in an image, the corresponding label information contains 4n anchor frame points to accurately describe the locations of these objects. Let the jth anchor frame point of the ith object in the label be The image width and height are (p w ,p h ), let the format of the converted contour point be The conversion formula is as follows: The above dataset is divided into training set, validation set and test set in a ratio of 4:2:1; (2) Conventional model training; that is, training the yolov8 target detection model, setting the batch size used for each training to 16, and the pixel size of the model's input image to 640x640; and defining the training rounds; through this process, a .pt model that can detect m categories is trained; the .pt file includes the structure and weight information of the network model and can be directly loaded and used; during the detection process, a series of 2D anchor boxes describing the contour of the detected object and its category information are obtained, which are specifically expressed as follows: bbox 2d ={{(x 0 ,y 0 ,w 0 ,h 0 ),(x 1 ,y 1 ,w 1 ,h 1 ),(x 2 ,y 2 ,w 2 ,h 2 ),…, (x n ,y n ,w n ,h n )}, (2) where, suppose there are n detected segmented targets, and the bounding box (bbox) of each target has four position parameters: coordinates x, y, width w, height h. The set of detected targets is recorded as bbox 2d ; (3) Constrained training: After the training in step (2) is completed, add an L1 regularization term to the loss function in the training and perform constrained training. Through constrained training, unimportant weights in the model are more likely to become 0, while keeping important weights unchanged. Unimportant weights refer to those weights that have little impact on the prediction results of the model. These weights may correspond to uncritical features or redundant connections in the model. In L1 regularization, smaller absolute values ​​of weights are considered unimportant because they have less impact on the model output. This can effectively reduce the impact of the pruning process on the prediction accuracy of the original model. (4) Pruning: Arrange the weight values ​​of each layer of the trained model in descending order, set the pruning ratio parameter, and calculate the corresponding threshold; filter out unimportant weights by comparing the weights and thresholds, and only retain weights with relatively large values, thereby achieving pruning; Since the performance of the pruned model on the validation set will decrease, the pruning should not exceed 10% each time, otherwise the mean average precision of the model will degenerate to 0, and subsequent fine-tuning operations will not be possible; therefore, the pruning ratio set each time needs to be verified multiple times, that is, multiple attempts are made to prune the ratio until the model size is reduced to the greatest extent while maintaining the accuracy of model prediction; the specific pruning ratio is set based on the performance of the experiment and the validation set; initially start with a smaller ratio, then gradually increase it, and observe the performance of the model on the validation set until a balance point is found; After pruning, the total number of weights retained in each layer is checked. If it is not an integer multiple of the target quantization base value, it is rounded to the nearest multiple of this base value; if the number of weights retained after pruning is too small, it is required to be aligned to at least 1 times the target quantization base value after pruning; (5) Fine-tuning: that is, deleting the regularization term in the loss function, restoring the loss function to the loss function of the original model, and then training the pruned model again; the performance of the model is improved again by retraining until the accuracy is consistent with the model before pruning, ensuring the accuracy of model prediction; (6) Repeat pruning multiple times; use the fine-tuned model as the starting model for the next round of pruning, and repeat steps (3), (4), and (5) in sequence, i.e., add an L1 regularization term to the loss function in step (3) and perform constrained training; then prune the constrained trained model in step (4); then remove the regularization term of the loss function in step (5) and perform fine-tuning training; ensure the prediction accuracy of the model after each pruning; and in the process of multiple consecutive pruning, the pruning ratio each time shall not exceed 10%.

2. The model compression method according to claim 1, It is characterized in that The constrained training described in step (3) first adds an L1 regularization term N to the loss function of the model; the regularization term is also called a penalty term or penalty value, and its expression is: N=λ·|w|, (3) Among them, λ is the penalty coefficient, w represents the weight vector of the model, and |w| represents the absolute value of the weight vector. In addition, a decay coefficient δ is added to the regularization term so that the penalty value gradually decreases with the increase of the training cycle. The formula of the penalty coefficient is as follows: Among them, λ i represents the penalty coefficient for the i-th iteration, λ 0 represents the penalty coefficient at the initial iteration, T is the total number of training cycles, i represents the i-th training; λ gradually decreases as the training cycle progresses; Use the torch.sign() function to calculate the sign of the weight: +1, -1 or 0, and then multiply it by the penalty coefficient to get a regularized gradient; the expression of the torch.sign function is: Apply the calculated regularized gradient to the original weight gradient; specifically, the loss function of the original yolov8 model is: L=L cls +L reg , (6) The original gradient is After adding regularization, the loss function and gradient update are: L′=L+λ i ·|w|, (7) Among them, L is the total loss of the original model, L cls is the classification loss term, L reg is the regression loss term; L′ is the total loss after adding the regular term; and They are the gradients before and after adding the regular term; During back propagation, the optimizer uses this updated gradient to update the weights, implementing L1 regularization to make the weights sparse; the weight values ​​are updated to: Among them, w and w′ are the weight values ​​before and after the update, and α is the learning rate parameter. Through this step, some weights with smaller absolute values ​​will tend to become 0, thus achieving weight sparsification.

3. The model compression method according to claim 2, It is characterized in that In step (4), since the batch normalization layer of the model summarizes the weights of each convolutional layer to which it is connected, first traverse each layer of the model, collect the weights w in each batch normalization layer, combine all the collected weights into a tensor, and sort the absolute values ​​of all weights in descending order; According to the given pruning ratio parameter, the threshold is calculated; the ratio parameter determines the proportion of the retained weights, that is, an index position is calculated, which determines which value is selected as the threshold from the sorted weight list; the index calculation is: index=floor(|W|·f), threshold=W index , (10) Among them, f is the scale parameter. When f is larger, the index value will be larger, which means that the later elements in the sorted list are selected; since the sorting is in descending order, the later elements are relatively small, and the threshold is a smaller value; for the target network layer, find the weights of the layer whose absolute value is greater than the current threshold and retain them, and prune the weights less than the threshold; the main target layers for pruning are the convolutional layers and batch normalization layers.

4. The model compression method according to claim 3, It is characterized in that The fine-tuning described in step (5), specifically, the detection accuracy map of the model after step (2) is executed 0 , the detection accuracy after executing step (4) is map p After step (5), the accuracy is map f , which should satisfy the relationship: map f ≥map 0 >map p , (11) By comparing the performance on the test set and the validation set, we perform small amounts of pruning and fine-tuning multiple times. Assuming the target pruning ratio is 10%, compared to pruning 10% once, we use multiple prunings to gradually reach a total pruning ratio of 10%, which can more accurately control the pruning process and reduce the risk of excessive pruning.

5. The model compression method according to claim 4, It is characterized in that Step (6) is described as looping through steps (3), (4), and (5). Objective three is to find the best pruning ratio for each round and perform fine-tuning. The best pruning ratio is a compromise between performance loss and model compression effect, that is, to find a pruning ratio that achieves the best balance between model accuracy and compression effect brought by pruning. To this end, the performance of the model under different pruning ratios is compared, and the best ratio is determined in combination with actual application requirements. If the detection accuracy of the pre-pruned model cannot be achieved after fine-tuning training, it means that too much pruning has an irreversible negative impact on the model, then return to step (4), reduce the pruning ratio of the current round, and then perform fine-tuning training again. Assume that the initial model size is a 0 , each pruning ratio is p i , the total number of pruning is i, and the target model size is a target , then the relationship satisfied is:

6. The model compression method according to claim 5, It is characterized in that For the final trained model, convert the format from .pt to .onnx to facilitate deployment and reasoning on embedded platforms.

Images