A traffic sign detection method and system based on YOLOv5

Abstract

The present invention discloses a traffic sign detection method and system based on YOLOv5. By using an online and offline hybrid data enhancement method to achieve data set amplification, designing a C2fGhost module and reconstructing a CSPDarkNet feature extraction network based on the module, it is possible to obtain richer gradient flow information while achieving lightweight; using shallow features P2 to replace deep features P5 for prediction, improving the model's small target detection capability; introducing Efficient‑RepGFPN as a feature fusion network to improve the model's multi-scale detection capability; using Wise‑IoU V3 loss function to replace the original CIoU loss function as a positioning loss function, improving the model's training effect on low-quality data sets; using confidence scaling distillation to further enhance the model's reasoning performance; performing INT8 quantization on the distilled model, significantly reducing the model's parameter amount and calculation amount. The present invention has high small target detection performance and reasoning speed, and has good real-time reasoning performance on performance-constrained platforms such as vehicle-mounted platforms.

Description

Technical Field

[0001] The present invention relates to the fields of computer vision technology and intelligent transportation technology, and in particular to a traffic sign image recognition method, specifically a traffic sign detection method based on YOLOv5, which can be applied to traffic sign detection in advanced driver assistance technology and autonomous driving technology. Background Art

[0002] With the development of deep learning technology, the addition of computer vision technologies such as traffic sign detection has brought revolutionary breakthroughs to advanced driver assistance systems and autonomous driving systems. Traffic sign detection uses deep learning methods to obtain the location and category information of traffic signs from digital images or video frames. By detecting traffic signs such as speed limits and turns, the driver assistance system can adjust the vehicle speed and plan the vehicle route, improve the intelligence and safety of the driver assistance system, and reduce traffic accidents caused by human errors or non-compliance with traffic rules.

[0003] The existing traffic sign detection methods have at least the following technical problems:

[0004] First, existing traffic sign detection methods have poor real-time performance. Many models achieve better detection performance by increasing parameters and FLOPs. However, in actual application scenarios, using accuracy alone as a model evaluation metric is not comprehensive and objective enough, and multiple indicators such as speed and efficiency need to be considered. Due to factors such as power consumption and cost, the computing resources on the vehicle platform are often limited, making it difficult to support tasks such as traffic sign detection that require extremely high real-time performance.

[0005] Secondly, the existing traffic sign detection methods have a high missed detection rate for small-sized targets. In the traffic sign detection task, small target detection is inevitable. As the vehicle moves forward, the traffic signs that initially appear as small targets will gradually increase in size as the distance decreases. Small targets are difficult to distinguish and locate because they have a small pixel area and contain less color and feature information. If the vehicle cannot detect small traffic signs, it will reduce the vehicle's reaction time and greatly affect the safety performance of the autonomous driving vehicle. Therefore, improving the accuracy of small target detection can effectively improve the safety performance of autonomous driving. Summary of the invention

[0006] The present invention provides a traffic sign detection method based on YOLOv5, which is used to solve or at least partially solve the problems of poor real-time detection and high missed detection rate of small-sized targets in existing traffic sign detection methods.

[0007] The technical solution of the present invention is a traffic sign detection method based on YOLOv5, which includes the following steps:

[0008] S1, prepare the data set, use the online and offline hybrid data enhancement method to preprocess the traffic sign detection data set to expand the data set and reduce the damage of Mosaic data enhancement to small target images;

[0009] S2, build a traffic sign detection network based on YOLOv5 as the target detection network, and improve the positioning loss function in the network;

[0010] S3, trains the object detection network, adopts the object detection loss function, and uses mixed precision training, warm-up learning, cosine annealing learning rate and early stopping training strategy;

[0011] S4, using confidence scaling distillation loss to perform knowledge distillation on the trained target detection network model, and finally combining the target detection loss function and the distillation loss function in a weighted manner to form the final loss function;

[0012] S5, use OpenVINO post-training optimization tool to perform INT8 quantization on the model after knowledge distillation.

[0013] Furthermore, the online and offline hybrid data enhancement method of step S1 includes the following steps:

[0014] S1.1, firstly adopts the copy-paste offline data augmentation algorithm to achieve dataset augmentation based on image context and scale information;

[0015] S1.2, then the probability Mosaic online data enhancement algorithm is used to determine the probability of using Mosaic data enhancement according to the proportion of small-scale annotations in the image; the small-scale definition here uses the MS COCO dataset standard: objects with a resolution less than 32 pixels × 32 pixels are defined as small-scale annotations, and the probability calculation method used by Mosaic is:

[0016]

[0017] Where n S To process the number of small-scale annotations of the image, n all The total number of annotations for processed images.

[0018] Furthermore, the traffic sign detection network based on YOLOv5 in step S2 includes three parts: CSPDarkNet feature extraction network, feature fusion network and detection head, and the specific implementation method is as follows:

[0019] The structure of the CSPDarkNet feature extraction network is as follows: first, the input image is processed using the CBS module to obtain the shallow feature P1, then a 4-level feature extraction module is formed through 4 groups of CBS modules with a step size of 2 and C2fGhost modules, and the shallow feature P1 is further processed to obtain P2-P5 in sequence, among which P3 and P4 are connected using CBS to achieve cross-scale feature fusion and information transfer as the input of the feature fusion module, and at the same time, the deep feature P5 is input into the SPPF structure to obtain the fused input feature, which is also used as the input of the feature fusion module;

[0020] Efficient-RepGFPN is introduced as a feature fusion network, and its processing process is as follows: first, the input features are fused through CBS and upsampling operations to obtain the first fused feature, the low-level features are mapped to the same scale as the high-level features, and then connected with the features of the original feature P4 after CBS processing to retain the information of the original features; then, the features are processed through Queen-Fusion and CBS to obtain the second fused feature to enhance the expression ability of the features, and then the detail information is restored through upsampling operations, and the upsampled features are connected with the feature P3 to realize information transmission and interaction between features, and then the output information is passed through the first Queen-Fusion. The en-Fusion module improves the resolution and uses it as the first-layer output F1 to describe large-scale semantic information. At the same time, the output of the first Queen-Fusion module is input into the CBS module for processing and fused with the second fusion feature to retain the original information. Then, through the second Queen-Fusion module, the feature at this time is used as the second-layer output F2 to describe the medium-scale semantic information; finally, the output of the second Queen-Fusion module is input into the CBS module for processing and fused with the first fusion feature, and then through the third Queen-Fusion module to obtain the third-layer output F3 to describe the small-scale semantic information.

[0021] Input F1, F2 and F3 into the detection head respectively to obtain the detection results.

[0022] Furthermore, in step S2, the Wise-IoU V3 loss function is used to replace the original CIoU loss function as the positioning loss function, and the Wise-IoU V3 loss calculation method is:

[0023]

[0024]

[0025]

[0026]

[0027]

[0028] Where IoU is the intersection over union ratio, is the IoU loss, (x, y) is the coordinate of the center point of the prediction box, (x gt ,y gt ) is the coordinate of the center point of the annotation box; W g and H g is the width and height of the minimum enclosing box, β is the definition of outlier degree, is the IoU loss of the current image, is the mean IoU loss of the current batch of images; α and β are hyperparameters used to control the outlier degree and gradient gain. is the Wise-IoU penalty term, which is used to significantly amplify the IoU loss of ordinary quality anchor boxes. It is the Wise-IoU v1 loss.

[0029] Furthermore, in step S3, the target detection loss L Detection The calculation formula is as follows;

[0030]

[0031]

[0032] Among them, f obj is the confidence loss, f cl is the classification loss, f bb is the positioning loss, is the confidence, category probability and coordinate information of the true bounding box, They are the confidence, category probability and coordinate information of the target detection network predicting the bounding box, is the confidence value after processing with the sigmoid function, which is used as the weight of the positioning loss and classification loss;

[0033] The classification loss is the Wise-IoU V3 loss function, and the confidence loss f obj and classification loss f cl The calculation formula is:

[0034]

[0035]

[0036] Among them, N is the number of positive and negative samples, N pos is the number of positive samples, Represents the class probability of the positive sample.

[0037] Further, in step S4, the target detection network in step S2 is used as a teacher network, the teacher network is used to infer the training data, the prediction result of the teacher network is obtained, the confidence difference between the target category and the background area is calculated, and then the output of the teacher network and the true label of the data are used to train the student network, wherein the network structure of the student network is the same as that of the teacher network;

[0038] Confidence-scaling distillation loss L Distill The calculation formula is as follows:

[0039]

[0040] Among them, f obj is the confidence loss, f cl is the classification loss, f bb is the positioning loss, The confidence, category probability and coordinate information of the bounding box predicted by the student network are respectively; is the confidence, category probability and coordinate information of the teacher network’s predicted bounding box; is the confidence value after processing with the sigmoid function. This value is used as the weight of the positioning loss and the classification loss. The calculation method is the same as formula (8);

[0041] Confidence loss f obj and classification loss f cl The calculation formula is:

[0042]

[0043]

[0044] Among them, N is the number of positive and negative samples, N pos is the number of positive samples; Represents the class probability of the positive sample.

[0045] Furthermore, the final loss of the student network is obtained by weighted summation of the distillation loss and the target detection loss. The calculation formula for the final loss is:

[0046]

[0047] Among them, λ D Represents weight.

[0048] Furthermore, step S5 specifically includes the following steps:

[0049] S5.1, implement a custom YOLOv5Dataloader subclass by inheriting the POT DataLoader base class;

[0050] S5.2, implement a custom COCOMetric subclass by inheriting the POT Metric base class;

[0051] S5.3, use the DQ (Default Quantization) quantization strategy; perform symmetric quantization on model weights and asymmetric quantization on model activations.

[0052] Furthermore, the method further includes step S6, in which the performance of the training model is evaluated, and the model is evaluated based on evaluation indicators such as detection accuracy, recall rate, average detection precision mAP, detection speed FPS and model size.

[0053] The present invention also provides a traffic sign detection system based on YOLOv5, comprising the following modules:

[0054] The preprocessing module is used to prepare the data set and use the online and offline hybrid data enhancement method to preprocess the traffic sign detection data set to expand the data set and reduce the damage of Mosaic data enhancement to small target images;

[0055] The network construction module is used to build a traffic sign detection network based on YOLOv5 as the target detection network and improve the positioning loss function in the network;

[0056] The network training module is used to train the target detection network, adopt the target detection loss function, and use mixed precision training, warm-up learning, cosine annealing learning rate and early stopping training strategy;

[0057] The knowledge distillation module is used to perform knowledge distillation on the trained target detection network model using confidence scaling distillation loss, and finally combines the target detection loss function and the distillation loss function in a weighted manner to form the final loss function;

[0058] The quantization module is used to perform INT8 quantization on the model after knowledge distillation using the OpenVINO post-training optimization tool.

[0059] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0060] The present invention provides an online and offline hybrid data enhancement method. By adopting a copy-paste offline data enhancement algorithm, instances of a smaller number of categories are added, the number of samples is effectively balanced and the expansion of a traffic sign detection data set is achieved. By adopting a probabilistic Mosaic online data enhancement algorithm, the damage of image compression to small target images during training is effectively reduced, thereby improving the small target detection capability of the model.

[0061] The present invention improves the YOLOv5 network, designs the C2fGhost module and reconstructs the CSPDarkNet feature extraction network based on the module, which can obtain richer gradient flow information while achieving lightweight; uses shallow features P2 to replace deep features P5 for prediction, which can improve the performance of small target detection of the model; introduces Efficient-RepGFPN as a feature fusion network to achieve higher detection accuracy without introducing additional computational burden; uses Wise-IoU V3 as a positioning loss function to improve the training effect of the model on low-quality data sets.

[0062] The present invention further optimizes the improved YOLOv5 model, uses the confidence scaling distillation method to perform knowledge distillation, and further improves the detection performance of the model; uses the OpenVINO post-training optimization tool to perform INT8 quantization on the model, significantly reduces the amount of model calculation and parameters, and improves the model reasoning speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0064] Figure 1 Flowchart of a traffic sign detection method based on YOLOv5 in an embodiment of the present invention;

[0065] Figure 2 This is a network architecture diagram of a traffic sign detection method based on YOLOv5 in an embodiment of the present invention;

[0066] Figure 3 This is a diagram of the CSPDarknet network architecture based on C2fGhost in an embodiment of the present invention;

[0067] Figure 4 Schematic diagram of the confidence scaling distillation process in an embodiment of the present invention.

[0068] Figure 5 It is a general flow chart of INT8 quantization based on POT API in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The technical problem to be solved by the present invention is that the existing traffic sign detection method has poor real-time detection and a high rate of missed detection of small-sized targets. The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0070] As an example, this embodiment uses the open source traffic sign dataset TT100K (Tsinghua-Tencent 100K) of the Tsinghua-Tencent Joint Laboratory, and selects 45 traffic sign categories that appear more than 100 times in the dataset as research objects.

[0071] As attached Figure 1 As shown, the embodiment of the present invention provides a traffic sign detection method based on YOLOv5, and the implementation steps include:

[0072] S1: Prepare the dataset. First, convert the annotation format of the TT100K dataset into the YOLO annotation format. Since the number of training samples in the TT100K dataset varies greatly and there are many small-scale targets, the present invention designs an online and offline hybrid data enhancement method. This method can expand the TT100K dataset and reduce the damage of Mosaic data enhancement to small target images. Finally, the training set is divided into a training set and a validation set in a ratio of 8:2.

[0073] Specifically, online and offline hybrid data enhancement methods include the copy-paste offline data enhancement algorithm and the probabilistic Mosaic online data enhancement algorithm. The former draws on the idea of ​​the Copy-Paste data enhancement algorithm, pastes based on image context, scale and other information, and achieves data amplification by copying and pasting other marks with the same aspect ratio in the data set, solving the problem of large differences in the number of training samples. The latter determines the probability of using Mosaic data enhancement based on the proportion of small-scale annotations in the image, reducing the damage of Mosaic data enhancement to small target images.

[0074] The specific implementation is as follows:

[0075] S1.1, firstly adopt the copy-paste offline data augmentation algorithm to paste based on image context, scale and other information to achieve dataset augmentation;

[0076] S1.2, then use the probabilistic Mosaic online data enhancement algorithm to determine the probability of using Mosaic data enhancement based on the proportion of small-scale annotations in the image. The definition of small scale here uses the MS COCO dataset standard: objects with a resolution less than 32 pixels × 32 pixels are defined as small-scale annotations. Mosaic uses the probability calculation method:

[0077]

[0078] Where n S To process the number of small-scale annotations of the image, n all The total number of annotations for processed images.

[0079] S2: Build a traffic sign detection network based on YOLOv5 as the target detection network, and improve the feature extraction network, feature fusion network and loss function of YOLOv5. The improved network architecture is shown in the attached figure. Figure 2 shown.

[0080] Specifically, the present invention designs a C2fGhost module and reconstructs the CSPDarkNet feature extraction network based on the module. Compared with the C3 module used by YOLOv5, the C2fGhost module can obtain richer gradient flow information while achieving lightweight. The CSPDarknet network architecture based on C2fGhost is shown in the attached figure. Figure 3 As shown. The present invention uses shallow features P2 to replace deep features P5 for prediction. The local receptive field of shallow features is smaller, and contains more position and detail information, which can improve the performance of small target detection of the model. The present invention introduces Efficient-RepGFPN as a feature fusion network. Efficient-RepGFPN uses the CSP structure for feature fusion. On this basis, Efficient-RepGFPN upgrades the CSP structure through the connection of the re-parameterization mechanism and the efficient layer aggregation network, and achieves higher detection accuracy without bringing additional computational burden. The present invention adopts the Wise-IoU V3 loss function to replace the original CIoU loss function as the positioning loss function, which can achieve better training effects on data sets of general quality.

[0081] The specific implementation is as follows:

[0082] S2.1, design the C2fGhost module, refer to the design of the C2f module, and use the GhostBottleneck module to replace the bottleneck structure. Reconstruct the CSPDarkNet feature extraction network based on the C2fGhost module. Compared with the C3 module used in YOLOv5, the C2fGhost module can achieve lightweight while obtaining richer gradient flow information.

[0083] The structure of the CSPDarkNet feature extraction network is as follows: First, the input image is processed using a CBS module with a size of 6×6, a step size of 2, and a padding of 2 to obtain the shallow feature P1. This module has the same effect as the Focus module and is used to achieve 2x downsampling and reduce the number of parameters and calculations. Then, a 4-level feature extraction module is formed through 4 groups of CBS modules with a step size of 2 and C2fGhost modules to obtain P2-P5 respectively, among which P3 and P4 are connected using CBS to achieve cross-scale feature fusion and information transfer as the input of the feature fusion module. Finally, the deep feature P5 is input into the SPPF structure as the input of the feature fusion module.

[0084] S2.2, a set of features with the same feature map size output by the feature extraction network is defined as a layer. CSPDarkNet will generate 5 layers of features from shallow to deep from P1 to P5. YOLOv5 uses P3, P4, and P5 layer features for prediction by default. The shallow feature P2 is used instead of the deep feature P5 for prediction. The local receptive field of the shallow feature is smaller, and it contains more position and detail information, which can improve the performance of the model for small target detection.

[0085] S2.3, introduces Efficient-RepGFPN as a feature fusion network. The network takes the fused feature map as input, first fuses the input features through CBS and upsampling operations to obtain the first fused feature, transfers and interacts information between features at different levels, maps low-level features to the same scale as high-level features, and then connects with the original feature P4 after CBS processing to retain the information of the original feature. Then, the features are processed by Queen-Fusion and CBS to obtain the second fused features to enhance the expressiveness of the features. Subsequently, the detail information is restored through upsampling operations, and the upsampled features are connected to the features P3 to realize information transmission and interaction between the features. The output information is then passed through the first Queen-Fusion module to improve the resolution, and this is used as the first-layer output F1 to describe large-scale semantic information. At the same time, the output of the first Queen-Fusion module is input into the CBS module for processing and fused with the second fused features to retain the original information. Then, through the second Queen-Fusion module, the features at this time are used as the second-layer output F2 to describe the medium-scale semantic information; finally, the output of the second Queen-Fusion module is input into the CBS module for processing and fused with the first fused features, and then through the third Queen-Fusion module to obtain the third-layer output F3 to describe the small-scale semantic information.

[0086] The above is a feature fusion network, which can improve the performance of the model and improve the accuracy and generalization ability of target detection tasks.

[0087] Efficient-RepGFPN uses the CSP structure for feature fusion. On this basis, Efficient-RepGFPN upgrades the CSP structure through the connection of the re-parameterization mechanism (Re-param) and the efficient layer aggregation network (ELAN), achieving higher detection accuracy without incurring additional computational burden.

[0088] A shallow feature detection head is used to replace the deep feature detection head, the prediction layer is reduced to three layers, and the size of the convolution kernel is adjusted to adapt to the input of shallow features.

[0089] S2.4, the Wise-IoU V3 loss function is used to replace the original CIoU loss function as the positioning loss function. The Wise-IoU V3 loss calculation method is:

[0090]

[0091]

[0092]

[0093]

[0094]

[0095] Where IoU is the intersection over union ratio, is the IoU loss. (x, y) is the coordinate of the center point of the prediction box, (x gt ,y gt ) is the coordinate of the center point of the annotation box. g and H g is the width and height of the minimum enclosing box. β is the definition of outlier degree, is the IoU loss of the current image, is the mean IoU loss of the current batch of images. α and β are hyperparameters used to control the outlier degree and gradient gain. is the Wise-IoU penalty term, which can significantly amplify the IoU loss of ordinary quality anchor boxes. It is the Wise-IoU v1 loss.

[0096] S3: Train the object detection network, using the object detection loss function, and using mixed precision training, warm-up learning, cosine annealing learning rate, and early stopping training strategies;

[0097] Specifically, this implementation uses a learning rate of 0.01, a cosine annealing hyperparameter of 0.2, a learning rate momentum of 0.937, 3.0 warmup learning iterations, 0.8 warmup learning rate momentum, 0.1 warmup learning rate, 200 iterations, and a batch size of 32.

[0098] Object detection loss L Detection The calculation formula is as follows;

[0099]

[0100]

[0101] Among them, f obj is the confidence loss, f cl is the classification loss, f bb is the positioning loss, is the confidence, category probability and coordinate information of the true bounding box, They are the confidence, category probability and coordinate information of the target detection network predicting the bounding box, is the confidence value after processing with the sigmoid function, which is used as the weight of the positioning loss and classification loss;

[0102] The classification loss is the Wise-IoU V3 loss function, and the confidence loss f obj and classification loss f cl The calculation formula is:

[0103]

[0104]

[0105] Among them, N is the number of positive and negative samples, N pos is the number of positive samples, Represents the class probability of the positive sample.

[0106] S4: Use confidence scaling distillation to distill the model’s knowledge. The confidence scaling distillation process diagram is shown in the attached figure. Figure 5 shown.

[0107] Specifically, confidence scaling distillation can be divided into two parts: teacher network training and student network training. First, select a suitable teacher model for training on the dataset. This implementation uses the target detection network in step S2 as the teacher model. Then, preload the teacher model and train the student network. After each iterative training, the confidence distillation loss is calculated using the predictions of the teacher network and the student network. At this time, the prediction output of the teacher network has not been processed by maximum suppression, so it contains a large number of background area predictions. According to the confidence scaling distillation algorithm, the student network's learning of the background area can be effectively reduced, and the model convergence speed can be accelerated. At the same time, the prediction results of the target detection network and the annotation information of the dataset are used to calculate the target detection loss. Finally, the weighted sum of the distillation loss and the target detection loss is used to obtain the final loss of the student network.

[0108] Confidence-scaling distillation loss L Distill The calculation formula is as follows:

[0109]

[0110] Among them, f obj is the confidence loss, f cl is the classification loss, f bb is the positioning loss, The confidence, category probability and coordinate information of the bounding box predicted by the student network are respectively; is the confidence, category probability and coordinate information of the teacher network’s predicted bounding box; is the confidence value after processing with the sigmoid function. This value is used as the weight of the positioning loss and the classification loss. The calculation method is the same as formula (8);

[0111] Confidence loss f obj and classification loss f cl The calculation formula is:

[0112]

[0113]

[0114] Among them, N is the number of positive and negative samples, N pos is the number of positive samples; Represents the class probability of the positive sample.

[0115] The final loss of the student network is obtained by weighted summation of the distillation loss and the target detection loss. The calculation formula for the final loss is:

[0116]

[0117] Among them, λ D Represents weight.

[0118] S5: Use OpenVINO Post-Training Optimization Tool (POT) to INT8 quantize the model.

[0119] Specifically, to implement the quantization pipeline of the POT API, you need to implement a custom DataLoader subclass and a Metric subclass, and complete the configuration of the POT quantization parameters. First, this implementation defines the YOLOv5Dataloader subclass by inheriting the POT DataLoader base class. You need to call the create_dataloader method of YOLOv5 to read the dataset, and implement adaptive image scaling through the letterbox method. After that, this implementation inherits the POT Metric base class to implement a custom COCOMetric subclass. This class integrates the non-maximum suppression function and accuracy calculation method of YOLOv5 to calculate the mean average accuracy. Finally, running the POT quantization pipeline requires setting parameters such as model, engine, dataset, metric, and algorithms. This implementation selects the DQ quantization strategy to quickly obtain the model with the best performance. In addition, for the YOLOv5 model using non-ReLU activation, this implementation sets the preset parameter to symmetrically quantize the model weights and asymmetrically quantize the model activations, so as to better ensure the accuracy of the quantized model.

[0120] S6: Perform performance evaluation on the trained model.

[0121] Specifically, this implementation evaluates the target detection network model based on knowledge distillation training from evaluation indicators such as detection accuracy, recall rate, average detection precision mAP, detection speed FPS and model size.

[0122] The present invention also provides a traffic sign detection system based on YOLOv5, comprising the following modules:

[0123] The preprocessing module is used to prepare the data set and use the online and offline hybrid data enhancement method to preprocess the traffic sign detection data set to expand the data set and reduce the damage of Mosaic data enhancement to small target images;

[0124] The network construction module is used to build a traffic sign detection network based on YOLOv5 as the target detection network and improve the positioning loss function in the network;

[0125] The network training module is used to train the target detection network, adopt the target detection loss function, and use mixed precision training, warm-up learning, cosine annealing learning rate and early stopping training strategy;

[0126] The knowledge distillation module is used to perform knowledge distillation on the trained target detection network model using confidence scaling distillation loss, and finally combines the target detection loss function and the distillation loss function in a weighted manner to form the final loss function;

[0127] The quantization module is used to perform INT8 quantization on the model after knowledge distillation using the OpenVINO post-training optimization tool.

[0128] The specific implementation method of each module is the same as each step and will not be described in detail in the present invention.

[0129] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0130] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Therefore, if these modifications and variations of the embodiments of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A traffic sign detection method based on YOLOv5, characterized in that: The steps include: S1, prepare the data set, use the online and offline hybrid data enhancement method to preprocess the traffic sign detection data set to expand the data set and reduce the damage of Mosaic data enhancement to small target images; S2, build a traffic sign detection network based on YOLOv5 as the target detection network, and improve the positioning loss function in the network; The traffic sign detection network based on YOLOv5 in step S2 includes three parts: CSPDarkNet feature extraction network, feature fusion network and detection head. The specific implementation method is as follows: The structure of the CSPDarkNet feature extraction network is as follows: first, the input image is processed using the CBS module to obtain the shallow feature P1, then a 4-level feature extraction module is formed through 4 groups of CBS modules with a step size of 2 and C2fGhost modules, and the shallow feature P1 is further processed to obtain P2-P5 in sequence, among which P3 and P4 are connected using CBS to achieve cross-scale feature fusion and information transfer as the input of the feature fusion module, and at the same time, the deep feature P5 is input into the SPPF structure to obtain the fused input feature, which is also used as the input of the feature fusion module; Efficient-RepGFPN is introduced as a feature fusion network, and its processing process is as follows: first, the input features are fused through CBS and upsampling operations to obtain the first fused feature, the low-level features are mapped to the same scale as the high-level features, and then connected with the features of the original feature P4 after CBS processing to retain the information of the original features; then, the features are processed through Queen-Fusion and CBS to obtain the second fused feature to enhance the expression ability of the features, and then the detail information is restored through upsampling operations, and the upsampled features are connected with the feature P3 to realize information transmission and interaction between features, and then the output information is passed through the first Queen-Fusion. The en-Fusion module improves the resolution and uses it as the first-layer output F1 to describe large-scale semantic information. At the same time, the output of the first Queen-Fusion module is input into the CBS module for processing and fused with the second fusion feature to retain the original information. Then, through the second Queen-Fusion module, the feature at this time is used as the second-layer output F2 to describe the medium-scale semantic information; finally, the output of the second Queen-Fusion module is input into the CBS module for processing and fused with the first fusion feature, and then through the third Queen-Fusion module to obtain the third-layer output F3 to describe the small-scale semantic information; Input F1, F2 and F3 into the detection head respectively to obtain the detection results; S3, trains the object detection network, adopts the object detection loss function, and uses mixed precision training, warm-up learning, cosine annealing learning rate and early stopping training strategy; S4, using confidence scaling distillation loss to perform knowledge distillation on the trained target detection network model, and finally combining the target detection loss function and the distillation loss function in a weighted manner to form the final loss function; S5, use OpenVINO post-training optimization tool to perform INT8 quantization on the model after knowledge distillation.

2. A traffic sign detection method based on YOLOv5 as claimed in claim 1, characterized in that: The online and offline hybrid data enhancement method of step S1 includes the following steps: S1.1, firstly adopts the copy-paste offline data augmentation algorithm to paste based on the image context and scale information to achieve dataset expansion; S1.2, then the probability Mosaic online data enhancement algorithm is used to determine the probability of using Mosaic data enhancement according to the proportion of small-scale annotations in the image; the small-scale definition here uses the MS COCO dataset standard: objects with a resolution less than 32 pixels × 32 pixels are defined as small-scale annotations, and the probability calculation method used by Mosaic is: Where n S To process the number of small-scale annotations of the image, n αll The total number of annotations for processed images.

3. A traffic sign detection method based on YOLOv5 as claimed in claim 1, characterized in that: In step S2, the Wise-IoU V3 loss function is used to replace the original CIoU loss function as the positioning loss function. The Wise-IoU V3 loss calculation method is: Where IoU is the intersection over union ratio, is the IoU loss, (x, y) is the coordinate of the center point of the prediction box, (x gt ,y gt ) is the coordinate of the center point of the annotation box; W g and H g is the width and height of the minimum enclosing box, β is the definition of outlier degree, is the IoU loss of the current image, is the mean IoU loss of the current batch of images; α and β are hyperparameters used to control the outlier degree and gradient gain. is the Wise-IoU penalty term, which is used to significantly amplify the IoU loss of ordinary quality anchor boxes. It is the Wise-IoU v1 loss.

4. A traffic sign detection method based on YOLOv5 as claimed in claim 3, characterized in that: In step S3, the target detection loss L Detection The calculation formula is as follows; Among them, f obj is the confidence loss, f cl is the classification loss, f bb For the positioning loss, is the confidence, category probability and coordinate information of the true bounding box, They are the confidence, category probability and coordinate information of the target detection network predicting the bounding box, is the confidence value after processing with the sigmoid function, which is used as the weight of the positioning loss and classification loss; The classification loss is the Wise-IoU V3 loss function, and the confidence loss f obj and classification loss f cl The calculation formula is: Among them, N is the number of positive and negative samples, N pos is the number of positive samples, Represents the class probability of the positive sample.

5. A traffic sign detection method based on YOLOv5 as claimed in claim 4, characterized in that: In step S4, the target detection network in step S2 is used as a teacher network, the teacher network is used to infer the training data, the prediction result of the teacher network is obtained, the confidence difference between the target category and the background area is calculated, and then the output of the teacher network and the true label of the data are used to train the student network, wherein the network structure of the student network is the same as that of the teacher network; Confidence-scaling distillation loss L Distill The calculation formula is as follows: Among them, f obj is the confidence loss, f cl is the classification loss, f bb is the positioning loss, The confidence, category probability and coordinate information of the bounding box predicted by the student network are respectively; is the confidence, category probability and coordinate information of the teacher network’s predicted bounding box; is the confidence value after processing with the sigmoid function. This value is used as the weight of the positioning loss and the classification loss. The calculation method is the same as formula (8); Confidence loss f obj and classification loss f cl The calculation formula is: Among them, N is the number of positive and negative samples, N pos is the number of positive samples; Represents the class probability of the positive sample.

6. A traffic sign detection method based on YOLOv5 as claimed in claim 5, characterized in that: The final loss of the student network is obtained by weighted summation of the distillation loss and the target detection loss. The calculation formula for the final loss is: Among them, λ D Represents weight.

7. A traffic sign detection method based on YOLOv5 as claimed in claim 1, characterized in that: Step S5 specifically includes the following steps: S5.1, implement a custom YOLOv5Dataloader subclass by inheriting the POT DataLoader base class; S5.2, implement a custom COCOMetric subclass by inheriting the POT Metric base class; S5.3, use the DQ (Default Quantization) quantization strategy; perform symmetric quantization on model weights and asymmetric quantization on model activations.

8. A traffic sign detection method based on YOLOv5 as claimed in claim 1, characterized in that: The method also includes step S6, which performs a performance evaluation on the training model, and evaluates the model based on evaluation indicators such as detection accuracy, recall rate, average detection precision mAP, detection speed FPS, and model size.

9. A traffic sign detection system based on YOLOv5, characterized in that: Includes the following modules: The preprocessing module is used to prepare the data set and use the online and offline hybrid data enhancement method to preprocess the traffic sign detection data set to expand the data set and reduce the damage of Mosaic data enhancement to small target images; The network construction module is used to build a traffic sign detection network based on YOLOv5 as the target detection network and improve the positioning loss function in the network; The traffic sign detection network based on YOLOv5 includes three parts: CSPDarkNet feature extraction network, feature fusion network and detection head. The specific implementation method is as follows: The structure of the CSPDarkNet feature extraction network is as follows: first, the input image is processed using the CBS module to obtain the shallow feature P1, then a 4-level feature extraction module is formed through 4 groups of CBS modules with a step size of 2 and C2fGhost modules, and the shallow feature P1 is further processed to obtain P2-P5 in sequence, among which P3 and P4 are connected using CBS to achieve cross-scale feature fusion and information transfer as the input of the feature fusion module, and at the same time, the deep feature P5 is input into the SPPF structure to obtain the fused input feature, which is also used as the input of the feature fusion module; Efficient-RepGFPN is introduced as a feature fusion network, and its processing process is as follows: first, the input features are fused through CBS and upsampling operations to obtain the first fused feature, the low-level features are mapped to the same scale as the high-level features, and then connected with the features of the original feature P4 after CBS processing to retain the information of the original features; then, the features are processed through Queen-Fusion and CBS to obtain the second fused feature to enhance the expression ability of the features, and then the detail information is restored through upsampling operations, and the upsampled features are connected with the feature P3 to realize information transmission and interaction between features, and then the output information is passed through the first Queen-Fusion. The en-Fusion module improves the resolution and uses it as the first-layer output F1 to describe large-scale semantic information. At the same time, the output of the first Queen-Fusion module is input into the CBS module for processing and fused with the second fusion feature to retain the original information. Then, through the second Queen-Fusion module, the feature at this time is used as the second-layer output F2 to describe the medium-scale semantic information; finally, the output of the second Queen-Fusion module is input into the CBS module for processing and fused with the first fusion feature, and then through the third Queen-Fusion module to obtain the third-layer output F3 to describe the small-scale semantic information; Input F1, F2 and F3 into the detection head respectively to obtain the detection results; The network training module is used to train the target detection network, adopt the target detection loss function, and use mixed precision training, warm-up learning, cosine annealing learning rate and early stopping training strategy; The knowledge distillation module is used to perform knowledge distillation on the trained target detection network model using confidence scaling distillation loss, and finally combines the target detection loss function and the distillation loss function in a weighted manner to form the final loss function; The quantization module is used to perform INT8 quantization on the model after knowledge distillation using the OpenVINO post-training optimization tool.

Images