Dynamic monitoring light-weight signal lamp detection method, system and equipment and storage medium

Abstract

The invention discloses a dynamic monitoring light-weight signal lamp detection method, system and device and a storage medium, and the method comprises the steps: obtaining a traffic signal lamp video stream data set G1, extracting a key frame from the G1, and generating a traffic signal lamp video stream key frame data set G2; an improved model of the YOLOv8 is constructed, and a public data set LISA is used for training; replacing the loss function of the YOLOv8 by using a self-defined SMIoU loss function to obtain a YOLOv8-SCP model, and training the YOLOv8-SCP by using a public data set LISA to determine model parameters; inputting the traffic signal lamp key frame data set G2 into the YOLOv8-SCP model, and obtaining a traffic signal lamp prediction data set R1 through detection; defining a threshold CL as a traffic light confidence threshold; judging whether a video stream key frame in the traffic signal lamp prediction data set R1 is a signal lamp or not according to a threshold CL, and obtaining a key frame data set with R2 being the signal lamp; according to the detection method provided by the invention, the detection performance is obviously optimized in traffic signal lamp detection.

Description

Technical Field

[0001] The present invention relates to a lightweight signal light detection method, system, device and storage medium, and in particular to a lightweight signal light detection method, system, device and storage medium for dynamic monitoring, belonging to the technical field of target detection. Background Art

[0002] Traffic light detection has significant research value in the field of modern traffic management and public safety. The system aims to improve the safety and smoothness of urban traffic and provide important support for public travel safety and social order maintenance. On the one hand, traffic light detection is a key link in ensuring road traffic safety. By accurately and quickly identifying the status of traffic lights on the road, it can provide drivers with instant and accurate driving guidance, thereby effectively avoiding traffic accidents caused by violating traffic light instructions and protecting the safety of pedestrians and vehicles. On the other hand, traffic light detection is crucial to improving traffic efficiency. In the intelligent transportation system, traffic light detection technology can assist the traffic management center to grasp the road conditions in real time, optimize the timing plan of traffic lights, reduce vehicle waiting time, and improve road traffic capacity, thereby alleviating urban traffic congestion. In addition, with the rapid development of autonomous driving technology, traffic light detection has become one of the necessary conditions for autonomous driving vehicles to achieve safe and autonomous driving, laying a solid foundation for the commercial application of autonomous driving technology.

[0003] YOLOv8 is a YOLO model launched by Ultralytics in 2023. It is an end-to-end target detection algorithm. The YOLOv8 algorithm model structure is mainly composed of the input, the backbone network and the head, of which the head includes the neck network and the detection head. In the input part, the functions include mosaic data enhancement, adaptive anchor calculation and other data enhancement technologies: in the backbone network part, the C2F module replaces the C3 module with the ELAN (Efficient Layer Aggregation Networks) idea of ​​YOLOv7. The final SPPF (Spatial Pyramid Pooling Fusion) module is composed of a convolution block and a maximum pooling layer. These improvements enable YOLOv8 to achieve faster detection speed, higher accuracy and ease of use, making it suitable for a variety of tasks including target detection, tracking, instance segmentation, image classification and posture estimation.

[0004] The SE attention mechanism, full name Squeeze-and-Excitation Attention, is an attention mechanism based on feature channels, which is widely used in the field of deep learning, especially in natural language processing and computer vision tasks. Its core idea is to dynamically adjust the weight of feature maps by learning the relationship between channels, thereby improving the model's attention to important features. Specifically, the SE attention mechanism includes two key steps: Squeeze and Excitation.

[0005] The P2 small target detection layer is an important component of the target detection network, especially when dealing with small target detection tasks. This detection layer is usually located in the shallower layer of the network structure, which can capture more detailed and rich feature information in the image, which is crucial for detecting small targets with small size and low resolution. In recent years, deep learning has been increasingly used in the field of traffic light detection. These methods have shown excellent detection performance in different traffic scenarios with their powerful feature extraction capabilities. The target detection algorithms based on deep learning can be roughly divided into two categories, one is the two-stage algorithm based on candidate regions, and the other is the one-stage algorithm based on regression. The former is mainly represented by the R-CNN series, and the latter is mainly represented by YOLO, CenterNet and SSD. Compared with the two-stage algorithm, the one-stage algorithm is efficient and can meet the real-time requirements, but the accuracy is not as good as the two-stage algorithm. In the field of traffic light detection, urban traffic is usually complex and most vehicles are traveling at a high speed, so the requirements for the real-time and detection accuracy of signal light detection are relatively high. However, the current signal light detection algorithm has problems such as false detection and missed detection, low detection rate and low detection accuracy. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a lightweight signal light detection method, system, device and storage medium for dynamic monitoring that can improve detection accuracy.

[0007] Technical solution: A dynamic monitoring lightweight signal light detection method according to the present invention comprises:

[0008] Step 1: Obtain the traffic light video stream dataset G1, extract key frames from G1, and generate the traffic light video stream key frame dataset G2;

[0009] Step 2: Build an improved model of YOLOv8 and use the public dataset LISA for training; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; use a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and add an SE attention mechanism module; add a P2 small target detection layer in the detection head;

[0010] Step 3: Use the custom SMIoU loss function to replace the YOLOv8 loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters;

[0011] Step 4: Input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection;

[0012] Step 5: Define the threshold CL as the traffic light confidence threshold; determine the key frames of the video stream that are greater than or equal to the threshold CL in the traffic light prediction data set R1 as traffic lights, and obtain R2 as the key frame data set of traffic lights.

[0013] Furthermore, the customized SPSFNet network structure described in step 2 is as follows:

[0014] Use a Conv_maxpool layer that combines a convolutional layer and a maximum pooling layer, with a kernel size of 3, a stride of 2, and 24 output channels, and then connect six ShuffleNetV2 modules and one SPPELAN module in series;

[0015] The ShuffleNetV2 module uses a channel split operation to change the number of output channels C2 to half of the number of input channels C1. The structure of one branch remains unchanged, and the other branch is composed of a 1×1 convolution, a 3×3 depth-separable convolution, and a 1×1 convolution in series. The two branches are then connected and a channel shuffle operation is performed.

[0016] The SPPELAN module consists of a 1×1 convolution module that converts the number of output channels C4 into half of the number of input channels C3, and then connects three MaxPool modules in series. Each MaxPool module and the convolution module that converts the number of output channels C6 into half of the number of input channels C5 are bitwise added, and finally a convolution operation is performed with the 1×1 convolution module.

[0017] Furthermore, the customized DWCFaster module described in step 2 is composed of a C2f-Faster module and a DWConv module. The improved C2f-Faster module is used to replace the C2f module in the YOLOv8 neck, and the DWConv module is used to replace the Conv module in the neck network.

[0018] The C2f-Faster module is composed of a Split module that converts the number of output channels C10 to half of the number of input channels C9, and then connects multiple FasterBlock modules in series. Each FasterBlock module and a convolution module that converts the number of output channels C12 to half of the number of input channels C11 perform channel fusion, and finally perform a convolution operation with a 1×1 convolution module.

[0019] The DwConv module is composed of six modules connected in series: a 3×3 depth convolution, a normalized BN module, a ReLU activation function, a 1×1 point-by-point convolution, a normalized BN module and a ReLU activation function.

[0020] Furthermore, the FasterBlock module uses partial convolution PConv to change the number of output channels C8 to half of the number of input channels C7. The structure of one branch remains unchanged, and the other branch is composed of two 1×1 convolutions connected in series, and then the two branches are added point by point.

[0021] Furthermore, the SE attention mechanism module is added as described in step 2, specifically, the number of output channels is changed to half of the number of input channels through the SE attention mechanism module, one branch structure remains unchanged, and the other branch is composed of a global average pooling, a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid function in series, and then the two branches are multiplied point by point and finally output.

[0022] Furthermore, the SMIoU loss function formula in step 3 is:

[0023]

[0024] Among them, IoU represents the intersection-over-union ratio of the real box and the predicted box, Δ represents the distance consumption between the real box and the predicted box, and Ω represents the shape consumption of the predicted box. Represents the sum of squared differences between the coordinates of the upper left corner of the predicted box and the true box. It represents the sum of square differences between the coordinates of the lower right corner of the predicted box and the true box, ω represents the width of the input image, h represents the height of the input image, and the value range of α and β is [0,1], and α+β=1.

[0025] Furthermore, in step 3, when training YOLOv8-SCP using the public data set LISA to determine the model parameters, the best training weight best.pt is obtained when the public data set LISA is used for training. The best training weight best.pt is used to verify the validation set of the public data set LISA, and then the model performance is evaluated. The evaluation content is as follows:

[0026] Precision rate P:

[0027]

[0028] Recall R:

[0029]

[0030] Average precision mAP:

[0031]

[0032] Among them, TP represents the number of positive samples correctly predicted by the model; FP represents the number of parent classes incorrectly predicted as positive by the model; FN represents the number of positive classes incorrectly predicted as negative by the model; u represents the total number of categories, s represents the index of the currently calculated category, and AP represents the average precision.

[0033] Based on the same inventive concept, the present invention also provides a dynamic monitoring lightweight signal light detection system, comprising:

[0034] An initialization module is used to obtain a traffic light video stream data set G1, extract key frames from G1, and generate a traffic light video stream key frame data set G2;

[0035] The model improvement module is used to build an improved model of YOLOv8 and train it using the public dataset LISA; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; uses a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and adds an SE attention mechanism module; adds a P2 small target detection layer in the detection head;

[0036] The model building module is used to replace the loss function of YOLOv8 with the customized SMIoU loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters;

[0037] A detection module is used to input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection;

[0038] The judgment module is used to define a threshold CL as a traffic light confidence threshold; the key frames of the video stream greater than or equal to the threshold CL in the traffic light prediction data set R1 are judged as traffic lights, and R2 is obtained as a key frame data set of traffic lights.

[0039] Based on the same inventive concept, the present invention also provides a computing device, comprising: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the dynamic monitoring lightweight traffic light detection method according to any one of the above items are implemented.

[0040] Based on the same inventive concept, the present invention also provides a storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the dynamic monitoring lightweight traffic light detection method described in any one of the above items.

[0041] Beneficial effects: Compared with the prior art, the present invention uses a customized SPSFNet network, namely SPPELANShuffleNetV2, to replace the backbone network Backbone in YOLOv8, while maintaining the computational efficiency and improving the accuracy of the model; secondly, a customized DWCFaster module, namely DWConv FasterBlock is used Bottleneck replaces the C2f and Conv modules of the YOLOv8 neck and adds the SE attention mechanism module to reduce the number of parameters; then, a P2 small target detection layer is added to the detection head to improve the model's perception of small targets and enhance the accuracy of detection; finally, a customized SPSFNet network, DWCFaster module, SE attention mechanism module and P2 small target detection layer are used to improve YOLOv8, and it is defined as a YOLOv8-SCP model. The public dataset LISA is used to train the YOLOv8-SCP device to determine the model parameters, and finally the traffic light detection model of the improved YOLOv8 algorithm is obtained; while ensuring the reduction of parameters, the recall rate of the YOLOv8 algorithm is improved, and a performance improvement of 4% average precision mean and 8.1% accuracy is obtained in the traffic light detection application, which can be used as a technical support for the construction of smart cities and can effectively promote the intelligence and refinement of urban traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flow chart of a method according to an embodiment of the present invention;

[0043] Figure 2 It is a YOLOv8-SCP model structure diagram of an embodiment of the present invention;

[0044] Figure 3 This is a structural diagram of ShuffleNetV2 in an embodiment of the present invention;

[0045] Figure 4 It is a SPPELAN structure diagram of an embodiment of the present invention;

[0046] Figure 5 It is a C2f_Faster structure diagram of an embodiment of the present invention;

[0047] Figure 6 This is a structural diagram of DwConv according to an embodiment of the present invention;

[0048] Figure 7 This is a structural diagram of the SE attention mechanism of an embodiment of the present invention;

[0049] Figure 8 A case comparison diagram of the detection results of an embodiment of the present invention;

[0050] Fig. 9 It is a parameter diagram of YOLOv8m in the performance parameter comparison of an embodiment of the present invention;

[0051] Fig.10 It is a parameter diagram of YOLOv8-SCP in the performance parameter comparison of an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0053] As attached Figure 1 As shown, the dynamic monitoring lightweight traffic light detection method of this embodiment obtains images or video streams and inputs them into the YOLOv8-SCP network model respectively; including:

[0054] Step 1: Obtain the traffic light video stream dataset G1, extract key frames from G1, and generate the traffic light video stream key frame dataset G2;

[0055] Step 2: Build an improved model of YOLOv8 and use the public dataset LISA for training; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; use a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and add an SE attention mechanism module; add a P2 small target detection layer in the detection head;

[0056] Step 3: Use the custom SMIoU loss function to replace the YOLOv8 loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters;

[0057] Step 4: Input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection;

[0058] Step 5: Define the threshold CL as the traffic light confidence threshold; determine the key frames of the video stream that are greater than or equal to the threshold CL in the traffic light prediction data set R1 as traffic lights, and obtain R2 as the key frame data set of traffic lights.

[0059] The present invention makes up for the shortcomings of the current traffic light detection technology and solves the problems of missed detection and false detection of traffic lights. In the present invention, a customized SPSFNet network, a customized DWCFaster module, an SE attention mechanism module and a P2 small target detection layer are added to build a traffic light intelligent detection network model of the YOLOv8-SCP device, that is, Figure 3 to Figure 7 shown.

[0060] As an efficient single-stage target detection algorithm, YOLOv8 is famous for its more tightly connected convolutional network and more optimized adaptive anchor frame. It covers five models with different configurations, from lightweight YOLOv8n to high-performance YOLOv8x, to meet the processing requirements of data sets of different sizes. The present invention selects YOLOv8m as the baseline model. The overall architecture of YOLOv8m is exquisitely designed and consists of four core modules: first, the Input input, which is responsible for receiving and processing image data; second, the Backbone backbone network, as the core of feature extraction, its depth and complexity lay a solid foundation for subsequent detection tasks; then the Neck structure, which improves the model's ability to understand complex scenes through feature fusion and enhancement; finally, the Head output layer, which is responsible for generating the final detection results. For the common challenges of occlusion and missed detection in traffic light detection, the present invention has made targeted improvements on the basis of YOLOv8m to obtain the YOLOv8-SCP network model, such as Figure 2 shown.

[0061] Use a custom SPSFNet network, i.e., SPPELAN ShuffleNetV2, to replace the backbone network Backbone in YOLOv8, i.e., layers 0, 1, 2, 3, 4, 5, 6, and 7. The SPSFNet network is as follows:

[0062] Define the ShuffleNetV2 module and use channel splitting, i.e., Channel split operation, to change the number of output channels C2 to half of the number of input channels C1. The structure of one branch remains unchanged, and the other branch consists of a 1×1 convolution, a 3×3 depth-separable convolution, and a 1×1 convolution in series. Then the two branches are connected and finally the channel shuffle is performed, i.e., Channel Shuffle operation.

[0063] The SPPELAN module is defined as a 1×1 convolution module that changes the number of output channels C4 to half of the number of input channels C3, and then three MaxPool modules are connected in series. Each MaxPool module and the convolution module that changes the number of output channels C6 to half of the number of input channels C5 are bitwise added, and finally convolved with the 1×1 convolution module;

[0064] The SPSFNet network is defined using a Conv_maxpool layer that combines a convolutional layer and a maximum pooling layer, with a convolution kernel size of 3, a step size of 2, and an output channel number of 24. It is then composed of six ShuffleNetV2 modules and one SPPELAN module connected in series. The ShuffleNetV2 module is used multiple times, increasing the complexity and computational complexity of the model.

[0065] Use a custom DWCFaster module, i.e., DWConv FasterBlock Bottleneck, to replace the C2f and Conv modules of the YOLOv8 neck, and introduce the SE attention mechanism module and the P2 small target detection layer, i.e., layers 10, 13, 16, 17, 19, 20, 22, 23, 25, 26, and 27. The DWCFaster module is as follows:

[0066] Define the FasterBlock module and use partial convolution, i.e., PConv, to reduce the number of output channels C8 to half of the number of input channels C7. The structure of one branch remains unchanged, and the other branch consists of two 1×1 convolutions in series. Then, the two branches are added point by point.

[0067] The C2f-Faster module is defined as a Split module that changes the number of output channels C10 to half of the number of input channels C9, and then multiple FasterBlock modules are connected in series. Channel fusion is performed between each FasterBlock module and the convolution module that changes the number of output channels C12 to half of the number of input channels C11, and finally a convolution operation is performed with the 1×1 convolution module.

[0068] The definition of the DwConv module is composed of six modules in series: a 3×3 depth convolution, a normalized BN module, a ReLU activation function, a 1×1 point-by-point convolution, a normalized BN module, and a ReLU activation function.

[0069] Define the DWCFaster module to consist of a C2f-Faster module and a DWConv module. Use the improved C2f-Faster module to replace the C2f module in the YOLOv8 neck, and use the DWConv module to replace the Conv module in the neck network.

[0070] Define the SMIoU loss function, its formula is:

[0071]

[0072] Among them, IoU represents the intersection-over-union ratio of the real box and the predicted box, Δ represents the distance consumption between the real box and the predicted box, and Ω represents the shape consumption of the predicted box. Represents the sum of squared differences between the coordinates of the upper left corner of the predicted box and the true box. It represents the sum of square differences between the coordinates of the lower right corner of the predicted box and the true box, ω represents the width of the input image, h represents the height of the input image, and the value range of α and β is [0,1], and α+β=1.

[0073] In order to further prove the effectiveness of the traffic light detection algorithm, different target detection algorithms were selected for comparative experiments. During the experiment, all methods were conducted on the same data set, and the training set, validation set, and test set were divided in the same way. The experimental hardware and software environment was also the same. It was compared with five models, SSD, Faster-RCNN, CenterNet, GHostNet, and efficientDET, on the public data set LISA. The experiment followed the principle of controlling variables, and the experimental facilities were kept consistent. The comparative experimental results are shown in Table 1.

[0074] Table 1 Comparison of model detection results

[0075]

[0076] According to Table 1, on the experimental data set, the improved model has an 8.1% higher precision, a 2% higher recall, a 4% higher mAP, and a 51.2% lower parameter count than YOLOv8; compared with SSD, its mAP is 43.4% higher, and the parameter count is 52.1% lower; compared with Faster-RCNN, its precision is 36% higher, and its mAP is 42.3% higher; compared with CenterNet, its mAP is 27.1% higher, and the computational complexity is 33.5% lower. Compared with GHostNet, its recall is 41.6% higher, and its mAP is 43.4% higher. In summary, the improved model has better detection performance for traffic light detection.

[0077] Related vocabulary or variable description:

[0078] Vocabulary or variable Explanation Import Import, input picture Conv_maxpool Convolution maxpooling, convolution and maximum pooling function SPPELAN Spatial Pyramid Pooling ELAN, spatial pyramid pooling Concat Concatenate Upsample Upsample SEAttention Squeeze-and-Excitation Attention, attention mechanism Detect Detection Layer PConv Partial Convolution Conv Convolution BN Batch Normalization ReLU Rectified Linear Unit, activation function Channel Split Channel Split DWConv Depthwise Separable Convolution, Depthwise Separable Convolution Channel Shuffle Channel Shuffle MaxPool MaxPooling, maximum pooling function Depthwise Conv Depthwise Convolution Global Average Pooling Global Average Pooling FC Fully Connected layer

[0079] An embodiment is used to determine the optimal parameters of the YOLOv8-SCP detection model for detecting traffic light data problems. The embodiment provides a training set, a validation set, a test set, and a training method for the model.

[0080] In this embodiment, a traffic light video stream dataset G1 is used as a research object, and key frame processing is performed on the video stream data to obtain a traffic light key frame dataset G2. By using the LabelImg data annotation tool, each image data in the traffic light key frame dataset G2 is labeled, and three categories Class are defined, which are set as Class = {0: tl_green, 1: tl_red, 2: tl_yellow}, where 0: tl_green represents the number and green light, 1: tl_red represents the number and red light, and 2: tl_yellow represents the number and yellow light. Each object is given a clear label information and a set of precise location information, which is intuitively displayed in the image in the form of a label box. After the labeling is completed, the label data of the traffic lights in each image is saved in a structured form in a txt file that matches the image name, and the traffic light label data set L1 is obtained, which consists of the category number, the x-axis coordinate of the center point, the y-axis coordinate of the center point, the width and height; the traffic light key frame data set G2 is divided into a training set V1, a validation set V2 and a test set V3 in a ratio of 8:1:1, where the training set V1 has 6400 data, the validation set V2 has 800 data, and the test set V3 has 800 data. For comparative experiments and parameter verification, the number of training rounds is set to 100, the batch size is 8, the initial learning rate is 0.0001, and the weight decay coefficient is 0.0001.

[0081] The test results of this embodiment, i.e., the performance parameter comparison diagram, are shown in FIG. Figures 8 to 10shown.

[0082] The divided training set V1 and validation set V2 are input into YOLOv8-SCP and YOLOv8 for training, and the best weights in the training process are selected. The best weights of the YOLOv8-SCP model are defined as P1 and the best weights of the YOLOv8 model are defined as P2.

[0083] The evaluation indicators of model performance include detection speed and detection accuracy. The embodiment uses a comprehensive evaluation index system, including precision P, recall R and average precision mAP50. Among them, the precision P predicts the proportion of positive sample instances that meet the positive sample conditions, the recall R is the proportion of accurate predictions in all positive categories, and the calculation of mAP is based on the average precision (AP) of each category, and then the average AP of all categories is obtained. It is a comprehensive and objective indicator for measuring the performance of the target detection model. The specific formula is as follows:

[0084] Precision rate P:

[0085]

[0086] Recall R:

[0087]

[0088] Average precision mAP:

[0089]

[0090] A custom SPSFNet network, a custom DWCFaster module, a SE attention mechanism module and a P2 small target detection layer are added to YOLOv8. It is verified that the YOLOv8-SCP model has a significant performance improvement effect. The method proposed in the present invention reduces the number of parameters of the YOLOv8 algorithm while ensuring the improvement of the accuracy of the algorithm. In the application of traffic light detection, a performance improvement of 4% average precision mean and 8.1% accuracy is achieved, which can be used as a technical support for the construction of smart cities and can effectively promote the intelligence and refinement of urban traffic management.

[0091] Based on the same inventive concept, this embodiment also provides a dynamic monitoring lightweight signal light detection system, including:

[0092] An initialization module is used to obtain a traffic light video stream data set G1, extract key frames from G1, and generate a traffic light video stream key frame data set G2;

[0093] The model improvement module is used to build an improved model of YOLOv8 and train it using the public dataset LISA; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; uses a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and adds an SE attention mechanism module; adds a P2 small target detection layer in the detection head;

[0094] The model building module is used to replace the loss function of YOLOv8 with the customized SMIoU loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters;

[0095] A detection module is used to input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection;

[0096] The judgment module is used to define a threshold CL as a traffic light confidence threshold; the key frames of the video stream greater than or equal to the threshold CL in the traffic light prediction data set R1 are judged as traffic lights, and R2 is obtained as a key frame data set of traffic lights.

[0097] Based on the same inventive concept, this embodiment also provides a computing device, including: one or more processors, one or more memories and one or more programs, wherein the programs are stored in the memories and configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the dynamic monitoring lightweight traffic light detection method according to any one of the above items are implemented.

[0098] Based on the same inventive concept, this embodiment also provides a storage medium, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the steps of the lightweight traffic light detection method based on dynamic monitoring of neural networks as described in any one of the above items.

[0099] The various technologies of the embodiments allow for flexible combination. In fact, if these combinations are logically consistent, combinations not directly listed in the embodiments should also be considered as implicitly covered by this specification, which is intended to maintain the simplicity of the description while ensuring the completeness of the scope.

Claims

1. A dynamic monitoring lightweight signal light detection method, characterized in that: include: Step 1: Obtain the traffic light video stream dataset G1, extract key frames from G1, and generate the traffic light video stream key frame dataset G2; Step 2: Build an improved model of YOLOv8 and use the public dataset LISA for training; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; use a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and add an SE attention mechanism module; add a P2 small target detection layer in the detection head; Step 3: Use the custom SMIoU loss function to replace the YOLOv8 loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters; Step 4: Input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection; Step 5: Define the threshold CL as the traffic light confidence threshold; determine the key frames of the video stream that are greater than or equal to the threshold CL in the traffic light prediction data set R1 as traffic lights, and obtain R2 as the key frame data set of traffic lights.

2. The dynamic monitoring lightweight signal light detection method according to claim 1, characterized in that: The customized SPSFNet network structure described in step 2 is as follows: Use a Conv_maxpool layer that combines a convolutional layer and a maximum pooling layer, with a kernel size of 3, a stride of 2, and 24 output channels, and then connect six ShuffleNetV2 modules and one SPPELAN module in series; The ShuffleNetV2 module uses a channel split operation to change the number of output channels C2 to half of the number of input channels C1. The structure of one branch remains unchanged, and the other branch is composed of a 1×1 convolution, a 3×3 depth-separable convolution, and a 1×1 convolution in series. The two branches are then connected and a channel shuffle operation is performed. The SPPELAN module consists of a 1×1 convolution module that converts the number of output channels C4 into half of the number of input channels C3, and then connects three MaxPool modules in series. Each MaxPool module and the convolution module that converts the number of output channels C6 into half of the number of input channels C5 are bitwise added, and finally a convolution operation is performed with the 1×1 convolution module.

3. The dynamic monitoring lightweight signal light detection method according to claim 1, characterized in that: The customized DWCFaster module described in step 2 is composed of a C2f-Faster module and a DWConv module. The improved C2f-Faster module is used to replace the C2f module in the YOLOv8 neck, and the DWConv module is used to replace the Conv module in the neck network. The C2f-Faster module is composed of a Split module that converts the number of output channels C10 to half of the number of input channels C9, and then connects multiple FasterBlock modules in series. Each FasterBlock module and a convolution module that converts the number of output channels C12 to half of the number of input channels C11 perform channel fusion, and finally perform a convolution operation with a 1×1 convolution module. The DwConv module is composed of six modules connected in series: a 3×3 depth convolution, a normalized BN module, a ReLU activation function, a 1×1 point-by-point convolution, a normalized BN module and a ReLU activation function.

4. The dynamic monitoring lightweight signal light detection method according to claim 3 is characterized in that: The FasterBlock module uses partial convolution PConv to change the number of output channels C8 to half of the number of input channels C7. The structure of one branch remains unchanged, and the other branch is composed of two 1×1 convolutions connected in series. Then, the two branches are added point by point.

5. The dynamic monitoring lightweight signal light detection method according to claim 1, characterized in that: Add the SE attention mechanism module as described in step 2. Specifically, the number of output channels is changed to half of the number of input channels through the SE attention mechanism module. One branch structure remains unchanged, and the other branch is composed of a global average pooling, a fully connected layer, a ReLU activation function, a fully connected layer and a Sigmoid function in series. Then the two branches are multiplied point by point and finally output.

6. The dynamic monitoring lightweight signal light detection method according to claim 1, characterized in that: The SMIoU loss function formula in step 3 is: Among them, IoU represents the intersection-over-union ratio of the real box and the predicted box, Δ represents the distance consumption between the real box and the predicted box, and Ω represents the shape consumption of the predicted box. Represents the sum of squared differences between the coordinates of the upper left corner of the predicted box and the true box. It represents the sum of square differences between the coordinates of the lower right corner of the predicted box and the true box, ω represents the width of the input image, h represents the height of the input image, and the value range of α and β is [0,1], and α+β=1.

7. The dynamic monitoring lightweight signal light detection method according to claim 1, characterized in that: In step 3, when training YOLOv8-SCP using the public dataset LISA to determine the model parameters, the best training weight best.pt is obtained when the public dataset LISA is used for training. The best training weight best.pt is used to verify the validation set of the public dataset LISA, and then the model performance is evaluated. The evaluation content is as follows: Precision rate P: Recall R: Average precision mAP: Among them, TP represents the number of positive samples correctly predicted by the model; FP represents the number of parent classes incorrectly predicted as positive by the model; FN represents the number of positive classes incorrectly predicted as negative by the model; u represents the total number of categories, s represents the index of the currently calculated category, and AP represents the average precision.

8. A dynamic monitoring lightweight signal light detection system, characterized in that: include: An initialization module is used to obtain a traffic light video stream data set G1, extract key frames from G1, and generate a traffic light video stream key frame data set G2; The model improvement module is used to build an improved model of YOLOv8 and train it using the public dataset LISA; the improved YOLOv8 detection model uses a custom SPSFNet network to replace the backbone network Backbone in YOLOv8; uses a custom DWCFaster module to replace the C2f and Conv modules of the YOLOv8 neck, and adds an SE attention mechanism module; adds a P2 small target detection layer in the detection head; The model building module is used to replace the loss function of YOLOv8 with the customized SMIoU loss function to obtain the YOLOv8-SCP model, and use the public dataset LISA to train YOLOv8-SCP to determine the model parameters; A detection module is used to input the traffic light key frame dataset G2 into the YOLOv8-SCP model, and obtain the traffic light prediction dataset R1 after detection; The judgment module is used to define a threshold CL as a traffic light confidence threshold; the key frames of the video stream greater than or equal to the threshold CL in the traffic light prediction data set R1 are judged as traffic lights, and R2 is obtained as a key frame data set of traffic lights.

9. A computing device, characterized in that include: One or more processors, one or more memories, and one or more programs, wherein the programs are stored in the memories and are configured to be executed by the processors, and when the programs are loaded into the processors, the steps of the dynamic monitoring lightweight signal light detection method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: The computer program comprises program instructions, which, when executed by a processor, enable the processor to perform the steps of the dynamic monitoring lightweight signal lamp detection method according to any one of claims 1 to 7.

Images