Traffic light fault real-time identification and alarm method based on improved Yolov5

Abstract

The invention discloses a traffic light fault real-time identification and alarm method based on improved Yolov5, and relates to the technical field of intelligent traffic, and the method comprises the steps: S1, building a traffic light fault data set in advance, and collecting traffic light images and video data under different scenes, illumination conditions and weathers; step S2, constructing an improved Yolov5 model; and S3, extracting images of a real-time video stream to be detected according to frames, inputting the images into the trained improved Yolov5 model, outputting position information, light color states and fault probabilities of traffic lights, and analyzing and judging fault types in combination with a preset threshold value and a multi-frame time sequence. Real-time video stream processing and multi-frame time sequence analysis are combined, the high-frame-rate real-time detection requirement can be met, time sequence faults can be accurately judged, and the limitation that manual inspection is low in efficiency and poor in real-time performance is broken through; the fault response and handling period is greatly shortened, and powerful technical support is provided for guaranteeing normal operation of the traffic lights and improving urban traffic management efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and more specifically, to a method for real-time identification and alarm of traffic light malfunctions based on an improved YOLOv5. Background Technology

[0002] Traffic lights, as core facilities for urban traffic management, play a vital role in ensuring road traffic order and reducing traffic accidents when operating normally. However, due to factors such as equipment aging, inclement weather, power outages, and poor wiring connections, traffic lights often experience abnormal illumination, such as multiple lights being on simultaneously, lights remaining off continuously, color deviations (e.g., red light color cast, insufficient green light brightness), and timing errors (e.g., incorrect signal switching logic).

[0003] Traditional traffic light fault detection mainly relies on manual inspection, which suffers from low efficiency, high cost, and poor real-time performance, making it difficult to meet the needs of refined urban traffic management. While existing computer vision-based recognition methods have been applied to traffic scenarios, they still have the following shortcomings in traffic light fault identification: First, general target detection models lack accuracy in recognizing small target traffic lights, especially in complex backgrounds such as at night or in rain, snow, or fog, leading to missed detections; second, there is a lack of dedicated feature extraction mechanisms designed for specific fault types of traffic lights, making it difficult to distinguish between normal and fault states; and third, there is a lack of coordinated design between real-time performance and alarm mechanisms, hindering rapid response and handling of faults.

[0004] Therefore, developing a traffic light fault identification and alarm method that combines high precision, strong robustness, and real-time performance has become an urgent technical problem to be solved in this field. Summary of the Invention

[0005] To address the problems in related technologies, this invention proposes a real-time traffic light fault identification and alarm method based on an improved YOLOv5, in order to improve the accuracy and efficiency of traffic light fault identification and achieve rapid fault detection and alarm.

[0006] The technical solution of this invention is implemented as follows:

[0007] A real-time fault identification and alarm method for traffic lights based on an improved Yolov5 system includes the following steps:

[0008] Step S1: Pre-construct a traffic light fault dataset, including collecting traffic light images and video data under different scenarios, lighting conditions and weather conditions, labeling the data and performing data augmentation processing;

[0009] Step S2: Construct an improved Yolov5 model and divide the dataset processed in Step S1 into training, validation, and test sets. Input the dataset into the improved Yolov5 model for training. Monitor the performance using the validation set and adopt an early stopping strategy. Evaluate the model performance using the test set to obtain the trained improved Yolov5 model. The improved Yolov5 model includes introducing an attention mechanism module at the input of the backbone network based on the Yolov5 model, adding a small object detection branch in the feature fusion network, and calibrating the loss function.

[0010] Step S3: Extract images from the real-time video stream to be detected frame by frame, input them into the trained improved Yolov5 model, and output the location information, color status and fault probability of the traffic lights. Combine the preset threshold and multi-frame time sequence analysis to determine the fault type.

[0011] Step S4: When a traffic light malfunction is detected, an alarm mechanism is automatically triggered to generate an alarm message containing fault information and push it to the warning terminal, while storing the alarm record.

[0012] Furthermore, the fault states of the traffic light fault dataset include: abnormal on / off states, color deviations, and timing errors.

[0013] Furthermore, the data augmentation process includes: random cropping, rotation, scaling, brightness adjustment, contrast adjustment, addition of Gaussian noise and rain / fog simulation, and the annotation adopts PASCALVOC or COCO format, and the annotation content includes the location coordinates of the traffic light, the light color status and fault type label.

[0014] Furthermore, the attention mechanism module is a CBAM attention module, comprising: a channel attention submodule and a spatial attention submodule, wherein;

[0015] The channel attention submodule outputs channel weights by performing global average pooling and global max pooling on each channel of the feature map, and then processing it through a multilayer perceptron.

[0016] The spatial attention submodule performs max pooling and average pooling on each spatial location of the feature map, and then outputs spatial weights after processing by a convolutional layer; the channel weights and spatial weights are multiplied by the original feature map to obtain the enhanced feature map.

[0017] Furthermore, the small target detection branch uses higher resolution feature maps for detection; and the calibration loss function includes using the CIoU loss function to calculate the bounding box regression loss, combined with the FocalLoss function to handle the sample imbalance problem.

[0018] Furthermore, the CIoU loss function calculates the bounding box regression loss, expressed as:

[0019]

[0020] Where IoU is the crossover-union ratio, ρ 2 (b,b gt ) is the squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box, c is the length of the diagonal of the minimum bounding rectangle, α is the weighting coefficient, and v is the aspect ratio consistency parameter.

[0021] Furthermore, the method of using the FocalLoss function to handle the sample imbalance problem is expressed as follows:

[0022] FL(p t )=-α t (1-p t ) γ log(p t );

[0023] Where, p t To predict the probability, α t The class weights are 0.7 for faulty samples and 0.3 for normal samples, with γ = 2 being the focusing parameter.

[0024] Furthermore, the multi-frame timing analysis to determine the fault type includes: comparing the sequence of light color status changes in consecutive multi-frame images with a preset normal timing logic; if the deviation exceeds a preset range, it is determined to be a timing disorder fault.

[0025] The beneficial effects of this invention are:

[0026] This invention constructs a complete closed-loop system from data processing to real-time alarms, achieving efficient and intelligent traffic light fault management. Through diverse data augmentation and the construction of professional datasets, it provides rich training samples for the model, ensuring its good generalization ability in practical applications. The combination of real-time video stream processing and multi-frame temporal analysis not only meets the requirements of high frame rate real-time detection but also accurately identifies temporal faults, overcoming the limitations of low efficiency and poor real-time performance of manual inspections. Furthermore, alarm information including location, type, time, and on-site images, coupled with a multi-channel push mechanism, can quickly link traffic management and maintenance departments, significantly shortening the fault response and handling cycle, and providing strong technical support for ensuring the normal operation of traffic lights and improving the efficiency of urban traffic management.

[0027] Meanwhile, the CBAM attention mechanism introduced into the YOLOv5 model can effectively focus on the core area of ​​traffic lights, suppress interference from complex backgrounds, and enhance the ability to extract light color features and fault details. The newly added small target detection branch, combined with high-resolution feature maps, solves the problem of easy missed detection of traffic lights at long distances or small sizes. The combined optimization of CIoU loss and FocalLoss not only improves the accuracy of bounding box localization but also enhances the model's sensitivity to the identification of a small number of fault samples, enabling various faults to be stably identified under different lighting, weather, and scene conditions, overcoming the shortcomings of traditional vision methods in terms of insufficient robustness in complex environments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a flowchart illustrating a real-time traffic light fault identification and alarm method based on an improved Yolov5 according to an embodiment of the present invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0031] According to an embodiment of the present invention, a real-time identification and alarm method for traffic light faults based on an improved YOLOv5 is provided.

[0032] like Figure 1 As shown, the real-time identification and alarm method for traffic light faults based on the improved Yolov5 according to an embodiment of the present invention includes the following steps:

[0033] Step S1: A traffic light malfunction dataset is pre-constructed. This dataset includes images and videos of traffic lights collected under various scenarios, such as urban main roads, intersections, tunnel entrances / exits, and different lighting conditions (sunny, cloudy, nighttime), as well as different weather conditions (rainy, snowy, foggy). The dataset covers normal states and various malfunction states, such as abnormal on / off states, color deviations, and timing errors. Simultaneously, the collected data is labeled with the traffic light's location coordinates, light color status, and malfunction type label, forming an initial dataset. Data augmentation processing is then performed on the initial dataset.

[0034] The annotations are in PASCALVOC or COCO format for easy model reading and training. Data augmentation processes include random cropping, rotation, scaling, brightness adjustment, contrast adjustment, adding Gaussian noise, and rain / fog simulation to improve the model's generalization ability.

[0035] Step S2: Construct an improved Yolov5 model and input the dataset processed in step S1 into the improved Yolov5 model for training to obtain a trained improved Yolov5 model.

[0036] The improved Yolov5 model includes: input layer, backbone network, feature fusion network (Neck), and head.

[0037] The input side includes the introduction of a CBAM (Convolutional Block Attention Module) attention module after augmenting the original Yolov5 Mosaic data. This module enhances the traffic light region features through dual channel and spatial weighting. The CBAM attention module includes a channel attention submodule and a spatial attention submodule, which multiplies the channel weights and spatial weights with the original feature map to obtain the enhanced feature map.

[0038] The channel attention submodule outputs channel weights by performing global average pooling and global max pooling on each channel of the feature map, and then processing it through a multilayer perceptron.

[0039] In this technical solution, the channel attention submodule performs global average pooling (GAP) and global max pooling (GMP) on the input feature map to obtain two 1×1×C feature vectors. After processing by a shared multilayer perceptron (MLP, containing one hidden layer with C / 16 neurons), the channel weight vector (1×1×C) is output through the sigmoid function to highlight the feature channel corresponding to the traffic light color and suppress background-irrelevant channels.

[0040] Among them, the spatial attention module outputs spatial weights by performing max pooling and average pooling on each spatial location of the feature map, and then processing it through a convolutional layer.

[0041] This technical solution performs max pooling and average pooling on the channel-weighted feature map to obtain two 1×H×W feature maps; these are then compressed into a single-channel feature map by a 3×3 convolutional layer (stride 1, padding 1), and the spatial weights (1×H×W) are output through the sigmoid function; these weights are used to focus the luminous area of ​​the traffic light and weaken the surrounding non-target areas.

[0042] The weighting process involves multiplying the channel weights and spatial weights sequentially by the original feature map to output an enhanced feature map, represented as:

[0043] F′=F×M c ×M s ;

[0044] Among them, M c M represents the channel weights. s For spatial weights.

[0045] The backbone network, based on the original Yolov5s CSPDarknet structure, adjusts the following parameters to optimize the convolutional kernel configuration and improve the ability to extract small target features, as follows:

[0046] The 6×6 convolutions in the first three layers of the backbone network are replaced with 3×3 convolutions to reduce the excessive compression of small target features; cross-layer residual connections are added in the C3 module to alleviate feature degradation in deep networks and retain more traffic light details; the first downsampling, i.e. from 640×640 to 320×320, is delayed to the fourth layer to extend the retention time of small target features in the high-resolution feature map.

[0047] The feature fusion network adopts the PANet structure, adding a feature propagation path for a small object detection branch. That is, based on the original PANet, a small object detection branch is added to form a 4-scale feature fusion mechanism, as detailed below:

[0048] The original three branches include feature maps downsampled by 32 times (20×20), 16 times (40×40), and 8 times (80×80) to detect large-sized traffic lights at long distances and medium-sized traffic lights at medium distances, respectively. A new small target branch is added, which is based on a feature map downsampled by 4 times (160×160) and extracts features through an additional C3 module and a 1×1 convolutional layer to specifically detect small-sized traffic lights at close distances.

[0049] This technical solution employs adaptive weight fusion, which learns the weight coefficients of features in each layer through 1×1 convolution to address the issue of feature distribution differences under different lighting conditions.

[0050] The detection head includes an optimized loss function combination to address the recognition accuracy issue of traffic light malfunction samples, as detailed below:

[0051] The bounding box regression loss, using the CIoU (CompleteIoU) loss, is expressed as follows:

[0052]

[0053] Where IoU is the crossover-union ratio, ρ 2(b,b gt ) is the squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box, c is the length of the diagonal of the minimum bounding rectangle, α is the weighting coefficient, and v is the aspect ratio consistency parameter.

[0054] This technical solution, compared to the original Yolov5 GIoU loss, more accurately measures the overlap of bounding boxes, and is especially suitable for regular rectangular targets such as traffic lights.

[0055] The classification loss, including the use of FocalLoss to address the imbalance between faulty and normal samples, is expressed as:

[0056] FL(p t )=-α t (1-p t ) γ log(p t );

[0057] Where, p t To predict the probability, α t The class weights are 0.7 for faulty samples and 0.3 for normal samples, with γ = 2 being the focusing parameter.

[0058] This technical solution is used to reduce the loss weight of a large number of normal samples and increase the model's attention to a small number of faulty samples.

[0059] In this technical solution, for the improved Yolov5 model mentioned above, the dataset processed in step S1 is divided into a training set, a validation set, and a test set. The training set is input into the improved Yolov5 model for training. During the training process, the model performance is monitored in real time through the validation set, and an early stopping strategy is adopted, that is, training is stopped when the loss of the validation set does not decrease for a consecutive preset number of rounds to prevent overfitting. After training is completed, the model is evaluated using the test set. If the model performance does not meet the preset index, the model hyperparameters are adjusted or training data is added, and retraining is performed until the model performance meets the index.

[0060] Step S3: Perform real-time identification and fault judgment. Extract images from the real-time video stream to be detected frame by frame and input them into the trained improved Yolov5 model. The model outputs the location information, color status and fault probability of the traffic light. At the same time, set a fault judgment threshold. When the fault probability is greater than or equal to the threshold, it is judged as a fault of the corresponding type.

[0061] For timing disorder faults, the sequence of light color status changes in multiple consecutive frames of images is compared with the preset normal timing logic. If the deviation exceeds the preset range, it is determined to be a timing disorder fault.

[0062] In this technical solution, the real-time video stream to be detected originates from a surveillance camera deployed at a road intersection, with the camera's frame rate not less than 25fps to ensure real-time performance.

[0063] Step S4 involves alarm and information push, including automatically triggering an alarm mechanism when a traffic light malfunction is detected, generating alarm information that includes the fault location, fault type, fault occurrence time, and on-site image / video clips; pushing the alarm information to the traffic management platform, operation and maintenance terminal, and relevant responsible persons via wired or wireless network, and storing alarm records locally in the system for subsequent traceability and statistical analysis.

[0064] The alarm mechanism includes triggering audible and visual alarms, SMS alarms, and APP push alarms on the management platform to ensure that relevant personnel receive fault information in a timely manner.

[0065] With the help of the above technical solution, the specific implementation process is as follows:

[0066] High-definition cameras with a resolution of 1920×1080 were deployed at major intersections in the city to collect traffic light images and video data for three months, covering scenarios such as sunny days, cloudy days, nighttime, rainy days, and foggy days. The data included normal conditions, with red, yellow, and green lights switching in the normal sequence, as well as fault conditions, such as red and green lights being on at the same time, yellow lights flashing continuously, green lights being insufficiently bright, and traffic lights switching irregularly.

[0067] Simultaneously, the LabelImg annotation tool was used to annotate the data in COCO format. Each annotation box included the top-left x-coordinate, top-left y-coordinate, width, and height of the traffic light, as well as a color status label and a fault type label. Data augmentation was performed on the annotated dataset, including: random cropping (cropping ratio 0.5-1.0), random rotation from -30° to 30°, random scaling (0.8-1.2 times), brightness adjustment (0.5-1.5 times), contrast adjustment (0.5-1.5 times), adding Gaussian noise with a mean of 0 and a variance of 0.01, and simulating rain and fog. The final dataset contained 100,000 images, which were divided into training, validation, and test sets in a 7:2:1 ratio.

[0068] An improved Yolov5 model was constructed and trained using the PyTorch framework. The initial learning rate was set to 0.01, the batch size to 16, the number of iterations to 300, and the optimizer to SGD. During training, accuracy, recall, and mAP (mean Average Precision) were calculated on the validation set every 10 epochs. An early stopping strategy was triggered when the mAP on the validation set did not improve for 15 consecutive epochs, thus obtaining the trained improved Yolov5 model.

[0069] The images from the real-time video stream of the road intersection surveillance camera at a frame rate of 25fps are extracted frame by frame and input into the trained improved Yolov5 model. The model processes 25 images per second and outputs the location, color status, and probability of various faults of the traffic lights.

[0070] Specifically, fault judgment thresholds are set: the threshold for abnormal brightness and color deviation faults is 0.8. For timing disorder faults, the light color sequence of 10 consecutive frames is compared with the preset normal timing sequence, such as red→yellow→green→red. If there are more than 3 mismatches, it is judged as timing disorder.

[0071] Simultaneously, upon detecting a fault, the system generates an alarm message within one second, including the fault location based on camera GPS positioning, fault type, timestamp, and a three-second video clip of the fault. The alarm message is pushed to the traffic management platform, maintenance personnel's mobile app, and the responsible person's SMS message via the 4G network, while alarm records, including processing status, are stored on a local server.

[0072] In summary, by employing the above-described technical solution of the present invention, the following effects can be achieved:

[0073] This invention constructs a complete closed-loop system from data processing to real-time alarms, achieving efficient and intelligent traffic light fault management. Through diverse data augmentation and the construction of professional datasets, it provides rich training samples for the model, ensuring its good generalization ability in practical applications. The combination of real-time video stream processing and multi-frame temporal analysis not only meets the requirements of high frame rate real-time detection but also accurately identifies temporal faults, overcoming the limitations of low efficiency and poor real-time performance of manual inspections. Furthermore, alarm information including location, type, time, and on-site images, coupled with a multi-channel push mechanism, can quickly link traffic management and maintenance departments, significantly shortening the fault response and handling cycle, and providing strong technical support for ensuring the normal operation of traffic lights and improving the efficiency of urban traffic management.

[0074] Meanwhile, the CBAM attention mechanism introduced into the YOLOv5 model can effectively focus on the core area of ​​traffic lights, suppress interference from complex backgrounds, and enhance the ability to extract light color features and fault details. The newly added small target detection branch, combined with high-resolution feature maps, solves the problem of easy missed detection of traffic lights at long distances or small sizes. The combined optimization of CIoU loss and FocalLoss not only improves the accuracy of bounding box localization but also enhances the model's sensitivity to the identification of a small number of fault samples, enabling various faults to be stably identified under different lighting, weather, and scene conditions, overcoming the shortcomings of traditional vision methods in terms of insufficient robustness in complex environments.

[0075] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Those skilled in the art, upon considering the disclosure in the specification and embodiments, will readily conceive of other embodiments of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0076] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A real-time fault identification and alarm method for traffic lights based on an improved Yolov5 system, characterized in that, Includes the following steps: Step S1: Pre-construct a traffic light fault dataset, including collecting traffic light images and video data under different scenarios, lighting conditions and weather conditions, labeling the data and performing data augmentation processing; Step S2: Construct an improved Yolov5 model and divide the dataset processed in Step S1 into training, validation, and test sets. Input the dataset into the improved Yolov5 model for training. Monitor the performance using the validation set and adopt an early stopping strategy. Evaluate the model performance using the test set to obtain the trained improved Yolov5 model. The improved Yolov5 model includes introducing an attention mechanism module at the input of the backbone network based on the Yolov5 model, adding a small object detection branch in the feature fusion network, and calibrating the loss function. Step S3: Extract images from the real-time video stream to be detected frame by frame, input them into the trained improved Yolov5 model, and output the location information, color status and fault probability of the traffic lights. Combine the preset threshold and multi-frame time sequence analysis to determine the fault type. Step S4: When a traffic light malfunction is detected, an alarm mechanism is automatically triggered to generate an alarm message containing fault information and push it to the warning terminal, while storing the alarm record.

2. The method for real-time identification and alarm of traffic light faults based on improved Yolov5 according to claim 1, characterized in that, The fault states in the traffic light fault dataset include: abnormal on / off status, color deviation, and timing disorder.

3. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 2, characterized in that, The data augmentation process includes random cropping, rotation, scaling, brightness adjustment, contrast adjustment, addition of Gaussian noise and rain / fog simulation, and the annotation adopts PASCALVOC or COCO format, and the annotation content includes the location coordinates of the traffic light, the light color status and fault type label.

4. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 1, characterized in that, The attention mechanism module is a CBAM attention module, comprising: a channel attention submodule and a spatial attention submodule, wherein; The channel attention submodule outputs channel weights by performing global average pooling and global max pooling on each channel of the feature map, and then processing it through a multilayer perceptron. The spatial attention submodule performs max pooling and average pooling on each spatial location of the feature map, and then outputs spatial weights after processing by a convolutional layer; the channel weights and spatial weights are multiplied by the original feature map to obtain the enhanced feature map.

5. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 1, characterized in that, The small target detection branch uses higher resolution feature maps for detection; and the calibration loss function includes using the CIoU loss function to calculate the bounding box regression loss, and combining the FocalLoss function to handle the sample imbalance problem.

6. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 5, characterized in that, The CIoU loss function calculates the bounding box regression loss, expressed as: Where IoU is the crossover-union ratio, ρ 2 (b,b gt ) is the squared Euclidean distance between the center of the predicted bounding box and the center of the ground truth bounding box, c is the length of the diagonal of the minimum bounding rectangle, α is the weighting coefficient, and v is the aspect ratio consistency parameter.

7. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 6, characterized in that, The method of using the FocalLoss function to handle the imbalanced sample problem is expressed as follows: FL(p t )=-a t (1-p t ) γ log(p t ); Where, p t To predict the probability, α t The class weights are 0.7 for faulty samples and 0.3 for normal samples, with γ = 2 being the focusing parameter.

8. The real-time identification and alarm method for traffic light faults based on improved Yolov5 according to claim 1, characterized in that, The multi-frame timing analysis to determine the fault type includes: comparing the sequence of light color status changes in consecutive multi-frame images with a preset normal timing logic; if the deviation exceeds a preset range, it is determined to be a timing disorder fault.

Images