Method for detecting and positioning thrown objects on expressway

Abstract

The invention relates to the technical field of image recognition, in particular to a method for detecting and positioning thrown objects on an expressway. The method comprises the following steps: S1, acquiring image data; s2, preprocessing the image data; s3, performing data annotation and constructing a data set; s4, performing improvement on the basis of the YOLOv8 original deep learning target detection network, and adding a P2 feature detection layer sensitive to a small target; s5, an output filtering mechanism is added and used for filtering the throwing objects which do not need to be treated; s6, training a target detection network to obtain a trained model; and S7, shooting an expressway video in real time, then performing detection by using the model trained in the step S6, and performing positioning and alarming when a to-be-processed throwing object is detected. By adopting the method, the high-risk thrown objects can be well identified.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for detecting and locating debris spilled on highways. Background Technology

[0002] As the mileage of highways both domestically and internationally increases year by year, incidents of debris being thrown onto highways are also on the rise. Debris is one of the important causes of traffic accidents and congestion.

[0003] Current technologies primarily rely on manual patrols and image detection via surveillance cameras to detect spilled materials. Manual patrols depend on highway inspectors conducting regular patrols along the highway, but this method is inefficient, cannot detect spilled materials in a timely manner, and poses certain risks. Surveillance camera detection uses relatively simple image recognition algorithms, resulting in low accuracy. Furthermore, there are many types of spilled materials, some high-risk and some low-risk, which current image recognition methods cannot effectively distinguish. Summary of the Invention

[0004] The technical problem to be solved by the present invention is: a method for detecting and locating debris on highways, which can effectively identify high-risk debris.

[0005] The technical solution adopted in this invention is: a method for detecting and locating spilled materials on highways, which includes the following steps: S1, Image data acquisition; S2, Image data preprocessing; S3, Data Labeling and Dataset Construction; S4. Improve the original YOLOv8 deep learning object detection network by adding a P2 feature detection layer that is sensitive to small targets; S5. Add an output filtration mechanism to filter out spills that do not need to be processed; S6. Train the object detection network and obtain the trained model; S7. Capture real-time video of the highway, and then use the model trained in step S6 to detect it. When spilled material that needs to be handled is detected, locate it and issue an alarm.

[0006] Preferably, the video images in steps S1 and S7 are all captured by drones, and the video images captured by drones all contain location information.

[0007] Preferably, the image data processing in step S2 includes random image rotation, random translation, scaling, flipping, and image blending. It also requires the use of structural similarity algorithms and peak signal-to-noise ratio algorithms to automatically filter the data and remove images with low quality or high repetition.

[0008] Preferably, the data annotation in step S3 specifically includes: S3.1 Tire fragments are labeled with the largest bounding rectangle and the label is set to "Tyre-skin"; S3.2. Rocks or hard objects are labeled with the largest bounding rectangle and the label is set to "Block". S3.3 Other common spilled materials are labeled with the largest bounding rectangle and the label is set to "Others"; Furthermore, after the annotation is completed, data augmentation methods are needed to further expand the data volume in order to build training and validation datasets of appropriate size.

[0009] Preferably, the improved YOLOv8 object detection network in step S4 specifically includes: The backbone network extracts multi-scale feature maps P2, P3, P4, and P5, which are composed of multiple convolutional and C2f modules, and an additional shallow feature map P2 output channel is added. The neck network performs upsampling on deep feature maps to increase their resolution; secondly, it concatenates the upsampled feature maps with shallow feature maps; and thirdly, it further fuses the concatenated feature maps through convolution and C2f modules. The detection head network incorporates a global attention mechanism.

[0010] Preferably, the global attention mechanism includes the following steps: 1) Perform global average pooling on the input feature map to obtain the feature vector; 2) The obtained feature vectors are processed through an activation function to obtain the channel weights; 3) Apply the calculated channel weights to the original feature map and perform weighting of the feature map through channel-by-channel multiplication.

[0011] Preferably, the output filtering mechanism in step S5 refers to setting a filtering function: φ(Bi)=α·pi+β·Ai / (H·W)+γ·Wci Where Bi is the target detection box, pi is the detection confidence, Ai is the area of ​​the rectangular bounding box of the detected target, H·W is the overall area of ​​the input image, Wci is the target category weight coefficient, and α, β, γ are weight coefficients, and α+β+γ=1. When the filtering function φ(Bi) is lower than the set threshold, the target is determined to be spilled material that does not need to be processed, and it is automatically filtered.

[0012] Compared with existing technologies, the present invention has the following advantages using the above structure: The present invention improves the YOLOv8 target detection network, increases the utilization of the P2 output layer, and integrates multi-scale fine-grained features in the decoding stage, thereby improving the detection effect of small targets (such as rags, shredded paper, glass fragments, etc.). In addition, it adds an output filtering mechanism to effectively filter low-impact targets such as leaves and paper bags that do not require processing, thereby reducing the false alarm rate. Detailed Implementation

[0014] The embodiments of the present invention are described in detail below.

[0015] Example 1: A method for detecting and locating debris spilled on highways, comprising the following steps: S1. Image Data Acquisition: A high-precision camera mounted on a drone platform is used to cruise and capture images along a predetermined highway section. The drone is programmed with an automatic cruise path, flying at a set altitude and speed according to the inspection route to acquire high-definition image data. The drone's flight path is set based on highway characteristics, selecting accident-prone areas, curved sections, bridges, tunnel entrances and exits, and areas with poor lighting conditions for regular patrols to ensure real-time coverage and high-quality image acquisition. Simultaneously, the drone is equipped with a GPS and RTK differential positioning system, capable of recording the precise geographic coordinates of each frame of image in real time, supporting subsequent positioning. S2. Image Data Preprocessing: The acquired video images are initially screened and edited using image processing techniques, retaining video segments containing projectile targets. Then, frame extraction is performed, extracting 5 frames per second. To improve the model's generalization ability and detection accuracy, the following data augmentation methods are used in the preprocessing stage: random image rotation (±25°), random translation (±50%), scaling (90%), vertical flipping (50% probability), and image blending (Mixup, Mosaic) processing to enhance the model's adaptability to complex real-world scenes. Furthermore, structural similarity (SSIM) and peak signal-to-noise ratio (PSNR) algorithms are used to automatically filter the data, removing low-quality or highly repetitive images to ensure high data quality and validity. S3. Data Labeling and Dataset Construction: The labeling tool LabelImg was used to label the highway spill data, including: S3.1 Tire fragments are labeled with the largest bounding rectangle and the label is set to "Tyre-skin"; S3.2. Rocks or hard objects are labeled with the largest bounding rectangle and the label is set to "Block". S3.3 Other common spilled items, such as plastic bags and small miscellaneous items, should be labeled with the largest bounding rectangle and set to "Others".

[0016] After labeling, data augmentation methods are used to further expand the data volume in order to build training and validation datasets of appropriate size; S4. Improve the original YOLOv8 deep learning object detection network by adding a P2 feature detection layer that is sensitive to small targets; The improved YOLOv8 object detection network includes the following structure: Backbone Improvement: Based on the original YOLOv8 structure, the extraction and application of shallow features (P2) are enhanced to more effectively capture and detect small debris on highways. Specifically, an additional shallow feature map output channel (P2 channel) is added to the existing convolutional and C2f modules, enabling the network to retain more high-resolution detailed features. Neck network structure optimization: A path aggregation network (PANet) approach is adopted to increase the feature fusion path, fusing the shallow feature map P2, which originally did not perform feature fusion, with the high-level semantic feature map. Specifically, the deep feature map is first upsampled to increase its resolution; second, the upsampled feature map is concatenated with the shallow feature map; third, the concatenated feature map is further fused through convolution and C2f modules to enhance the network's ability to capture fine-grained information. Head Enhancement: In terms of head design, a Global Attention Mechanism (GAM) is introduced. By calculating attention weights in the channel and spatial dimensions of the feature map, the model can more accurately focus on important regions related to target detection, thereby improving the model's accuracy in identifying highway debris. The specific implementation of the global attention mechanism includes the following steps: 1) Perform global average pooling on the input feature map to obtain channel-level feature vectors; 2) Process the obtained channel feature vectors through an activation function (such as ReLU or Sigmoid) to obtain the channel weights; 3) Apply the calculated channel weights to the original feature map, and perform channel-by-channel multiplication to weight the feature map, highlighting important feature regions related to the spill detection task.

[0017] The above improvements significantly enhance the accuracy and robustness of the YOLOv8 network in highway debris detection, especially small target detection tasks. The specific code improvements are as follows: |Lines of code|Corresponding improvement points|Technical goals |Scales parameter modification|Shallow channel expansion / deep channel compression|Optimized allocation of computing resources |Triple upsampling in the head | High-resolution feature fusion at P2 layer | Improved small target detection capability | Concat cross-level index join | Bidirectional cross-level feature concatenation | Multi-scale context enhancement |Detect Input 4 Layers| Four-layer Cascaded Detection (P2-P5) | Multi-scale Coverage Expansion In the original YOLOv8, the max_channels for the n-scale was 640, but it has now been changed to 1024, which allows for a higher number of channels in shallow layers (such as P2 / P3) to improve the feature capacity of small targets. In the original YOLOv8, the max_channels at the x-scale was 1024, but it has now been changed to 512, which limits the number of channels at deeper layers (P4 / P5) to reduce computational load. A shallow high-resolution feature is introduced (the P2 feature map is 160x160 when the input is 640x640), and the resolution is gradually restored through three upsampling steps (layers 12 / 15 / 18). In the downsampling path, the features of the current layer are concatenated with those of higher layers, and cross-scale context aggregation is achieved through the C2f module (similar to the FPN++ structure). The original YOLOv8 had a 3-layer input (P3-P5), while this improved version adds a P2 layer to form a four-layer detection. S5. Add an output filtration mechanism to filter out spills that do not need to be processed; Set a filter function: φ(Bi)=α·pi+β·Ai / (H·W)+γ·Wci Where Bi is the target detection box, pi is the detection confidence, Ai is the area of ​​the rectangular bounding box of the detected target, H·W is the overall area of ​​the input image, Wci is the target category weight coefficient (tire skin = 1.0, stone = 0.9, other debris = 0.2), α, β, γ are weight coefficients, and α+β+γ=1. When the filtering function φ(Bi) is lower than the set threshold, in this embodiment it is set to 0.3, the target is determined to be spilled material that does not need to be treated, and it is automatically filtered.

[0018] The detection confidence score pi is not manually set, but automatically output by the YOLOv8 model during object detection. Specifically, the YOLOv8 model outputs three parts for each detection box: Classification Score Objectness Score Target location coordinates (Bounding Box) Here, pi is the objectness score output by the YOLOv8 model, which ranges from 0 to 1 and represents the degree of confidence that the model has detected a valid target within the bounding box. In practice, this value is obtained directly from the model's prediction output and does not require additional calculation or manual setting.

[0019] The purpose of calculating Ai / (H·W) is to standardize the relative size of the detected target and more objectively judge the potential impact of the target on driving safety. For example, a target of the same size will occupy a smaller area in a higher resolution image and its danger will be reduced accordingly. α emphasizes the model's confidence in the target, ensuring that high-confidence targets are prioritized. β emphasizes the impact of the target's relative size on safety; larger targets pose a higher safety risk. The γ category weighting coefficient is used to reflect the differences in the degree of impact of different types of objects on highway safety. S6. The object detection network is trained to obtain the completed model. Training is performed using a labeled dataset and an improved YOLOv8 model, employing the Adam optimizer, a learning rate of 0.0001, and a training cycle of 300 epochs. An early stopping mechanism (patience=150) is used to improve training efficiency and detection accuracy. The loss value is monitored in real-time during training, and the image enhancement strategy strength is dynamically adjusted based on the loss reduction trend to further improve training performance.

[0020] S7. Video footage captured in real-time by the drone is processed by a trained model. When the model detects spilled material with an effectiveness score exceeding a threshold, it immediately records the information and converts the target location into the corresponding highway marker using a GPS coordinate conversion module. This information is then pushed to the highway management center. Upon receiving the information, the management center activates the alarm system, generating an alarm message containing the location, category, and detection image. This message is then sent to on-duty personnel for further handling, ensuring highway operational safety. Furthermore, the system supports the automatic generation of log records containing location information and image evidence for subsequent statistical analysis and system maintenance.

[0021] This invention enables efficient and accurate detection and location of debris spilled on highways, effectively improving road safety management efficiency and reducing the cost and risk of manual inspection.

[0022] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

[0023] For those skilled in the art, various changes and modifications will undoubtedly be apparent after reading the above description. Therefore, the appended claims should be construed as covering all changes and modifications that encompass the true intent and scope of the invention. Any and all equivalent scope and content within the scope of the claims should be considered to remain within the intent and scope of the invention.

Claims

1. A method for detecting and locating spilled materials on highways, characterized in that, It comprises the following steps: S1, image data acquisition; S2, image data preprocessing; S3, data labeling and constructing data set; S4, improving the original YOLOv8 deep learning target detection network, adding a P2 feature detection layer sensitive to small targets; S5, adding an output filtering mechanism to filter out unnecessary litter; S6, training the target detection network to obtain the trained model; S7, real-time shooting of the expressway video, and then using the trained model in step S6 to detect and locate the litter that needs to be treated and alarm.

2. The method of claim 1, wherein: The video images in steps S1 and S7 are shot by a UAV, and the video images shot by the UAV all have position information.

3. The method of claim 1, wherein: The image data processing in step S2 includes image random rotation, random translation, scaling, up-down flipping and image mixing processing, and the structure similarity algorithm and peak signal-to-noise ratio algorithm are also used to automatically screen the data and remove images with low quality or high repetition.

4. The method of claim 1, wherein: The data labeling in step S3 specifically includes: S3.1, tire debris is labeled with the maximum circumscribed rectangle, and the label is set as "Tyre-skin"; S3.2, stone or hard object is labeled with the maximum circumscribed rectangle, and the label is set as "Block"; S3.3, other common litter is labeled with the maximum circumscribed rectangle, and the label is set as "Others"; After labeling, the data volume is further expanded through data enhancement method to construct a training and verification data set with appropriate scale.

5. The method of claim 1, wherein: The improved YOLOv8 target detection network in step S4 specifically includes: The backbone network extracts multi-scale feature maps P2, P3, P4 and P5, which are composed of multiple convolution and C2f modules, and an additional shallow feature map P2 output channel is added; The neck network performs upsampling operation on the deep feature map to expand the feature map resolution; secondly, the upsampling feature map is spliced with the shallow feature map; thirdly, the spliced feature map is further fused and processed through convolution and C2f module; The detection head network adds a global attention mechanism.

6. The method of claim 5, wherein: The global attention mechanism comprises the following steps: 1) performing global average pooling operation on the input feature map to obtain a feature vector; 2) processing the obtained feature vector through an activation function to obtain a channel weight; 3) applying the calculated channel weight to the original feature map to realize weighting of the feature map through channel-by-channel multiplication.

7. The method of claim 1, wherein: The output filtering mechanism in step S5 refers to setting a filtering function: φ(Bi)=α·pi+β·Ai / (H·W)+γ·Wci Where Bi is the target detection box, pi is the detection confidence, Ai is the area of the rectangular circumscribed box of the detection target, H·W is the overall area of the input image, Wci is the target class weight coefficient, and α, β, γ are weight coefficients, and α+β+γ=1; When the filtering function φ(Bi) is lower than the set threshold, it is determined that the target is litter that does not need to be treated and is automatically filtered out.