Road debris detection method, system and device suitable for low-computing-power edge device

Abstract

The application belongs to the technical field of intelligent transportation systems, and discloses a road litter detection method, system and device suitable for low-computing-power edge equipment. The video sampling frame is scaled and then input into the detection model, and the two-dimensional boundary box and the category confidence of the suspected road litter are output. The detection model is obtained by embedding an ECA module at the output end of the depth separable convolution module of YOLO Nano. The two-dimensional boundary box is screened, and only the target box with the center point in the preset ROI region and the category confidence greater than the set threshold is reserved as the effective target box. The IoU of the effective target box in different sampling frames is calculated. If the IoU is continuously greater than the set threshold and the maintenance time exceeds the time threshold, it is determined that the target is a static road litter. The key frame of the video is intercepted and labeled, and is sent to the cloud server. The application effectively improves the feature perception ability of small and micro road litter and reduces the small target missing detection rate without increasing the computing power overhead of the edge equipment.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation systems technology, specifically relating to a method, system, and device for detecting road debris suitable for low-computing-power edge devices. Background Technology

[0002] In the daily operation of highways and urban expressways, "road debris / spillage" such as cargo fragments falling from trucks, abandoned tires, and construction cones left behind are safety hazards that can cause serious traffic accidents such as chain-reaction rear-end collisions and vehicle rollovers.

[0003] Currently, vision-based solutions for detecting road debris have significant limitations in low-computing-power scenarios at the edge:

[0004] Existing high-precision detection solutions mostly rely on heavy-duty networks with large parameter counts, such as YOLOv5 and YOLOv8. These networks require expensive GPUs or high-end NPUs to provide computing power, making them difficult to popularize at the edge. Meanwhile, surveillance cameras along highways are often powered by solar energy or limited by the space in the distribution box, and can only be equipped with low-power ARM chips or entry-level edge computing boxes (such as the basic Jetson Nano platform) costing tens to hundreds of yuan. Traditional heavy-duty networks cannot meet the frame rate requirements for real-time processing.

[0005] Lightweight networks face the problem of high false negative rates for small targets. When using conventional ultra-lightweight networks such as YOLO Nano and MobileNet-SSD without modification, excessive channel compression and feature dimension reduction can lead to loss of feature information. This is especially true when detecting small debris (such as bricks or cardboard boxes) on the road surface 50 meters away from the camera, where the false negative rate is more pronounced.

[0006] Furthermore, the core characteristic of debris is that it is "stationary on the lane". To distinguish between moving vehicles and stationary debris, existing technologies often introduce complex background modeling methods such as Gaussian Mixture Model (GMM) or multi-target trajectory tracking algorithms such as DeepSORT and Kalman filtering. Such logic design and engineering implementation are difficult: these algorithms not only further consume the limited computing power of edge devices, but also are prone to frequent tracking ID loss (ID Switch) due to changes in light, swaying tree shadows or vehicle occlusion during the actual code writing process, resulting in a large number of false alarms and increasing implementation and debugging costs.

[0007] Therefore, existing traffic incident detection systems, when deployed at the edge, suffer from technical problems such as high hardware costs, insufficient sensitivity of lightweight networks for detecting small-sized targets, and high engineering complexity and computational overhead in distinguishing between moving and static targets. The industry urgently needs a road debris detection method that can run in real time on low-computing-power edge devices, has high detection accuracy for small-sized debris, and is easy to implement in engineering. Summary of the Invention

[0008] The purpose of this invention is to address the shortcomings of existing technologies by providing a method, system, and device for detecting road debris on edge devices with low computing power. This invention is based on the existing lightweight target detection network YOLO Nano and its efficient channel attention (ECA) mechanism. Without altering the original internal structure of the YOLO Nano network and ECA module, the core solution achieves scenario-specific adaptation by optimizing module integration, designing a simplified static verification logic, and constructing an edge-cloud collaborative architecture. This effectively solves the technical pain points of edge-side road debris detection.

[0009] Specifically, the present invention is implemented using the following technical solutions.

[0010] On one hand, the present invention provides a method for detecting road debris suitable for low-computing-power edge devices, comprising:

[0011] Lightweight target detection: Receives road video streams collected by front-end edge detection devices, samples the road video streams using down-sampling, scales each video sample frame and inputs it into the ECA-YOLO Nano target detection model, and outputs the coordinates of the two-dimensional bounding boxes and the category confidence of all suspected road debris;

[0012] Region filtering: Based on the preset ROI region, the two-dimensional bounding boxes are filtered, and only two-dimensional bounding boxes whose center point is located within the ROI region and whose category confidence is greater than the set confidence threshold are retained as valid target boxes, and the coordinate data of the valid target boxes are output.

[0013] Pure geometric static verification: Construct a static candidate pool using the effective target boxes and record the initial timestamp. Calculate the IoU value for the effective target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, output an alarm trigger command and the coordinate data of the effective target box.

[0014] Evidence collection and reporting: After receiving the alarm trigger command and the coordinate data of the valid target box, the corresponding video sampling frame of the valid target box is marked as a key frame, and the alarm trigger command and the relevant lightweight data of the valid target box are encapsulated into an event alarm message for output; the relevant lightweight data, in addition to the coordinate data of the valid target box, also includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence, and key frames;

[0015] The ECA-YOLO Nano target detection model is obtained by embedding an ECA module at the output of each depthwise separable convolutional module in the later stage of feature extraction in the YOLO Nano backbone network. The depthwise separable convolutional module contains at least two sets of depthwise separable convolution operations. The ECA module directly achieves cross-channel information aggregation through one-dimensional convolution with a window size of k, where k is the adaptive kernel size.

[0016] Furthermore, the specific steps of the pure geometric static verification include:

[0017] 3-1) When a new valid target box is received, an initial timestamp is assigned to it, and the coordinates and initial timestamp of the valid target box are stored in a lightweight candidate dictionary; the lightweight candidate dictionary records the valid target box and its initial timestamp and latest timestamp;

[0018] 3-2) In each video sampling frame, extract the new valid target box B, traverse the valid target boxes A stored in the lightweight candidate dictionary, and calculate the IoU between the valid target box B and the valid target box A;

[0019] 3-3) If the calculated IoU is greater than the set threshold, update the latest timestamp of the valid target box A with the initial timestamp of the valid target box B; if IoU=0, remove the valid target box from the lightweight candidate dictionary.

[0020] 3-4) When the duration of a valid target box in the lightweight candidate dictionary, i.e., the latest timestamp minus the initial timestamp, is greater than or equal to the set time threshold, it is determined that there is road debris on the road surface, and an alarm trigger command and the coordinate data of the valid target box are output.

[0021] Furthermore, the road debris detection method suitable for low-computing-power edge devices also includes: using an inference framework to convert the weights of the trained ECA-YOLO Nano target detection model into an INT8 quantization model.

[0022] Furthermore, the event alarm message is sent to the background cloud receiving server and / or the roadside broadcasting equipment at a specified distance ahead.

[0023] Furthermore, the ROI area is the main driving lane.

[0024] Further, the step of filtering the two-dimensional bounding boxes according to the preset ROI region, retaining only the two-dimensional bounding boxes whose center point is located within the ROI region as valid target boxes, and outputting the coordinate data of the valid target boxes includes:

[0025] When filtering the two-dimensional bounding boxes, the center point coordinates of all two-dimensional bounding boxes suspected of being road debris are compared with the spatial position of the main driving lane. Target boxes whose center points fall outside the ROI area are directly discarded, and only two-dimensional bounding boxes whose center points are located within the main driving lane are retained as valid target boxes.

[0026] On the other hand, the present invention also provides a road debris detection system suitable for low computing power edge devices, which implements the above-mentioned road debris detection method suitable for low computing power edge devices, including a front-end edge detection device and a back-end cloud receiving server;

[0027] The front-end edge detection device includes an intelligent capture camera, a low-power edge computing board, a circular memory buffer, and a communication module.

[0028] The intelligent capture camera collects road video streams;

[0029] The low-power edge computing board receives road video streams captured by a smart capture camera, samples the road video stream using down-sampling, and inputs each video sample frame into ECA-YOLO after scaling. The Nano target detection model outputs the coordinates and category confidence scores of 2D bounding boxes for all suspected road debris. Based on a preset Region of Interest (ROI), the 2D bounding boxes are filtered, retaining only those whose center point is within the ROI and whose category confidence score is greater than a set confidence threshold as valid target boxes. A static candidate pool is constructed using these valid target boxes, and an initial timestamp is recorded. The Interchange of Unit (IoU) value is calculated for each valid target box in the video sampling frames at set intervals. When the IoU value is consistently greater than a set threshold and the duration exceeds a set time threshold, a road debris event is triggered. The corresponding video sampling frame of the valid target box is marked as a keyframe, and an alarm trigger command and the coordinate data of the valid target box are output. The alarm trigger command and the relevant lightweight data of the valid target box are encapsulated into an event alarm message for output. The relevant lightweight data, in addition to the coordinate data of the valid target box, also includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence score, and keyframe.

[0030] The circular memory buffer provides buffering for each video sampling frame and support for keyframe extraction.

[0031] After receiving the event alarm message, the communication module reports the event alarm message to the background cloud receiving server and / or the roadside broadcasting equipment at a specified distance ahead.

[0032] The backend cloud receiving server receives event alarm messages sent by the front-end edge detection device and displays lightweight data related to the event alarm messages.

[0033] Furthermore, the low-power edge computing board runs an ECA-YOLO Nano improved network inference module, an ROI filtering module, an IoU static verification module, and an image rendering module; each module interacts sequentially according to the data flow order to complete the entire process from road video stream input to event alarm message reporting;

[0034] The ECA-YOLO Nano improved network inference module receives road video streams collected by front-end edge detection devices, samples the road video streams using down-sampling, scales each video sample frame and inputs it into the ECA-YOLO Nano target detection model, and outputs the coordinates of the two-dimensional bounding boxes and the category confidence of all suspected road debris.

[0035] The ROI filtering module filters the two-dimensional bounding boxes according to the preset ROI region, retaining only the two-dimensional bounding boxes whose center point is located within the ROI region and whose category confidence is greater than the set confidence threshold as valid target boxes, and outputs the coordinate data of the valid target boxes.

[0036] The IoU static verification module constructs a static candidate pool using the effective target boxes and records the initial timestamp. It calculates the IoU value for the effective target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, it outputs an alarm trigger command and the coordinate data of the effective target box.

[0037] Upon receiving the alarm trigger command and the coordinate data of the valid target box, the image rendering module marks the corresponding video sampling frames of the valid target box as keyframes, and encapsulates the alarm trigger command and the coordinate data of the valid target box into an event alarm message for output.

[0038] In another aspect, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the road debris detection method applicable to low-computing-power edge devices described above.

[0039] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that: when the computer program is executed by a processor, it implements the steps of the road debris detection method applicable to low-computing-power edge devices described above.

[0040] The beneficial effects of the road debris detection method, system, and equipment of the present invention, applicable to low-computing-power edge devices, are as follows:

[0041] Regarding the computational power requirements and small target detection accuracy, this invention addresses the problems of conventional lightweight networks, which suffer from excessive feature dimensionality reduction and are prone to losing feature information for distant road debris (such as a single cardboard box), leading to missed detections of small targets. It also addresses the issues of heavy-duty detection networks, which, due to their large parameter count, cannot achieve real-time inference on low-computing-power edge devices. Specifically, this invention makes domain-specific structural improvements to the YOLO Nano model, designed specifically for IoT edge devices. Based on the existing lightweight YOLO Nano network and the ECA one-dimensional convolutional channel attention mechanism, the network structure is improved by embedding an ECA module at the output of the YOLO Nano depthwise separable convolutional module (ECA-YOLO Nano). Without altering the original module's internal structure, cross-channel information interaction is achieved through one-dimensional fast convolution. This significantly enhances the network's ability to perceive and extract features of small, distant road debris with almost no increase in computational cost (FLOPs) or model size. This invention can adapt to the real-time detection needs of low-computing-power edge devices. The improved model size is still controlled within 5MB. Without increasing the computing power overhead of the edge NPU (Neural Processing Unit), it effectively improves the network's ability to perceive the features of small road debris at a distance of 50-80 meters and reduces the false negative rate of small targets.

[0042] Regarding engineering implementation difficulty and false alarm control, this invention addresses the problems of existing technologies employing complex algorithms such as multi-target trajectory tracking (e.g., DeepSORT), which result in high code complexity and are prone to tracking ID loss in scenarios with occlusion or changes in lighting, leading to detection failure and a sharp increase in false alarm rates. This invention abandons traditional computationally expensive multi-target trajectory tracking algorithms and complex background modeling algorithms, and designs an easily engineeringable "spatiotemporal high IoU geometric matching" static determination mechanism based on a sliding time window. It distinguishes between moving and static targets through pure geometric operations, comparing the geometric overlap of detection boxes within a short time window to lightweightly and efficiently filter out moving vehicle targets and accurately locate stationary road debris. This invention significantly reduces engineering implementation difficulty; the core static determination logic requires only a small amount of basic Python / C++ code and does not rely on third-party tracking algorithm libraries. The system's anti-interference capability and stability are significantly improved, while effectively reducing the CPU computing power overhead of edge devices and decreasing false alarms.

[0043] Regarding communication bandwidth consumption and cloud pressure, this invention addresses the problems of existing technologies that employ a "front-end video stream push + back-end cloud analysis" architecture, which not only incurs high video transmission traffic costs but is also susceptible to network fluctuations, leading to interruptions and failures in the detection and analysis process. This invention is fully compatible with low-cost, low-computing-power edge detection devices, requiring no expensive GPUs or high-end NPUs, thus simplifying engineering implementation. Employing a "front-end edge closed-loop analysis + uploading only event result images and text" data flow mode, the edge completes the entire detection and judgment process, only reporting data to the cloud after confirming a valid event, significantly reducing communication bandwidth overhead between the edge and the cloud. This invention completely eliminates the video transmission bandwidth bottleneck between the edge and the cloud, perfectly adapting to roadside monitoring deployment scenarios on highways and urban expressways, effectively achieving automated and accurate detection and early warning of road debris. Under normal detection conditions, it consumes almost no uplink communication bandwidth, significantly reducing the data throughput pressure on the traffic management cloud platform, and is perfectly adaptable to the deployment environment of remote highways with solar power supply and 4G IoT. Attached Figure Description

[0044] Figure 1 This is a system structure block diagram of an embodiment of the present invention.

[0045] Figure 2 This is a schematic diagram of the target detection model structure according to an embodiment of the present invention.

[0046] Figure 3 This is a schematic diagram of the method flow according to an embodiment of the present invention.

[0047] Figure 4 This is a schematic diagram of the pure geometric static verification logic in an embodiment of the present invention. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the embodiments and the accompanying drawings.

[0049] One embodiment of the present invention is a road debris detection system suitable for low-computing-power edge devices, such as... Figure 1 As shown, it includes a front-end edge detection device and a back-end cloud receiving server.

[0050] The front-end edge detection device includes an intelligent capture camera, a low-power edge computing board, a circular memory buffer, and a communication module.

[0051] Intelligent capture cameras collect road video streams, for example, by pulling local road video streams via the RTSP protocol.

[0052] A low-power edge computing board serves as the core execution unit. It receives road video streams captured by a smart capture camera, samples the video stream using downsampling, scales each video sample frame, and inputs it into the ECA-YOLONano target detection model. The model outputs the coordinates and category confidence scores of all suspected road debris bounding boxes. Based on a preset Region of Interest (ROI), the bounding boxes are filtered, retaining only those whose center point is within the ROI and whose category confidence score is greater than a set confidence threshold. A static candidate pool is constructed using these valid bounding boxes, and an initial timestamp is recorded. The IoU value is calculated for each valid bounding box in the video sample frames at set intervals. When the IoU value consistently exceeds a set threshold for a duration exceeding a set time threshold, a road debris event is triggered. The corresponding video sample frame of the valid bounding box is marked as a keyframe, and an alarm trigger command and the coordinate data of the valid bounding box are output. The alarm trigger command and the relevant lightweight data of the valid bounding box are encapsulated into an event alarm message and output. In addition to the coordinates of the valid target bounding boxes, the relevant lightweight data also includes, but is not limited to, message information such as the front-end edge detection device identifier, location information, event timestamps, category confidence, and keyframes.

[0053] A circular memory buffer provides support for caching each video sampling frame and extracting keyframes.

[0054] After receiving the event alarm message, the communication module reports the event alarm message to the background cloud receiving server and / or the roadside broadcasting equipment (e.g., C-V2X roadside RSU) at a specified distance ahead.

[0055] The backend cloud-based receiving server receives event alarm messages sent by the frontend edge detection devices and displays lightweight data related to these messages, enabling early warning and data storage for spill events. The backend cloud-based receiving server includes a data receiving module, an early warning display module, and a data storage module. The data receiving module receives event alarm messages sent by the frontend edge detection devices; the early warning display module displays lightweight data related to these messages; and the data storage module stores the event alarm messages sent by the frontend edge detection devices.

[0056] The present invention provides a road debris detection system suitable for low-computing-power edge devices. The overall architecture embodies an edge-cloud collaborative design of "edge closed-loop detection + lightweight cloud reporting", which is suitable for roadside deployment scenarios with low computing power and weak network.

[0057] Preferably, in another embodiment, the ECA-YOLO Nano improved network inference module, ROI filtering module, IoU static verification module and image rendering module are run on the low-power edge computing board; each module interacts sequentially according to the data flow order to complete the entire process from road video stream input to event alarm message reporting.

[0058] The ECA-YOLO Nano improved network inference module uses the ECA-YOLO Nano target detection model for lightweight target detection. It receives road video streams collected by front-end edge detection devices, samples the road video streams using downsampling, scales each video sample frame, and inputs it into the ECA-YOLO Nano target detection model. It outputs the coordinates of the two-dimensional bounding boxes and the category confidence of all suspected road debris. The ECA-YOLO Nano target detection model is obtained by embedding an ECA (channel attention) module at the output of the depthwise separable convolution module of the YOLO Nano network. The depthwise separable convolution module contains at least two sets of depthwise separable convolution operations. The ECA module directly achieves cross-channel information aggregation through one-dimensional convolution with a window size of k, where k is the adaptive kernel size.

[0059] The ROI filtering module filters the two-dimensional bounding boxes according to the preset ROI region, retaining only the two-dimensional bounding boxes whose center point is located within the ROI region and whose category confidence is greater than the set confidence threshold as valid target boxes, and outputs the coordinate data of the valid target boxes to eliminate invalid detection areas, filter out valid detection areas, and reduce the computing power consumption of subsequent modules.

[0060] The IoU static verification module constructs a static candidate pool using the valid target boxes and records the initial timestamp. It calculates the IoU value for the valid target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, it outputs an alarm trigger command and the coordinate data of the valid target box. By calculating the IoU of the valid target boxes in different video sampling frames, static debris determination is achieved without the need for multi-target tracking.

[0061] The image rendering module completes video keyframe annotation and event alarm message encapsulation. This includes receiving the alarm trigger command and the coordinate data of the valid target box, annotating the corresponding video sampling frame of the valid target box as a keyframe, and encapsulating the alarm trigger command and the relevant lightweight data of the valid target box into an event alarm message for output. The relevant lightweight data includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence, and keyframes, in addition to the coordinate data of the valid target box.

[0062] The low-power edge computing board can be integrated into a smart capture camera, and can be equipped with a hardware solution featuring Rockchip RV1109 / RV1126 or the basic version of NVIDIA Jetson Nano.

[0063] The construction and training methods of the ECA-YOLO Nano object detection model are as follows:

[0064] In mainstream deep learning frameworks such as PyTorch, the open-source YOLO Nano code is retrieved, and the depthwise convolutional feature extraction layer is located. An ECA channel attention layer is then inserted at the output of this feature extraction layer; for example, ... Figure 2 As shown, the backbone feature extraction layer (YOLO Nano backbone layer) is configured with two sets of feature units in series. Each set of feature units consists of a depthwise separable convolutional module and an ECA channel attention module connected in series. An ECA channel attention module is set after each depthwise separable convolutional module to optimize features. Each depthwise separable convolutional module contains at least two sets of depthwise separable convolutional operations. The two sets of feature units are cascaded to complete multi-level feature extraction. This ECA channel attention layer does not use traditional fully connected layers for dimensionality reduction, but instead directly achieves cross-channel information aggregation through one-dimensional convolution with a window size of k, where k is an adaptive kernel size, such as 3 or 5, which can be adjusted according to the actual needs of the road debris detection scenario. Subsequently, a dataset containing various types of road debris such as cardboard boxes, tire fragments, and cones is used to train the ECA-YOLO Nano target detection model with targeted weights to obtain model weights adapted for edge detection.

[0065] The quantization and forward inference methods of the ECA-YOLO Nano improved network inference module are as follows:

[0066] Using an inference framework (such as TensorRT or NCNN), the weights of the trained ECA-YOLO Nano object detection model are converted into an INT8 quantized model to reduce the computational overhead of model inference. The quantized ECA-YOLO Nano object detection model is then loaded into the runtime memory of a low-power edge computing board. The intelligent capture camera samples the road video stream using a down-sampling method (e.g., at a set frame rate of 25 FPS). Each video sample frame is scaled and then input into the ECA-YOLO Nano object detection model in the improved ECA-YOLO Nano network inference module. The improved ECA-YOLO Nano network inference module completes the inference, outputting the category confidence scores and 2D bounding box coordinates of all suspected obstacles, such as the diagonal coordinates (x1, y1, x2, y2), and passes them to the ROI filtering module. (x1, y1) are the coordinates of one vertex of the 2D bounding box, (x2, y2) are the coordinates of the other vertex of the 2D bounding box, and (x1, y1) and (x2, y2) are the diagonal points.

[0067] The ROI filtering module's core function is to receive the category confidence scores and 2D bounding box coordinates of all suspected obstacles output by the ECA-YOLO Nano improved network inference module (i.e., the ECA-YOLONano object detection model). Based on a preset ROI (Region of Interest), it filters the 2D bounding boxes, retaining only those whose center point is within the ROI and whose category confidence score is greater than a set confidence threshold. These are considered valid target boxes, and the module outputs their coordinates, eliminating invalid detection areas and reducing computational overhead for subsequent modules. Specifically, it pre-reads the vertex coordinates of the "main lane" polygon (half rectangular or trapezoidal) defined by maintenance personnel in the local configuration file. It then compares the center point coordinates of all suspected obstacle 2D bounding boxes output by the ECA-YOLO Nano improved network inference module with the preset ROI region. Target boxes whose center point coordinates fall outside the ROI region are discarded, and only those whose center point coordinates are within the ROI region are retained as valid target boxes. The filtered valid target box coordinates are then transmitted to the IoU static verification module. Preferably, in another embodiment, the front-end edge detection device draws the Region of Interest (ROI) of the vehicle lane, generates an ROI coordinate configuration file, and stores it locally on the front-end edge detection device (in a low-power edge computing board). Preferably, in another embodiment, the ROI region is the main driving lane. When filtering two-dimensional bounding boxes, the ROI filtering module compares the center point coordinates of all two-dimensional bounding boxes of suspected road debris output by the ECA-YOLO Nano improved network inference module with the spatial position of the main driving lane, directly discarding target boxes whose center points fall outside the ROI region such as the emergency lane, the central median, or the shoulder, and retaining only two-dimensional bounding boxes whose center points are located within the main driving lane as valid target boxes.

[0068] The IoU (Intersection over Union) static verification module is the core module for determining static debris. It abandons complex multi-target trajectory tracking techniques such as Re-ID and Kalman filtering, employing a pure geometric intersection method to distinguish between moving and static targets. By calculating the IoU of valid target boxes in different video sampling frames, when the IoU value consistently exceeds a set threshold and this high overlap persists for more than a set time threshold, it is determined to be a road debris event or an illegal parking event. An alarm trigger command and the coordinate data of the valid target box are output, thus achieving static debris determination. This simplified determination logic is adaptable to the operational needs of low-computing-power edge devices.

[0069] The image rendering module is the system's alarm and evidence collection module, forming a closed-loop link in the detection process. After receiving the alarm trigger command and the coordinate data of the valid target box, it marks the corresponding video sampling frame of the valid target box as a key frame, and encapsulates the alarm trigger command and the relevant lightweight data of the valid target box into an event alarm message for output, completing on-site evidence collection and lightweight data reporting without consuming a large amount of communication bandwidth.

[0070] The road debris detection method of this invention, applicable to low-computing-power edge devices, requires prior deployment of edge computing nodes and lightweight access to road video streams. Low-power edge computing boards are deployed on roadside monitoring poles on highways or urban expressways. The devices can retrieve high-definition video streams from local cameras via the RTSP protocol. Simultaneously, users can draw regions of interest (ROIs) for motor vehicle lanes in the management backend, masking interference information from non-detection areas.

[0071] like Figure 3 As shown, the road debris detection method of the present invention for low-computing-power edge devices includes the following steps:

[0072] I. Lightweight Target Detection

[0073] The improved ECA-YOLO Nano network inference module receives road video streams collected by front-end edge detection devices, samples the road video stream using downsampling, scales each sampled frame, and inputs it into the ECA-YOLO Nano object detection model. It outputs the coordinates (x1, y1, x2, y2) and class confidence scores of all suspected road debris. (x1, y1) are the coordinates of one vertex of the 2D bounding box, (x2, y2) are the coordinates of the other vertex, and (x1, y1) and (x2, y2) are the diagonal points. The ECA-YOLO Nano object detection model is obtained by embedding an ECA channel attention module at the output of each depthwise separable convolutional module in the later stages of feature extraction in the YOLO Nano backbone network. After attention mechanism processing, the improved ECA-YOLO Nano object detection model can output the class confidence scores and 2D bounding boxes of suspected obstacles in the image at a high frame rate with an increase of less than 0.1MB in size.

[0074] The improved ECA-YOLO Nano target detection model of this invention is based on the existing lightweight detection network YOLONano and the existing ECA high-efficiency channel attention module. It does not change the internal structure of the YOLO Nano network and the ECA module, but only achieves scene adaptation by embedding the ECA module at a specified position in YOLO Nano. The core objective is to accurately identify small abnormal objects in the driving lane from the complex road background under the premise of strictly controlling the computing power consumption, so as to provide effective detection targets for subsequent modules.

[0075] II. Area Filtering

[0076] Regional filtering can eliminate invalid detection regions and select valid detection regions, thereby reducing the computing power consumption of subsequent modules.

[0077] The ROI filtering module receives the class confidence and two-dimensional bounding box coordinates output by the ECA-YOLO Nano improved network inference module. Based on the preset ROI region, it filters the two-dimensional bounding boxes, retaining only those whose center point is located within the ROI region and whose class confidence is greater than the set confidence threshold as valid target boxes, and outputs the coordinate data of the valid target boxes.

[0078] The preset ROI region is set in the ROI coordinate configuration file. Preferably, the ROI region coordinates are the vertex coordinates of the main driving lane polygon, which can be set by the operation and maintenance personnel through the local configuration file.

[0079] When filtering the received 2D bounding boxes, the coordinates of the center point of all received 2D bounding boxes are compared with the spatial position of the polygon of the main driving lane. Invalid target boxes whose center point falls on the outside of the emergency lane, the central median strip, or the shoulder, which are not in the detection area, are directly discarded. Only the detection boxes whose center point is located in the main driving lane are retained as valid target boxes.

[0080] III. Verification using pure geometry at rest

[0081] The IoU static verification module performs static feature confirmation of debris based on simplified spatiotemporal IoU matching. It constructs a static candidate pool using the effective target boxes and records the initial timestamp. It calculates the IoU value of the effective target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, it is determined to be a road debris event, and an alarm trigger command and the coordinate data of the effective target box are output.

[0082] Specifically, the following steps are included:

[0083] 3-1) When a new valid target box is received, an initial timestamp is assigned to it, and the coordinates and initial timestamp of the valid target box are stored in a lightweight candidate dictionary; the lightweight candidate dictionary records the valid target box and its initial timestamp and latest timestamp.

[0084] The method for constructing a lightweight candidate dictionary is as follows: a lightweight dictionary object (Static_Candidates) is maintained in the running memory of the edge card as a static target candidate pool; when the static feature locking engine based on spatiotemporal IoU receives a new valid target box, it is assigned an initial timestamp, and the coordinates and initial timestamp of the valid target box are stored in the lightweight candidate dictionary.

[0085] 3-2) Preferably, to further reduce computational overhead, a simplified inter-frame comparison judgment logic is adopted, eliminating the need for frame-by-frame comparison of the video stream, and inter-frame matching is performed using a down-sampling method. In each video sampling frame, a new valid target box B is extracted, and the valid target boxes A stored in the lightweight candidate dictionary are traversed to calculate the IoU between the valid target box B and the valid target box A.

[0086] For example, the sampling frequency can be set to once every 10 frames in a 25FPS video stream (approximately 0.4 seconds / sample), or it can be adjusted to 3 times per second depending on the device's computing power. In subsequent video sampling frames, a new valid target box B is extracted, and the valid target boxes A already stored in the lightweight candidate dictionary are traversed. The geometric intersection-union ratio (IoU) between the new valid target box B and the existing valid target box A in the lightweight candidate dictionary is calculated. The area of ​​the intersection region between boxes A and B is... Calculate using the following formula:

[0087] in,( , ) represents the coordinates of the top-left corner of the valid target bounding box A. , ) represents the coordinates of the lower right corner of the valid target bounding box A. , ) represents the coordinates of the top-left corner of the valid target box B. , () represents the coordinates of the lower right corner of the valid target box B.

[0088] The geometric intersection-union ratio (IoU) is calculated using the following formula:

[0089] in, The area of ​​the valid target box A. The area of ​​the effective target box B.

[0090] 3-3) If the calculated IoU is greater than the set threshold (e.g., 0.85, the threshold can be adjusted according to the scene), it indicates that the two detection boxes (effective target box B and effective target box A) almost completely overlap, and the target has not moved in the physical space. At this time, the initial timestamp (t_start) of the effective target box B is used to update the latest timestamp (t_current) of the effective target box A. For dynamic targets such as high-speed vehicles, even if they are misidentified as obstacles, their positions will be significantly shifted in subsequent sampling frames, resulting in IoU=0. At this time, the effective target box is regarded as a "transient motion interference item" and automatically removed from the lightweight candidate dictionary, thereby avoiding false alarms.

[0091] 3-4) When the duration of a valid target box in the lightweight candidate dictionary, i.e., the latest timestamp (t_current) minus the initial timestamp (t_start), is greater than or equal to the set time threshold (t_alarm) (e.g., 10 seconds, the time threshold can be adjusted according to the safety requirements of the traffic scenario), the state machine triggers a flip, determines that there is a stationary dangerous road debris on the road, and outputs an alarm trigger command and the coordinate data of the valid target box.

[0092] IV. Evidence Collection and Reporting

[0093] After receiving the alarm trigger command and the coordinate data of the valid target box, the image rendering module marks the corresponding video sampling frame of the valid target box as a keyframe, and encapsulates the alarm trigger command and the relevant lightweight data of the valid target box into an event alarm message for output. The relevant lightweight data includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence, and keyframes, in addition to the coordinate data of the valid target box.

[0094] The front-end edge detection device extracts three high-definition keyframes—before, during, and after—the occurrence of litter (the moment of event confirmation) from the circular memory buffer of the intelligent capture camera. These frames are then annotated using a computer vision library, such as OpenCV (Open Source Computer Vision Library), and compressed (e.g., to JPEG format). This data, along with the front-end edge detection device identifier, location information, event timestamp, and category confidence level, is converted into lightweight text data (e.g., encapsulated as a JSON message). This data is then sent to the back-end cloud receiving server via a communication module, achieving a closed-loop processing of evidence collection and early warning. Alternatively, the data can be directly distributed to roadside broadcasting devices (e.g., C-V2X roadside RSU broadcasting devices) at a specified distance (e.g., two kilometers ahead) to warn following vehicles to slow down and avoid the area, completing the closed loop of edge-side road litter detection and early warning.

[0095] Specifically, the following steps are included:

[0096] 4-1) Evidence-gathering rendering

[0097] Upon receiving an alarm trigger command, a computer vision library, such as OpenCV (Open Source Computer Vision Library), is invoked to extract a high-resolution image of the moment of event confirmation from the circular memory buffer of the front-end edge detection device. Based on the received target bounding box coordinate data of the debris, a bounding box of the debris is drawn on the high-resolution image with a conspicuous marker (e.g., a yellow solid line), and a text label is attached above the bounding box (including the type of debris within the bounding box and the duration of the target box, such as "Alarm: Stationary Obstacle - Lasts 12 seconds"), forming an evidence image of the road debris.

[0098] 4-2) Lightweight data reporting

[0099] The road debris images obtained from the completed evidence rendering are compressed (e.g., into JPEG format). At the same time, information such as the front-end edge detection device identifier, location information, event timestamp, and category confidence level are encapsulated into a message (e.g., into a JSON format message). Through the communication module, the message is sent to the traffic management bureau's back-end cloud receiving server, or it can be directly sent to roadside broadcasting devices (e.g., C-V2X roadside RSU broadcasting devices) at a specified distance ahead (e.g., two kilometers) to remind vehicles behind to slow down and avoid them.

[0100] In some embodiments, certain aspects of the above-described techniques may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly implemented on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the above-described techniques. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices such as flash memory, cache, random access memory (RAM), or other non-volatile memory devices. The executable instructions stored on the non-transitory computer-readable storage medium may be source code, assembly language code, object code, or other instruction formats interpreted or otherwise executed by one or more processors.

[0101] Computer-readable storage media can include any storage medium or combination of storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media can include, but are not limited to, optical media (e.g., optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable storage media can be embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or universal serial bus-based (USB) flash memory), or coupled to a computer system via a wired or wireless network (e.g., network-accessible storage (NAS)).

[0102] Please note that not all activities or elements described in the general description above are essential, a particular activity or part of the apparatus may not be essential, and one or more further activities or included elements may be performed in addition to those described. Furthermore, the order in which the activities are listed does not necessarily represent the order in which they are performed. Moreover, these concepts have been described with reference to specific embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure as set forth in the following claims. Therefore, the specification and drawings are to be considered illustrative rather than restrictive, and all such modifications are included within the scope of this disclosure.

[0103] The benefits, other advantages, and solutions to problems have been described above with respect to specific embodiments. However, any benefits, advantages, solutions to problems, and any features that may lead to or make any benefit, advantage, or solution more apparent should not be construed as critical, essential, or essential features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and implemented in different but equivalent ways that would be apparent to those skilled in the art benefiting from the teachings herein. There is no intention to limit the details of the constructions or designs shown herein other than those described in the claims. Therefore, it is apparent that the specific embodiments disclosed above can be altered or modified, and all such changes are considered to be within the scope of the disclosed subject matter.

Claims

1. A method for detecting road debris suitable for low-computing-power edge devices, characterized in that, include: Lightweight target detection: Receives road video streams collected by front-end edge detection devices, samples the road video streams using down-sampling, scales each video sample frame and inputs it into the ECA-YOLO Nano target detection model, and outputs the coordinates of the two-dimensional bounding boxes and the category confidence of all suspected road debris; Region filtering: Based on the preset ROI region, the two-dimensional bounding boxes are filtered, and only two-dimensional bounding boxes whose center point is located within the ROI region and whose category confidence is greater than the set confidence threshold are retained as valid target boxes, and the coordinate data of the valid target boxes are output. Pure geometric static verification: Construct a static candidate pool using the effective target boxes and record the initial timestamp. Calculate the IoU value for the effective target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, output an alarm trigger command and the coordinate data of the effective target box. Evidence collection and reporting: After receiving the alarm trigger command and the coordinate data of the valid target box, the corresponding video sampling frame of the valid target box is marked as a key frame, and the alarm trigger command and the relevant lightweight data of the valid target box are encapsulated into an event alarm message and output. In addition to the coordinate data of the effective target box, the relevant lightweight data also includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence, and keyframes; The ECA-YOLO Nano target detection model is obtained by embedding an ECA module at the output of each depthwise separable convolutional module in the later stage of feature extraction in the YOLO Nano backbone network. The depthwise separable convolutional module contains at least two sets of depthwise separable convolution operations. The ECA module directly achieves cross-channel information aggregation through one-dimensional convolution with a window size of k, where k is the adaptive kernel size.

2. The method for detecting road debris suitable for low-computing-power edge devices according to claim 1, characterized in that, The specific steps of the pure geometric static verification include: 3-1) When a new valid target box is received, an initial timestamp is assigned to it, and the coordinates and initial timestamp of the valid target box are stored in a lightweight candidate dictionary; the lightweight candidate dictionary records the valid target box and its initial timestamp and latest timestamp; 3-2) In each video sampling frame, extract the new valid target box B, traverse the valid target boxes A stored in the lightweight candidate dictionary, and calculate the IoU between the valid target box B and the valid target box A; 3-3) If the calculated IoU is greater than the set threshold, update the latest timestamp of the valid target box A with the initial timestamp of the valid target box B; if IoU=0, remove the valid target box from the lightweight candidate dictionary. 3-4) When the duration of a valid target box in the lightweight candidate dictionary, i.e., the latest timestamp minus the initial timestamp, is greater than or equal to the set time threshold, it is determined that there is road debris on the road surface, and an alarm trigger command and the coordinate data of the valid target box are output.

3. The method for detecting road debris suitable for low-computing-power edge devices according to claim 1, characterized in that, Also includes: The weights of the trained ECA-YOLO Nano object detection model are converted into an INT8 quantized model using an inference framework.

4. The method for detecting road debris suitable for low-computing-power edge devices according to claim 1, characterized in that, The event alarm message is sent to the background cloud receiving server and / or the roadside broadcasting equipment at a specified distance ahead.

5. The method for detecting road debris suitable for low-computing-power edge devices according to any one of claims 1-4, characterized in that, The ROI area is the main driving lane.

6. The method for detecting road debris suitable for low-computing-power edge devices according to claim 5, characterized in that, The step of filtering the two-dimensional bounding boxes according to the preset ROI region, retaining only the two-dimensional bounding boxes whose center point is located within the ROI region as valid target boxes, and outputting the coordinate data of the valid target boxes includes: When filtering the two-dimensional bounding boxes, the center point coordinates of all two-dimensional bounding boxes suspected of being road debris are compared with the spatial position of the main driving lane. Target boxes whose center points fall outside the ROI area are directly discarded, and only two-dimensional bounding boxes whose center points are located within the main driving lane are retained as valid target boxes.

7. A road debris detection system suitable for low-computing-power edge devices, implementing the road debris detection method for low-computing-power edge devices as described in any one of claims 1-8, characterized in that, This includes front-end edge detection devices and back-end cloud receiving servers; The front-end edge detection device includes an intelligent capture camera, a low-power edge computing board, a circular memory buffer, and a communication module. The intelligent capture camera collects road video streams; The low-power edge computing board receives road video streams captured by a smart capture camera, samples the road video streams using down-sampling, scales each video sample frame and inputs it into the ECA-YOLO Nano target detection model, outputting the coordinates and category confidence scores of all suspected road debris bounding boxes; based on a preset ROI region, the two-dimensional bounding boxes are filtered, retaining only those whose center point is located within the ROI region and whose category confidence score is greater than a set confidence threshold as valid target boxes; A static candidate pool is constructed using the effective target boxes, and an initial timestamp is recorded. The IoU value of the effective target boxes in the video sampling frames is calculated at a set interval. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, it is determined that a road debris event has been triggered. The corresponding video sampling frame of the effective target box is marked as a keyframe, and an alarm trigger command and the coordinate data of the effective target box are output. The alarm trigger command and the relevant lightweight data of the effective target box are encapsulated into an event alarm message and output. In addition to the coordinate data of the effective target bounding box, the relevant lightweight data also includes, but is not limited to, the front-end edge detection device identifier, location information, event timestamp, category confidence, and keyframes; The circular memory buffer provides buffering for each video sampling frame and support for keyframe extraction. After receiving the event alarm message, the communication module reports the event alarm message to the background cloud receiving server and / or the roadside broadcasting equipment at a specified distance ahead. The backend cloud receiving server receives event alarm messages sent by the front-end edge detection device and displays lightweight data related to the event alarm messages.

8. The road debris detection system for low-computing-power edge devices according to claim 7, characterized in that, The low-power edge computing board runs an ECA-YOLO Nano improved network inference module, ROI filtering module, IoU static verification module, and image rendering module; each module interacts sequentially according to the data flow order to complete the entire process from road video stream input to event alarm message reporting; The ECA-YOLO Nano improved network inference module receives road video streams collected by front-end edge detection devices, samples the road video streams using down-sampling, scales each video sample frame and inputs it into the ECA-YOLO Nano target detection model, and outputs the coordinates of the two-dimensional bounding boxes and the category confidence of all suspected road debris. The ROI filtering module filters the two-dimensional bounding boxes according to the preset ROI region, retaining only the two-dimensional bounding boxes whose center point is located within the ROI region and whose category confidence is greater than the set confidence threshold as valid target boxes, and outputs the coordinate data of the valid target boxes. The IoU static verification module constructs a static candidate pool using the effective target boxes and records the initial timestamp. It calculates the IoU value for the effective target boxes in the video sampling frames at set intervals. When the IoU value is continuously greater than a set threshold and the duration exceeds a set time threshold, it outputs an alarm trigger command and the coordinate data of the effective target box. After receiving the alarm trigger command and the coordinate data of the valid target box, the image rendering module marks the corresponding video sampling frame of the valid target box as a key frame, and encapsulates the alarm trigger command and the coordinate data of the valid target box into an event alarm message for output.

9. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the steps of the road debris detection method for low-computing-power edge devices as described in any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the road debris detection method for low-computing-power edge devices as described in any one of claims 1-6.

Images