Highway abnormal event real-time early warning method driven by roadside edge calculation

Abstract

The invention discloses a highway abnormal event real-time early warning method driven by roadside edge calculation, and belongs to the field of video image detection. Manual monitoring of expressway anomalies is low in efficiency and poor in accuracy. Dynamically grouping the expressway monitoring video stream data; extracting a frame of picture in parallel from each group of obtained video streams at preset time intervals to form a corresponding group of new video streams; inputting a plurality of groups of new video streams into a pre-trained recognition model in parallel, outputting a multi-dimensional tensor by each frame of picture, removing a bounding box in the multi-dimensional tensor which does not meet the requirement, tracking the same vehicle in each frame of picture by adopting a DeepSort algorithm, distributing a unique ID for each vehicle, and obtaining a center point coordinate sequence of each ID in combination with a recognition result; the Euclidean distance between every two adjacent center point coordinates in the center point coordinate sequence of each ID is calculated, and if one Euclidean distance is smaller than a preset threshold value, the state of the vehicle corresponding to the ID is a parking state, and an alarm is given. The method is used for monitoring expressway anomalies.

Description

Technical Field

[0001] The present invention belongs to the field of video image detection. Background Art

[0002] With the development of transportation infrastructure and the rapid increase in the number of motor vehicles, the traffic flow on highways shows a continuous growth trend. The parking behavior on highways caused by abnormal events such as vehicle failures and traffic accidents not only affects the road traffic capacity but also may induce serious traffic safety accidents. How to achieve efficient and accurate detection of highway abnormal events and issue early warnings in a timely manner has become a key technical problem that needs to be solved urgently by current traffic management departments. However, the current identification of highway abnormal events still relies on manually viewing real-time monitoring videos and manually issuing early warnings, and the traditional manual monitoring method has many limitations. On the one hand, due to the long length of highway sections, large traffic volume, and wide monitoring coverage, in the manual method, it is necessary to rotate shifts continuously for 24 hours and simultaneously monitor multiple monitoring video screens, resulting in a large monitoring burden, low efficiency, and easy delay in early warning due to untimely response, which cannot meet the real-time requirements; on the other hand, relying on manual discovery of abnormal events in highway monitoring videos, due to subjective judgment errors or fatigue and other factors, false judgments and missed judgments are likely to occur, which cannot meet the accuracy requirements.

[0003] It can be seen that for the current problem of real-time early warning of highway abnormal events, it is necessary to overcome technical and management challenges such as wide monitoring range, large amount of data, strong manual dependence, and high response delay, focus on the real-time analysis and accurate identification of large-scale monitoring video streams, and at the same time need to improve the automation level to achieve automatic early warning and reduce manual dependence. Therefore, new technologies are urgently needed to solve the problem of real-time early warning of abnormal events for large-scale highway monitoring videos. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems of large monitoring burden, low efficiency, easy occurrence of false judgments and missed judgments, and inability to meet the accuracy requirements in the current identification of highway abnormal events relying on manual monitoring, and propose a real-time early warning method for highway abnormal events driven by roadside edge computing.

[0005] A real-time early warning method for highway abnormal events driven by roadside edge computing, the method includes the following steps:

[0006] Step 1, read the highway monitoring video stream data, dynamically group the video stream data, and obtain multiple groups of video streams;

[0007] Step 2, parallelly extract one frame of picture from each group of video streams at preset time intervals to form a corresponding new group of video streams;

[0008] Step 3: Input multiple groups of new video streams into the pre-trained recognition model in parallel. Each frame outputs a multi-dimensional tensor, which includes: the center coordinates of the vehicle bounding box in each frame, the vehicle category within each bounding box, and the confidence of each bounding box.

[0009] Step 4: Compare the confidence of each bounding box in each frame with the preset confidence range, discard the bounding boxes that do not meet the preset confidence range and the vehicles within them, obtain the processed frame, use the DeepSort algorithm to track the same vehicle in the processed frame, assign a unique ID to each vehicle, and combine the recognition results in Step 3 to obtain the center point coordinate sequence of each ID.

[0010] Step 5: Calculate the Euclidean distance between every two adjacent center point coordinates in the center point coordinate sequence of each ID. If one Euclidean distance is lower than the preset vehicle displacement pixel threshold, determine that the vehicle state corresponding to this ID is parked, and then alarm the monitoring center.

[0011] Preferably, Step 1 further includes: performing format decoding and conversion on multiple groups of video streams, and converting the binary data stream in the compressed format of multiple groups of video streams into a pixel matrix.

[0012] Preferably, Step 1 further includes: adjusting the resolution of multiple groups of video streams in the form of a pixel matrix to a preset resolution.

[0013] Preferably, in Step 3, the pre-trained recognition model uses the trained YOLO object detection model.

[0014] Preferably, Step 5 further includes: performing format encoding and conversion on the frame corresponding to the parking state, converting the pixel matrix of the frame corresponding to the parking state into a binary data stream in the compressed format, and transmitting it to the monitoring center.

[0015] Preferably, in Step 1, the highway monitoring video stream data is read from a monitoring camera.

[0016] Preferably, Steps 1 to 5 are implemented by a roadside edge computing device.

[0017] The beneficial effects of the present invention are:

[0018] The present invention proposes a real-time early warning device for highway abnormal events driven by roadside edge computing. The key points of this device are as follows. First, an edge computing architecture is adopted to sink the computing tasks to the roadside edge nodes near the cameras. Distributed edge computing is used to implement vehicle recognition, tracking, and parking detection on the edge side, and only alarm events are reported. Second, an efficient vehicle detection and tracking technology is adopted. The YOLO object detection algorithm is combined with the DeepSort object tracking algorithm. YOLO11 is used for real-time object detection, which can identify multiple targets in a single-frame image and has both high detection accuracy and high-speed inference ability. Then, DeepSort is used for multi-object tracking. Combining the appearance features and motion information of the detection boxes, it can robustly track the trajectories of multiple targets in the video sequence and effectively reduce the impact of target occlusion, overlap, and loss. Third, an efficient parallel processing technology is adopted. Through parallel technologies such as YOLO batch inference and single-process multi-thread, the detection task is accelerated, and the model is lightweighted through the TensorRT architecture for deployment on edge devices.

[0019] The core idea of the present invention is to use edge computing technology, deep learning algorithms, etc. to achieve real-time analysis of large-scale highway surveillance videos. Aiming at the problem that the early warning of abnormal events in highway surveillance videos currently highly relies on manual work, efficient object detection and tracking algorithms are used to achieve real-time recognition and tracking of vehicles in the surveillance video images, and a displacement pixel threshold of the vehicle within a certain number of frames is set to automatically perform parking detection and alarm, which can effectively improve the speed and accuracy of detection and early warning. At the same time, concentrating all surveillance videos on the computing center for processing has problems such as high computing pressure and high consumption of network bandwidth. Therefore, the present invention adopts the method of sinking multiple computationally intensive tasks such as reading video streams, object detection and tracking, and parking detection to roadside edge devices, and only uploading alarm events to the computing center. To achieve high detection and recognition accuracy as well as parking detection accuracy, the processing time per frame meets the requirements of real-time analysis, and then realizes real-time, low-latency, and efficient large-scale traffic surveillance and early warning. At the same time, it reduces the central computing load and bandwidth cost, which is of great significance for reducing manual dependence and improving the efficiency of highway traffic safety management.

[0020] The present invention uses an edge computing architecture, which can significantly reduce network bandwidth consumption, reduce the computing load of the remote computing center, thereby reducing data transmission delay, improving the real-time performance of the system, and effectively reducing the system deployment cost.

[0021] The present invention integrates vehicle detection and tracking algorithms, combines the YOLO object detection and DeepSort object tracking algorithms, and can realize continuous recognition and real-time tracking of vehicles in highway surveillance videos, thereby improving the early warning accuracy of abnormal events.

[0022] The present invention utilizes efficient parallel processing and model optimization, uses parallel processing technology to accelerate object detection and tracking, improves computational efficiency, reduces latency, and lightweight processes the model, thereby ensuring the stable operation of edge computing devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the principle of a real-time early warning method for highway abnormal events driven by roadside edge computing;

[0024] Figure 2 It is a flowchart of a real-time early warning method for highway abnormal events driven by roadside edge computing. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0026] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0027] Next, the present invention will be further described in conjunction with the accompanying drawings and specific embodiments, but it is not a limitation of the present invention.

[0028] Embodiment:

[0029] A real-time early warning method for highway abnormal events driven by roadside edge computing, the method includes the following content:

[0030] Step 1, read the highway surveillance video stream data, dynamically group the video stream data, and obtain multiple groups of video streams;

[0031] Step 2, parallel extract one frame of picture from each group of video streams at a preset time interval to form a corresponding new group of video streams;

[0032] Step 3, parallel input multiple groups of new video streams into the pre-trained recognition model, and each frame of picture outputs a multi-dimensional tensor, and the multi-dimensional tensor includes: the center coordinates of the vehicle bounding box in each frame of picture, the vehicle category in each bounding box, and the confidence of each bounding box;

[0033] Step 4: Compare the confidence level of each bounding box in each frame of the video with the preset confidence range, discard the bounding boxes and the vehicles inside them that do not meet the preset confidence range, obtain the processed video frames, use the DeepSort algorithm to track the same vehicle in each processed video frame, assign a unique ID to each vehicle, and combine the recognition results in Step 3 to obtain the sequence of center point coordinates for each ID.

[0034] Step 5: Calculate the Euclidean distance between every two adjacent center point coordinates in the sequence of center point coordinates for each ID. If one of the Euclidean distances is lower than the preset vehicle displacement pixel threshold, determine that the vehicle state corresponding to this ID is parked, and then send an alarm to the monitoring center.

[0035] Specifically, there are many vehicles in each frame of the video, and each vehicle is circled by a bounding box. However, due to problems such as shadow occlusion, the vehicle appearance may sometimes cause inaccurate vehicle positioning and inappropriate bounding box ranges. Therefore, by setting the preset confidence range, the bounding boxes with inaccurate positioning or misclassification are filtered out to ensure that the recognized vehicle information is what is needed.

[0036] Figure 1 It is an overall architecture based on edge computing. The monitoring camera group is responsible for collecting raw data, the roadside edge computing device group completes real-time analysis, and the monitoring center makes decisions, forming a collaborative perception system in the "perception - decision - control" closed-loop paradigm. Multiple perspective high-definition cameras covering the highway transmit the video stream to the edge computing nodes; the roadside edge computing devices, as the core nodes, are deployed in the adjacent areas of the monitoring cameras and directly receive and process the raw video stream. The edge computing group is equipped with high-performance embedded devices, with a YOLO object detection model accelerated by TensorRT and a DeepSort multi-object tracking algorithm built-in to achieve real-time analysis of the video stream and detection of abnormal events; the monitoring center receives the structured alarm information uploaded by the edge nodes and takes timely measures to respond.

[0037] Figure 2 It is a flowchart of the real-time early warning algorithm for abnormal events in the monitoring video of edge devices. A multi-stage algorithm process is used to achieve real-time detection of abnormal vehicle parking. Through the preprocessing of the input video stream, a YOLO model accelerated by TensorRT is used to batch identify vehicle targets, then the DeepSort algorithm is used to achieve multi-object tracking, and finally, based on the displacement threshold, the stationary state is determined to trigger an abnormal alarm and report it to the monitoring center.

[0038] The DeepSort algorithm is a multi-object tracking algorithm (MOT) based on object detection. The main steps of MOT are:

[0039] 1. Given the initial frame of the video; 2. Run object detection algorithms, such as YOLO, Faster R-CNN, etc., to detect each frame of the video and obtain the detection bounding boxes; 3. Crop the image according to the detection bounding boxes to obtain the detection targets, and then extract features (appearance features or motion features) from the targets in turn; 4. Calculate the similarity matrix between two consecutive frames based on the extracted features; 5. Perform data association and assign target IDs to each object.

[0040] In step 2, to balance real-time performance and computational overhead, a frame sampling strategy and batch processing optimization are adopted to significantly reduce the number of processed frames without affecting the video analysis effect, enabling multi-frame synchronous inference and improving GPU utilization.

[0041] Further defined, step 1 also includes: performing format decoding and conversion on multiple groups of video streams, converting the binary data streams in the compressed format of the multiple groups of video streams into pixel matrices.

[0042] Specifically, the original data required for the YOLO model to input is a pixel matrix (image pixel data frame by frame), while the surveillance video is generally a binary data stream in compressed format, that is, the purpose of decoding is to convert the compressed data into a pixel matrix (such as an image frame).

[0043] Further defined, step 1 also includes: adjusting the resolution of multiple groups of video streams in the form of pixel matrices to a preset resolution.

[0044] Specifically, there are two purposes for resolution adjustment: one is to meet the input image size required by the recognition model, and the other is to reduce the computational complexity and balance accuracy and speed.

[0045] Further defined, in step 3, the pre-trained recognition model uses the trained YOLO object detection model.

[0046] Further defined, step 5 also includes: performing format encoding and conversion on the screen corresponding to the parking state, converting the pixel matrix of the screen corresponding to the parking state into a binary data stream in compressed format, and transmitting it to the monitoring center.

[0047] Specifically, in order to store the visualization results for future use, it is converted into other video formats.

[0048] Further defined, in step 1, highway surveillance video stream data is read from a surveillance camera.

[0049] Specifically, as Figure 1 shown, surveillance camera is used to monitor video stream data.

[0050] Further defined, steps 1 to 5 are implemented using roadside edge computing devices.

[0051] Specifically, asFigure 1 As shown, a roadside edge computing device is used to determine whether the vehicle state is parked.

[0052] Although the present invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely examples of the principles and applications of the present invention. Therefore, it should be understood that many modifications can be made to the exemplary embodiments, and other arrangements can be designed, as long as they do not depart from the spirit and scope of the present invention as defined by the appended claims. It should be understood that different dependent claims and the features described herein can be combined in a manner different from that described in the original claims. It should also be understood that the features described in connection with a single embodiment can be used in other described embodiments.

Claims

1. A real-time early warning method for highway abnormal events driven by roadside edge computing, characterized in that, The method includes the following steps: Step 1: Read the highway surveillance video stream data, dynamically group the video stream data, and obtain multiple groups of video streams; Step 2: Extract one frame from each group of video streams at a preset time interval in parallel to form a corresponding new group of video streams; Step 3: Input multiple groups of new video streams into the pre-trained recognition model in parallel. Each frame outputs a multi-dimensional tensor, and the multi-dimensional tensor includes: the center coordinates of the vehicle bounding box in each frame, the vehicle category within each bounding box, and the confidence of each bounding box; Step 4: Compare the confidence of each bounding box in each frame with the preset confidence range, discard the bounding boxes and the vehicles inside that do not meet the preset confidence range to obtain the processed frame. Use the DeepSort algorithm to track the same vehicle in the processed frame, assign a unique ID to each vehicle, and combine the recognition results in Step 3 to obtain the sequence of center point coordinates for each ID; Step 5: Calculate the Euclidean distance between every two adjacent center point coordinates in the sequence of center point coordinates for each ID. If one Euclidean distance is lower than the preset vehicle displacement pixel threshold, determine that the vehicle state corresponding to this ID is parked, and then alarm the monitoring center.

2. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 1, wherein, Step 1 also includes: performing format decoding and conversion on multiple groups of video streams to convert the binary data stream in the compressed format of multiple groups of video streams into a pixel matrix.

3. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 2, wherein Step 1 also includes: adjusting the resolution of multiple groups of video streams in the form of a pixel matrix to the preset resolution.

4. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 1, characterized in that, In Step 3, the pre-trained recognition model uses the trained YOLO object detection model.

5. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 1, characterized in that Step 5 also includes: performing format encoding and conversion on the frame corresponding to the parking state to convert the pixel matrix of the frame corresponding to the parking state into a binary data stream in the compressed format and transmit it to the monitoring center.

6. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 1, wherein In Step 1, read the highway surveillance video stream data from the surveillance camera.

7. The real-time early warning method for highway abnormal events driven by roadside edge calculation according to claim 1, characterized in that Steps 1 to 5 are implemented using a roadside edge computing device.

Images