Motor / non-motor vehicle illegal converse motion behavior detection method

Abstract

The invention discloses a motor / non-motor vehicle illegal converse motion behavior detection method. The method comprises the following steps: acquiring real-time scene video stream data by adopting video stream acquisition equipment; aiming at a first frame video stream image of the collected scene video stream data, a dynamic 3D lane line segmentation detection algorithm is adopted to carry out lane line extraction and segmentation, a reference driving area is divided, and a correct reference driving direction in the reference driving area is calculated; tracking all maneuvering / non-motor vehicle targets by using a DeepSort real-time stream target tracking algorithm; pixel motion curve tracks of all maneuvering / non-motor vehicle targets in a period of time are fitted into a motion track straight line, a target motion direction is obtained, then an included angle between the target motion direction and a reference correct driving direction is calculated, and whether a target vehicle in a driving area has a retrograde motion behavior or not is judged through a self-adaptive threshold value. The method solves the problem of low detection precision of illegal converse driving behaviors of motor vehicles / non-motor vehicles at present.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle driving detection, and particularly to a method for detecting illegal reverse driving behaviors of motor / non-motor vehicles. Background Art

[0002] In the field of smart city construction, the detection of illegal reverse driving of traffic vehicles, including motor / non-motor vehicles, is a very important part of the smart traffic direction. With the continuous development of computer vision technology, real-time detection and tracking solutions for motor / non-motor vehicles through deep learning algorithms are becoming more and more mature. However, there are still great difficulties in understanding, judging, and automatically recording their illegal reverse driving behaviors in traffic. In the early stage, lane lines were detected to determine the driving area and preset the correct driving direction, and at the same time, the driving direction of motor vehicles was obtained through vehicle detection and tracking algorithms to judge whether there were illegal reverse driving behaviors. This method depends on the detection accuracy of lane lines, and the early lane line detection methods have low detection accuracy in the case of long shooting distances and complex shooting scenes, resulting in a high omission and error rate of illegal reverse driving judgment.

[0003] Currently, there is no fully automatic judgment scheme for illegal reverse driving of motor / non-motor vehicles that determines the driving area and reference driving direction based on dynamic 3D lane line detection methods. Therefore, it is necessary to develop a scheme for realizing high-precision detection of illegal reverse driving of motor / non-motor vehicles in complex road scenes and complex shooting scenes. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for detecting illegal reverse driving behaviors of motor / non-motor vehicles. This method combines interactive and fully automatic detection schemes, which can effectively solve the problem of low detection accuracy of current illegal reverse driving behaviors of motor / non-motor vehicles, and provide a full-process solution from illegal reverse driving behavior detection to behavior recording.

[0005] The purpose of the present invention is achieved through the following technical solutions:

[0006] A method for detecting illegal reverse driving behaviors of motor / non-motor vehicles, the method comprising:

[0007] Step 1: Use a video stream acquisition device with a long focal length shooting ability and an image resolution higher than 5 million pixels to collect real-time scene video stream data;

[0008] Step 2: For the image data taken at night, use a double-branch synchronous structure to enhance the night image data to improve the detection, segmentation, and tracking accuracy of motor / non-motor vehicles in the video stream data;

[0009] Step 3: For the first-frame video stream image of the collected scene video stream data, use a dynamic 3D lane line segmentation detection algorithm to extract and segment lane lines;

[0010] Step 4: Divide the reference driving area according to the classification results of different types of lane lines obtained in Step 3, and calculate the correct reference driving direction within the reference driving area;

[0011] Step 5: Use the object detection algorithm to detect newly emerging objects in the scene, and use the DeepSort real-time streaming object tracking algorithm to track all motor / non-motor vehicle objects to obtain the pixel motion curve trajectory of the objects over a period of time;

[0012] Step 6: Fit the pixel motion curve trajectories of all motor / non-motor vehicle objects over a period of time into a straight motion trajectory to obtain the object motion direction, and then calculate the angle between the object motion direction and the reference correct driving direction, and determine whether there is a reverse behavior of the target vehicle in the driving area through an adaptive threshold.

[0013] An electronic device includes a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method.

[0014] A computer storage medium stores multiple instructions, and the instructions are suitable for being loaded and executed by a processor to execute the method.

[0015] It can be seen from the technical solutions provided by the present invention above that the above method combines the interactive and fully automatic detection solutions, can effectively solve the problem of low detection accuracy of illegal reverse behaviors of current motor / non-motor vehicles, and provides a full-process solution from illegal reverse behavior detection to behavior recording. Description of the Drawings

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0017] Figure 1 Schematic flow chart of the method for detecting illegal reverse behaviors of motor / non-motor vehicles provided by the embodiments of the present invention;

[0018] Figure 2 Schematic diagram of the class vision transformer structure described in the embodiments of the present invention. Detailed Embodiments

[0019] The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments, which does not constitute a limitation to the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0020] As Figure 1 shown in the schematic flowchart of the method for detecting illegal reverse driving behavior of motor / non-motor vehicles provided by the embodiment of the present invention, the method includes:

[0021] Step 1: Use a video stream acquisition device with the ability to shoot in a long focal length range and an image resolution higher than 5 million pixels to collect real-time scene video stream data;

[0022] In this step, to adapt to complex actual scenes, the data acquisition device needs to have the ability to shoot in a long focal length range. At the same time, to achieve higher-precision detection efficiency, a camera with a resolution higher than 5 million pixels is required.

[0023] For application scenarios where the shooting target distance is greater than 50 m, a camera with a focal length of 200 mm or more and an image resolution higher than 8 million pixels is used for data acquisition;

[0024] For application scenarios where the shooting target distance range is 5 m - 10 m, a camera with a focal length of 50 mm - 100 mm or more and an image resolution higher than 5 million pixels is used for data acquisition, and it is required that the camera has the ability to collect and upload real-time stream images.

[0025] Step 2: For the image data taken at night, a dual-branch synchronous structure is used to enhance the night image data to improve the detection, segmentation, and tracking accuracy of motor / non-motor vehicles in the video stream data;

[0026] In this step, the process of using the dual-branch synchronous structure to enhance the night image data is specifically as follows:

[0027] The global branch structure uses a lightweight encoder to extract global-level image features from high-dimensional resolution image data for global prediction, and then uses a two-layer feedforward network to output the global color optimization coefficient and gamma optimization coefficient of the image to perform corresponding enhancement on each pixel of the image;

[0028] The local branch structure is used to enhance the local details of the image. The local branch uses a vision transformer-like structure, such as Figure 2 shown in the schematic diagram of the vision transformer-like structure described in the embodiment of the present invention, Figure 2In Chinese: EmbeddedPatches inputs the input data into the network in different blocks; Norm is a normalization module that normalizes the data to between 0 and 1; Muti-HeadAttention is a standard attention mechanism module; MLP is a multi-layer artificial neural network structure. Different from the traditional U-Net network structure that performs downsampling operations on the input image, this structure can keep the input resolution unchanged, thereby retaining the complete input image and avoiding the loss of input information caused by downsampling. At the same time, this lightweight structure design can replace the deep convolutional self-attention mechanism to achieve pixel-by-pixel image data enhancement on the image, so as to improve the subsequent processing accuracy.

[0029] The above dual-branch synchronous structure has been proven effective in enhancing the image data collected at night in experiments, so as to improve the detection, segmentation, and tracking accuracy of motor / non-motor vehicles in the video stream image data.

[0030] Step 3: For the first-frame video stream image of the collected scene video stream data, use a dynamic 3D lane line segmentation detection algorithm to extract and segment the lane lines;

[0031] In this step, first predict the local rough lane shape and obtain the local lane line prediction direction, and then generate the corresponding lane lines along the lane line prediction direction;

[0032] Adopt a pyramid feature integration method for the lane line detection task, sample from features at different levels to obtain a global view of the lane lines, and then use a basic classification model to classify the detected lane lines, including the central lane line (usually a double yellow line) and the boundary lane line (usually a single white solid line).

[0033] Step 4: Divide the reference driving area according to the classification results of different types of lane lines obtained in Step 3, and calculate the correct reference driving direction within the reference driving area;

[0034] In this step, after the lane line segmentation detection, an outer bounding rectangle of the lane line will be obtained. By the geometric relationship between the long and short sides of the outer bounding rectangle, combined with the preset basic rules of driving on the left or right, calculate the correct reference driving direction within the reference driving area. If it is preset to drive on the right after obtaining the central lane line, then forward in the driving area on the right side of the central lane line is the correct reference driving direction;

[0035] Among them, to cope with complex scene conditions, a scheme for presetting the correct reference driving direction is also compatible.

[0036] Step 5: Use an object detection algorithm to detect newly emerging objects in the scene, and use the DeepSort real-time stream object tracking algorithm to track all motor / non-motor vehicle objects to obtain the pixel motion curve trajectory of the objects over a period of time;

[0037] In this step, the YOLOv11 algorithm is specifically used as the object detection algorithm to identify the position of the object in each frame of the scene video stream and mark the bounding box (x, y, w, h) of the object, which are the upper left coordinate x, the upper left coordinate y, the width w of the object box, and the height h of the object box of the object bounding box respectively, so as to determine the exact position of the object in the video stream image. The main advantage of this algorithm is that it can achieve real-time detection on edge computing;

[0038] Based on the bounding box of the detected object, the DeepSort real-time stream object tracking algorithm is used as the object tracking model. This object tracking model combines deep learning feature extraction and traditional tracking algorithms. First, it extracts the features of the object based on the deep learning model, and then associates the objects in different frames through these features. At the same time, through the cross design of long and short frame intervals, it is ensured that even if the object is invisible in some frames or a certain segment of frames, the object can be re-identified through feature matching;

[0039] The object tracking model realizes tracking by maintaining a list of objects and the motion states of the objects. This list of objects is updated according to the addition of new objects and the departure of old objects, so as to detect newly emerging objects in real time and continuously track all motor / non-motor vehicle objects, and record their motion trajectory curves in the scene video stream. These motion trajectories are in pixels and represent the change of the pixel positions of motor / non-motor vehicle objects in the scene video stream image over time.

[0040] Step 6: Fit the pixel motion curve trajectories of all motor / non-motor vehicle objects within a period of time into a motion trajectory straight line to obtain the object motion direction, and then calculate the angle between the object motion direction and the reference correct driving direction, and judge whether the target vehicle in the driving area has a reverse behavior through an adaptive threshold.

[0041] In this step, specifically, the pixel motion curve trajectories of all motor / non-motor vehicle objects within a period of time are linearly fitted at the two-dimensional pixel points in the phase plane to obtain the object motion direction and normalize it to obtain a direction vector;

[0042] Among them, the direction vector is obtained through the least squares linear fitting, so as to obtain the angle between the object motion direction and the reference correct driving direction. If the angle exceeds the preset adaptive threshold, it is judged that the target vehicle in the driving area is illegally reversing, and the images of a fixed number of frames before and after the determination of reversing are synthesized into a video and saved for subsequent analysis and processing;

[0043] The adaptive threshold is flexibly adjusted according to different monitoring environments and requirements to improve the accuracy of reverse detection.

[0044] In complex roads, the included angle threshold between the correct driving direction of the vehicle and the actual driving direction of the vehicle is modified according to the angle between the road and the camera. For example, in the case where the shooting direction of the camera is parallel to the road direction, a threshold of 110 degrees is set to exclude the normal driving of the vehicle and the vehicle being perpendicular to the correct reference driving direction, so as to determine whether the vehicle is driving in reverse.

[0045] It should be noted that the content not described in detail in the embodiments of the present invention belongs to the prior art well-known to those skilled in the art.

[0046] An embodiment of the present invention further provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method.

[0047] An embodiment of the present invention further provides a computer storage medium. The computer storage medium stores multiple instructions, and the instructions are suitable for being loaded and executed by the processor to execute the method.

[0048] In summary, the method described in the embodiments of the present invention has the following advantages:

[0049] 1. The method of 3D lane line segmentation detection can better extract road lane lines, and based on this, generating a reference driving area and a reference correct driving direction can more accurately extract and classify lane lines. This method obtains a running speed higher than 200 FPS in the CULane test set, and at the same time, the F1 score reaches 75%, which proves the high efficiency, accuracy and robustness of this method in various complex scenarios;

[0050] 2. By retaining the scheme of manually presetting the reference correct driving direction, it can cope with more complex road conditions;

[0051] 3. Using the least squares method to fit the vehicle trajectory curve can obtain a result closer to the true driving direction of the vehicle;

[0052] 4. Save the video data of the detected illegal reversing vehicle for subsequent analysis and processing.

[0053] In addition, those of ordinary skill in the art can understand that all or part of the steps in implementing the method of the above embodiments can be completed by instructing relevant hardware through a program, and the corresponding program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a disk or an optical disc, etc.

[0054] As described above, it is only the preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims. The information disclosed in the background art part of this article is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as an admission or an indication in any form that this information constitutes the prior art known to those skilled in the art.

Claims

1. A method for detecting illegal reverse driving behavior of motor vehicles / non-motor vehicles, characterized in that: The method comprises: Step 1: Use a video stream acquisition device with a telephoto shooting capability and an image resolution higher than 5 million pixels to collect real-time scene video stream data; Step 2: For the image data taken at night, a dual-branch synchronization structure is used to enhance the night image data to improve the detection, segmentation and tracking accuracy of motorized / non-motorized vehicles in the video stream data; Step 3: For the first frame of the collected scene video stream data, a dynamic 3D lane line segmentation detection algorithm is used to extract and segment the lane line; Step 4: Divide the reference driving area according to the classification results of different types of lane lines obtained in step 3, and calculate the correct reference driving direction in the reference driving area; Step 5: Use the target detection algorithm to detect new targets in the scene, and use the DeepSort real-time stream target tracking algorithm to track all motorized / non-motorized vehicle targets to obtain the pixel motion curve trajectory of the target over a period of time; Step 6: Fit the pixel motion curve trajectories of all motorized / non-motorized vehicle targets within a period of time into motion trajectory straight lines to obtain the target motion direction, and then calculate the angle between the target motion direction and the reference correct driving direction, and use the adaptive threshold to determine whether the target vehicle in the driving area has reverse behavior.

2. The method for detecting illegal wrong-way behavior of motor vehicles / non-motor vehicles according to claim 1, characterized in that: In step 1, for application scenarios where the target distance is greater than 50m, a camera with a focal length of 200mm or more and an image resolution of more than 8 million pixels is used for data collection; For application scenarios where the target distance range is 5m-10m, a camera with a focal length of 50mm-100mm or above and an image resolution of more than 5 million pixels is used for data collection, and the camera is required to have the ability to collect and upload real-time streaming images.

3. The method for detecting illegal wrong-way behavior of motor vehicles / non-motor vehicles according to claim 1, characterized in that: In step 2, the process of enhancing the night image data using the dual-branch synchronization structure is as follows: The global branch structure uses a lightweight encoder to extract global image features from high-dimensional resolution image data, performs global prediction, and then uses a two-layer feedforward network to output the global color optimization coefficient and gamma optimization coefficient of the image to enhance the image pixel by pixel accordingly. The local branch structure is used to enhance the local details of the image. The local branch adopts a visual transformer-like structure, which can keep the input resolution unchanged, thereby retaining the complete input image and not causing input information loss due to downsampling. This lightweight structural design can replace the deep convolution self-attention mechanism to achieve pixel-by-pixel image data enhancement on the image to improve the accuracy of subsequent processing.

4. The method for detecting illegal wrong-way behavior of motor vehicles / non-motor vehicles according to claim 1, characterized in that: In step 3, firstly, the local rough lane shape is predicted, and the local lane line prediction direction is obtained, and then the corresponding lane line is generated along the lane line prediction direction; A pyramid feature integration method is adopted for lane detection tasks to sample features from different levels to obtain a global view of lane lines. Then a basic classification model is used to classify the detected lane lines, including central lane lines and boundary lane lines.

5. The method for detecting illegal wrong-way behavior of motor vehicles / non-motor vehicles according to claim 1, characterized in that: In step 4, after the lane line segmentation detection, the outer enclosing rectangular frame of the lane line will be obtained. Through the geometric relationship between the long and short sides of the outer enclosing rectangular frame, combined with the preset left-hand or right-hand driving basic rules, the correct reference driving direction in the reference driving area is calculated.

6. The method for detecting illegal wrong-way behavior of motor vehicles and non-motor vehicles according to claim 1, characterized in that: In step 5, the YOLOv11 algorithm is used as the target detection algorithm to identify the position of the target in each frame of the scene video stream and mark the target's bounding box (x, y, w, h), which are the upper left corner coordinate x, the upper left corner coordinate y, the target box width w and the target box height h of the target bounding box, so as to determine the exact position of the target in the video stream image; Based on the bounding box of the detected target, the DeepSort real-time streaming target tracking algorithm is used as the target tracking model. This target tracking model combines deep learning feature extraction and traditional tracking algorithms. First, the target features are extracted based on the deep learning model, and then these features are used to associate targets in different frames. At the same time, the cross-design of long and short frame intervals ensures that even if the target is not visible in some frames or a certain period of frames, the target can be re-identified through feature matching. The target tracking model achieves tracking by maintaining a target list and the target's motion status. The target list is updated according to the addition of new targets and the departure of old targets, so as to detect newly appeared targets in real time and continuously track all motorized / non-motorized vehicle targets, and record their motion trajectory curves in the scene video stream. These motion trajectories are in pixels, indicating the changes in the pixel position of the motorized / non-motorized vehicle targets in the scene video stream image over time.

7. The method for detecting illegal wrong-way behavior of motor vehicles / non-motor vehicles according to claim 1, characterized in that: In step 6, specifically, the pixel motion curve trajectories of all motorized / non-motorized vehicle targets within a period of time are linearly fitted at the two-dimensional pixel points of the phase plane to obtain the target motion direction, and normalized to obtain the direction vector; The direction vector is obtained by least squares straight line fitting, so as to obtain the angle between the target moving direction and the reference correct driving direction. If the angle exceeds the preset adaptive threshold, the target vehicle in the driving area is judged to be illegally driving in the wrong direction, and the images of a fixed number of frames before and after the judgment of the wrong direction are synthesized and saved for subsequent analysis and processing; The adaptive threshold is flexibly adjusted according to different monitoring environments and requirements to improve the accuracy of retrograde detection.

8. An electronic device, comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 7.

9. A computer storage medium, characterized in that: The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

Images