A real-time video stream vehicle reverse event detection method and system

Abstract

The application discloses a kind of real-time video stream vehicle reverse event detection method and system, which comprises the following steps: setting preset direction on frame chart;And the position of the virtual marker in the opposite direction of preset direction on frame chart is set to the vehicle to be detected θ;Based on frame chart to establish plane rectangular coordinate system;The coordinate distance of the projection of the vehicle to be detected in x axis in frame chart at any time is obtained from the origin of plane rectangular coordinate system d1, and the coordinate distance of the projection of virtual marker in x axis from the origin of plane rectangular coordinate system d2, when d1-d2<0, determine that suspected reverse event occurs to the vehicle to be detected.The application proposes a new vehicle driving state determination method for the motion characteristics of the vehicle to be detected in video stream: based on target tracking algorithm to extract the position change of the vehicle to be detected, according to the continuous position change result to reflect the current motion state of the vehicle to be detected.

Description

Technical Field

[0001] This invention belongs to the field of vehicle safety detection technology, specifically relating to a method and system for detecting vehicle reverse driving events in real-time video streams. Background Technology

[0002] Driving against the flow of traffic is a serious violation of traffic regulations. At best, it causes traffic congestion and reduces road capacity; at worst, it leads to traffic accidents and even fatalities. According to statistics from my country's traffic police department, most major traffic accidents each year are caused by drivers driving against the flow of traffic.

[0003] Traditional traffic video surveillance systems install surveillance cameras at important locations such as major urban traffic arteries, highways, and national roads, with human personnel monitoring road conditions in real time. However, with the emergence of massive amounts of video data, the shortcomings of manual monitoring are becoming increasingly apparent, such as visual fatigue, insufficient real-time performance, and high manpower consumption. Summary of the Invention

[0004] This invention provides a real-time video stream vehicle wrong-way driving event detection method and system. Addressing the requirements of video stream monitoring for fast, efficient, higher-precision, and more robust vehicle wrong-way driving event detection technology, this invention proposes a robust real-time video stream vehicle wrong-way driving event detection algorithm based on target detection algorithms and vehicle trajectory tracking algorithms. This algorithm can automatically detect whether a vehicle is driving in the wrong direction, reducing manual intervention.

[0005] This invention is achieved through the following technical solution:

[0006] On one hand, the present invention provides a method for detecting vehicle reverse driving events in real-time video streams, comprising the following steps: setting a preset direction on a frame; setting a virtual marker at a position θ away from the vehicle to be detected in the opposite direction of the preset direction on the frame, the maximum distance between the virtual marker and the vehicle to be detected being θ, and when the distance between the vehicle to be detected and the virtual marker is about to exceed θ, the coordinates of the virtual marker move relative to the vehicle to be detected so that the distance between the coordinates of the virtual marker and the vehicle to be detected remains at θ; establishing a Cartesian coordinate system based on the frame, the angle between the preset direction and the positive x-axis of the Cartesian coordinate system being [0, 90°); obtaining the distance d1 between the coordinates of the vehicle to be detected projected on the x-axis and the origin of the Cartesian coordinate system at any time, and the distance d2 between the coordinates of the virtual marker projected on the x-axis and the origin of the Cartesian coordinate system; when d1-d2<0, determining that the vehicle to be detected has experienced a suspected reverse driving event.

[0007] In some embodiments, during the process of obtaining the position of the vehicle to be detected, the center position of the vehicle to be detected is used as the position of the vehicle to be detected in the frame.

[0008] In some embodiments, the step of obtaining the center position of the vehicle to be detected includes: obtaining the detection location box of the vehicle to be detected in the frame image.

[0009]

[0010] in, To detect the x-coordinate of the top-left corner of the location bounding box; To detect the y-coordinate of the top-left corner of the location bounding box; Detect the x-coordinate of the bottom right corner of the location box; Detect the y-coordinate of the bottom right corner of the detection bounding box; obtain the center position of the vehicle to be detected in the frame image, where the center position is:

[0011]

[0012] In some embodiments, before setting a virtual marker at a position θ away from the vehicle to be detected in the opposite direction of a preset direction on the frame, the process includes the following steps: obtaining the status information of the vehicle to be detected.

[0013] In some embodiments, obtaining the status information of the vehicle to be detected includes: video stream initialization; obtaining the trajectory line of the vehicle to be detected; and obtaining the status information of the vehicle to be detected based on the trajectory line of the vehicle to be detected.

[0014] In some embodiments, obtaining the trajectory of the vehicle to be detected includes the following steps: obtaining the detection position of the j-th vehicle to be detected in the i-th frame of the video:

[0015]

[0016] in, To detect the x-coordinate of the top-left corner of the location bounding box; To detect the y-coordinate of the top-left corner of the location bounding box; Detect the x-coordinate of the bottom right corner of the location box; Detect the y-coordinate of the bottom right corner of the location bounding box; obtain the center position of the j-th vehicle to be detected in the i-th frame image, where the center position is:

[0017]

[0018] Obtain the trajectory line of the j-th vehicle to be detected, where the line connecting the center coordinates of the detection position boxes of consecutive frames of the j-th vehicle to be detected is the trajectory line of the vehicle to be detected.

[0019] In some embodiments, obtaining the status information of the vehicle to be detected based on its trajectory line includes the following steps:

[0020] Calculate the distance between the center position of the j-th vehicle detection bounding box in the (i+1)-th frame and the corresponding j-th vehicle detection bounding box in the (i+1)-th frame of the i-th image. If this distance is less than a certain threshold for a continuous time period t... If the vehicle is stopped, it is determined to be in motion; otherwise, it is in motion.

[0021] In some embodiments, after determining that the vehicle under test has experienced a suspected reverse driving event, the method further includes the following steps: timing the duration of the suspected reverse driving event, and determining that a confirmed reverse driving event exists when the timing exceeds a first preset duration.

[0022] In some embodiments, the state timing for the duration of a suspected wrong-way driving event on the vehicle under test includes the following steps: when the vehicle under test changes from a state with a suspected wrong-way driving event to a normal state, and then changes back to a state with a suspected wrong-way driving event within a second preset time, the current timing state is reset.

[0023] On the other hand, this application provides a real-time video stream vehicle wrong-way event detection system, including a video acquisition module, a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the real-time video stream vehicle wrong-way event detection method of any of the above embodiments.

[0024] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0025] (1) This invention proposes a new method for determining the driving state of a vehicle to be detected based on the motion characteristics of the vehicle to be detected in the video stream. The method is to extract the position change of the vehicle to be detected based on the target tracking algorithm and reflect the current motion state of the vehicle to be detected based on the continuous position change results.

[0026] (2) Based on target detection and tracking algorithms, this invention combines the position coordinates and motion state of the vehicle to be detected. According to the physical mechanism of the reverse driving event of the vehicle to be detected in the video stream, an unsupervised, real-time dynamic identification of reverse driving events is adopted. The algorithm is efficient, concise and more operable.

[0027] (3) The present invention uses a combination of physical model design and multiple mechanisms to determine the reverse driving event of the vehicle to be detected. The algorithm is relatively accurate and effective, with strong robustness and higher accuracy. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart illustrating the real-time video stream vehicle wrong-way driving event detection method provided in an embodiment of the present invention;

[0030] Figure 2 This is a flowchart illustrating the steps for acquiring a vehicle to be detected based on real-time video, provided for different embodiments of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0032] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., used to indicate the orientation or positional relationship are based on the orientation or positional relationship shown in the accompanying drawings, or the orientation or positional relationship that the product of this invention is usually placed in during use. They are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0033] Furthermore, the use of terms such as "horizontal" and "vertical" in the description of this invention does not imply that the components are required to be absolutely horizontal or suspended, but rather that they can be slightly tilted. For example, "horizontal" simply means that its direction is more horizontal than "vertical," and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0034] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "set," "install," "connect," and "link" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0035] The terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or device.

[0036] On the one hand, embodiments of this application provide a method for detecting vehicle reverse driving events in real-time video streams, including the following steps:

[0037] S10. Based on real-time video, obtain the status information of the vehicles to be detected. S10 mainly involves classifying the vehicles in the real-time video, distinguishing between vehicles requiring reverse driving detection and those not, to improve the system's computational efficiency. In a specific example, this is mainly achieved by determining whether the vehicle is in motion. If the vehicle is stationary, it is filtered out, and the subsequent reverse driving detection process is not performed.

[0038] In some embodiments, S10 may specifically include the following steps:

[0039] S101, Video Stream Initialization. In S101, the region of each frame in the real-time video can be divided under any task, and the position coordinates of the selected region under that task can be obtained to facilitate the detection of vehicles driving in the wrong direction in different regions for different tasks. In a specific example, the selected region can be a part of the corresponding frame image in the real-time video or all regions. On the other hand, video stream initialization can also initialize and set parameters needed for vehicle driving in the detection of wrong direction events, such as distance thresholds between vehicles, time thresholds, etc., which will be used later. Other parameters, such as region attributes, can also be initialized to facilitate data reading and saving.

[0040] S102. Obtain the vehicle trajectory lines. In S102, based on task requirements, such as all vehicles, some vehicles, or specified vehicles within a selected area, the trajectory lines of these vehicles are obtained.

[0041] For example, obtaining the trajectory of a vehicle may specifically include the following steps:

[0042] S1021. Obtain the detection location of the j-th vehicle in the i-th frame of the real-time video. This detection location can be a bounding box that includes the j-th vehicle. The i-th frame and the j-th vehicle are generic terms and can be any frame or any vehicle in the image.

[0043] For example, the location bounding box is illustrated using a rectangle. A rectangle centered on the j-th vehicle in the i-th frame is obtained. The four coordinate points of this rectangle are primarily detected to obtain the detection location information of the j-th vehicle in the i-th frame.

[0044]

[0045] in, To detect the x-coordinate of the top-left corner of the location bounding box; To detect the y-coordinate of the top-left corner of the location bounding box; Detect the x-coordinate of the bottom right corner of the location box; The detection location is determined by the y-coordinate of the bottom right corner of the bounding box. The detection locations of the j-th vehicle in consecutive frames, including the i-th frame, are connected to form the vehicle's trajectory. The number of consecutive frames can be set as needed.

[0046] For example, when connecting the detection locations, the center position of the obtained detection location information of the j-th vehicle can be used as a reference, and the center position of the j-th vehicle in the i-th frame image is:

[0047]

[0048] Then, the center positions of the j-th vehicle in consecutive frames, including the i-th frame, are connected to form the vehicle's trajectory. Comparatively, the trajectory obtained by connecting the center positions provides a more intuitive and accurate representation of the vehicle's trajectory. In other examples, a point on the edge of the vehicle's bounding box can also be used as a reference, such as a corner point of the rectangle, to connect and form the trajectory.

[0049] S103. Obtain vehicle status information based on the vehicle's trajectory. For example, taking the j-th vehicle as an example, after obtaining the trajectory of the j-th vehicle, calculate the distance between the detection position of the vehicle in the (i+1)-th frame and the detection position of the vehicle in the i-th frame. Within a continuous video duration t, the calculated distance between adjacent frames is always less than a certain threshold. If the vehicle is determined to be stationary, and if, within a continuous video duration t, there exists an image with a distance greater than or equal to the threshold θ between two adjacent frames, then the vehicle is determined to be in motion. When the vehicle is in motion, it is identified as the vehicle to be detected. Alternatively, in other embodiments, vehicle determination can be performed using other methods, such as directly obtaining the vehicle's status information.

[0050] S20. Set a preset direction on the frame. In S20, the frame is any frame or multiple frames in the real-time video. The preset direction can be the normal driving direction of the vehicle to be detected. The preset direction is used as the reference direction for the subsequent reverse driving judgment process.

[0051] S30. A virtual marker is placed at a position θ away from the vehicle to be detected in the opposite direction of the preset direction. The virtual marker is displayed on the real-time video in the form of coordinates. The maximum distance between the virtual marker and the vehicle to be detected is θ. The coordinates of the virtual marker can move relative to the vehicle to be detected, ensuring that the distance between the virtual marker and the vehicle never exceeds θ. When the distance between the vehicle and the virtual marker is about to exceed θ, the coordinates of the virtual marker move relative to the vehicle to be detected, ensuring that the distance between the virtual marker and the vehicle remains at θ. In S30, after the initial moment, when the angle between the movement direction of the vehicle to be detected and the preset direction is 0-90°, it is equivalent to the vehicle dragging the virtual marker via a retractable rope. When the angle between the movement direction of the vehicle to be detected and the preset direction is outside the 0-90° range, the vehicle moves towards the virtual marker, and the coordinates of the virtual marker remain unchanged until the distance between the vehicle and the virtual marker reaches the maximum distance θ again. In S30, θ should be set large enough to avoid judging a suspected reverse driving event within one frame after the initial moment. This would cause the false alarm rate of event detection to increase due to the position drift of the vehicle detection. Therefore, it is necessary to increase the redundancy space of θ. Therefore, the recommended minimum value should be the movement pixel distance of the vehicle in one frame.

[0052] S40. Establish a Cartesian coordinate system based on the frame image, with the angle between the preset direction and the positive x-axis of the Cartesian coordinate system being [0, 90°].

[0053] S50. Obtain the distance d1 between the coordinates of the vehicle to be detected projected on the x-axis and the origin of the Cartesian coordinate system in the frame at any time, and the distance d2 between the coordinates of the virtual marker projected on the x-axis and the origin of the Cartesian coordinate system.

[0054] S60. When d1 - d2 < 0, it is determined that the vehicle under test has experienced a suspected reverse driving event. Initially, the distance between the vehicle under test and the virtual marker is at its maximum distance θ. When the vehicle under test moves in the preset direction (the normal driving direction), the virtual marker will follow the vehicle in that direction. Therefore, at time t1, the distance d2 that the virtual marker has moved is equal to the distance d1 that the vehicle under test has moved. t =d1-d2, if 0≤d t <+θ, at time t+1 d t+1The maximum value can only reach +θ. At time t+1, due to the contraction (or bending) of the rope, d t+1 <0 means the distance between the projection of the vehicle's center position onto the origin and the origin. The distance between the projection of the virtual marker's center position in the preset direction and the origin. The difference between the two satisfies The conditions for determining a suspected case of reverse driving have been met.

[0055] In a specific example, a Cartesian coordinate system can be established with any corner of the frame as the origin, and the coordinates of the projection of the center position of the vehicle to be detected onto ray l (parallel to the preset direction) at time t can be recorded. The distance from the origin is The coordinates of the projection of the virtual marker center onto ray l (parallel to the preset driving direction). The distance from the origin is The difference between the two distances is the trajectory projection distance. Clearly, at time t0, we have (Note that the + sign indicates that the direction is the same as the preset direction of the vehicle being tested, and if it is not, it indicates that the direction is opposite to the preset direction of the vehicle being tested.)

[0056] In some embodiments, all moving vehicles to be detected in the current frame can be acquired, and the motion parameters of each moving vehicle to be detected in the current frame can be acquired according to steps S10 to S60; and in each subsequent frame, the motion of these moving vehicles to be detected can be statistically analyzed, and it can be determined whether these vehicles to be detected are driving in the wrong direction.

[0057] S70. When a suspected wrong-way driving event occurs with the vehicle to be detected, the duration of this state is timed. When the time exceeds a first preset duration, a confirmed wrong-way driving event is determined. In S70, the first preset duration can be set according to the complexity of the detection environment. By setting a countdown mechanism, suspected wrong-way driving events appearing in consecutive frames within the countdown time are determined as wrong-way driving events. This effectively avoids false alarms of wrong-way driving events caused by incorrect detection of the vehicle to be detected or the vehicle's position drifting, thus effectively reducing the false alarm rate.

[0058] In some embodiments, in the current frame, according to step S10, the position information and motion parameters of all moving vehicles in the current frame are obtained. According to steps S20-S60, the vehicle trajectory projection distance of the physical model of all moving vehicles is calculated simultaneously (the distance between the coordinates of the vehicle projected on ray l and the origin, the distance between the coordinates of the virtual marker center projected on ray l and the origin, and the difference between the two distances), and those with trajectory projection distances less than 0 are recorded as suspected reverse driving events. In each subsequent frame, all moving vehicles in the current frame are counted. For each moving vehicle, a check is performed cyclically. If the ID of a moving vehicle appears in the suspected wrong-way driving event record, the countdown mechanism of step S70 is used for final judgment, and the recorded countdown parameters for suspected wrong-way driving events are updated. The countdown parameters are decremented by 1 for each consecutive frame. If the countdown parameter corresponding to a suspected wrong-way driving event is 0, a wrong-way driving event warning is issued. At this time, the virtual marker is moved to the center position of the vehicle that issued the warning in the current frame, and the countdown parameter is restored to its maximum value, waiting for the next wrong-way driving event warning. Otherwise, the ID of the vehicle involved in the suspected wrong-way driving event is added to the latest suspected wrong-way driving event record. For already recorded suspected wrong-way driving events, a penalty is applied, the countdown parameter is updated to its maximum value, and the countdown restarts. This method enables simultaneous detection of multiple vehicle wrong-way driving events with high computational efficiency.

[0059] S80. During the timekeeping process when a vehicle under inspection is suspected of driving in the wrong direction, if the vehicle changes from a state of suspected wrong-way driving to a normal state, and then changes back to a state of suspected wrong-way driving within a second preset time period, the current timekeeping state is reset. By setting a penalty mechanism, wrong-way driving events can be further constrained, the correct warning of wrong-way driving events can be further realized, and the robustness of the system can be improved.

[0060] On the other hand, this application provides a real-time video stream vehicle wrong-way event detection system, including a video acquisition module, a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, it implements the real-time video stream vehicle wrong-way event detection method of any of the above embodiments.

[0061] This application also provides a computer storage medium storing a computer program, which is loaded by a processor to execute the real-time video stream parking lot vehicle reverse driving event detection method of any of the above embodiments.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications or equivalent changes made to the above embodiments based on the technical essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A real-time video stream vehicle reverse event detection method, characterized in that, The method comprises the following steps: Setting a preset direction on a frame graph; Setting a virtual marker at a position on the frame graph which is θ away from the vehicle to be detected in the opposite direction of the preset direction, the maximum distance between the virtual marker and the vehicle to be detected being θ; when the distance between the virtual marker and the vehicle to be detected is about to exceed θ, the coordinate of the virtual marker moves relative to the vehicle to be detected so that the distance between the coordinate of the virtual marker and the vehicle to be detected remains θ; Establishing a plane rectangular coordinate system based on the frame graph, the angle between the preset direction and the positive direction of the x-axis of the plane rectangular coordinate system being [0, 90°); Obtaining the distance d1 between the coordinate of the projection of the vehicle to be detected on the x-axis and the origin of the plane rectangular coordinate system and the distance d2 between the coordinate of the projection of the virtual marker on the x-axis and the origin of the plane rectangular coordinate system at any time, When d1-d2<0, determining that the vehicle to be detected has a suspected reverse movement event; After determining that the vehicle to be detected has a suspected reverse movement event, the method further comprises the following steps: Timing the duration of the suspected reverse movement event of the vehicle to be detected, and determining that there is a certain reverse movement event when the timing exceeds a first preset duration; the timing of the duration of the suspected reverse movement event of the vehicle to be detected comprises the following steps: When the vehicle to be detected changes from the state of having a suspected reverse movement event to a normal state and then changes to the state of having a suspected reverse movement event again within a second preset duration, resetting the current timing state.

2. The real-time video stream vehicle reverse event detection method of claim 1, wherein, In the process of obtaining the position of the vehicle to be detected, the center position of the vehicle to be detected is taken as the position of the vehicle to be detected in the frame graph.

3. The real-time video stream vehicle retrograde event detection method of claim 2, wherein, The step of obtaining the center position of the vehicle to be detected comprises: Obtaining the detection position frame of the vehicle to be detected in the frame graph; wherein, is the detected position box upper left x coordinate; is the detected position box upper left y coordinate; is the detected position box lower right x coordinate; is the detected position box lower right y coordinate; Obtaining the center position of the vehicle to be detected in the frame graph, the center position being:

4. The real-time video stream vehicle retrograde event detection method of claim 3, wherein, Before setting the virtual marker at a position on the frame graph which is θ away from the vehicle to be detected in the opposite direction of the preset direction, the method comprises the following steps: obtaining the state information of the vehicle to be detected.

5. The real-time video stream vehicle retrograde event detection method of claim 4, wherein, The step of obtaining the state information of the vehicle to be detected comprises: Video stream initialization; Obtaining the trajectory line of the vehicle to be detected; Obtaining the state information of the vehicle to be detected based on the trajectory line of the vehicle to be detected.

6. The real-time video stream vehicle retrograde event detection method of claim 5, wherein, The step of obtaining the trajectory line of the vehicle to be detected comprises the following steps: Obtaining the detection position of the jth vehicle to be detected in the i-th frame image of the video; wherein, is the detected position box top-left x coordinate; is the detected position box top-left y coordinate; is the detected position box bottom-right x coordinate; is the detected position box bottom-right y coordinate; Obtaining the center position of the jth vehicle to be detected in the i-th frame image, the center position being: Obtaining the trajectory line of the jth vehicle to be detected, wherein the line connecting the center coordinates of the detection position frames of the jth vehicle to be detected in consecutive frames is the trajectory line of the vehicle to be detected.

7. The real-time video stream vehicle retrograde event detection method of claim 6, wherein, The step of obtaining the state information of the vehicle to be detected based on the trajectory line of the vehicle to be detected comprises the following steps: The distance between the center position of the jth to-be-detected vehicle detection position frame in the i+1th frame image and the corresponding jth to-be-detected vehicle detection position frame of the ith frame image in the i+1th frame image is calculated, and if the distance is less than a certain threshold φ for t consecutive time lengths, the to-be-detected vehicle is determined to be in a stop state, otherwise, the to-be-detected vehicle is determined to be in a running state.

8. A real-time video stream vehicle reverse event detection system, characterized in that, The real-time video stream vehicle reverse movement event detection method comprises a video acquisition module, a processor and a memory, the memory stores a computer program, and the computer program is executed by the processor to realize the real-time video stream vehicle reverse movement event detection method according to any one of claims 1 to 7.

Images