Intersection vehicle abnormal driving detection method based on machine vision detection and road passing region modeling

Abstract

The application relates to an intersection vehicle abnormal driving detection method based on machine vision detection and road passing area modeling, relates to the intelligent traffic technology field, and is characterized in that, in the real-time video picture of the electric police monitoring at the intersection of the urban road, a machine vision algorithm is used to detect the red and green light state and the spatial motion information of the vehicle; then, the road passing area is modeled based on perspective transformation, and a one-to-one mapping relationship between the video picture and the real road coordinates is obtained; whether the vehicle exists in the abnormal driving traffic events such as overspeed, abnormal parking, green light non-passing and red light running in the detection area is judged; more valuable road condition information can be provided for signal regulation and real-time scheduling of the command center; and under the development concept of the intelligent city construction, more diversified basic data can be provided for AI decision-making.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling. Background Technology

[0002] In recent years, with the rapid development of computer vision and deep learning technologies, intelligent traffic monitoring systems based on video analytics have gradually become a research hotspot. These systems can automatically identify and analyze traffic light status and vehicle movement information using real-time video data. Currently, most traffic sensing devices can accurately collect general traffic data, such as traffic flow, queue length, and vehicle saturation, providing important data for traffic light control. Through these conventional indicators, traffic management departments can optimize traffic light timing and alleviate some traffic congestion problems.

[0003] However, relying solely on routine traffic data cannot fully reflect the traffic conditions at intersections, as abnormal traffic events are also a significant cause of intersection congestion and traffic accidents. For example, abnormal behaviors such as speeding, improper parking, failing to use a green light, and running a red light often disrupt the stability of traffic flow, directly causing intersection congestion and even triggering serious traffic accidents. Therefore, in intelligent transportation systems, in addition to collecting routine traffic data, real-time detection of abnormal events is particularly necessary. Summary of the Invention

[0004] To overcome the shortcomings of the above technologies, this invention provides a method for accurately determining whether a vehicle is speeding, abnormally parking, failing to stop at a green light, or running a red light at an intersection within the detection area.

[0005] The technical solution adopted by this invention to overcome its technical problems is:

[0006] A method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling includes:

[0007] S1. Obtain video stream information from the intersection's electronic police camera;

[0008] S2. Preprocess the acquired video stream information to obtain... Image set of intersection road images , , For the preprocessed first Frame intersection road image, ;

[0009] S3. Based on the image set Calibration yields the vehicle detection area , No. Each lane area Red light violation monitoring area Light status detection area In the Frame intersection road image The nearest and far coordinate points are determined, and the first point is calculated. Frame intersection road image Middle lane width Actual distance ;

[0010] S4. Based on the vehicle inspection area Obtain the vehicle detection area image set According to the light status detection area Obtain the image set of the lamp state detection region ;

[0011] S5. Light status detection image set Visual classification is performed to obtain the lamp status information management structure. ;

[0012] S6. Image set of vehicle detection area Target detection and tracking are performed to obtain the vehicle information management structure. ;

[0013] S7. Based on nearby coordinates, distant coordinates, and lane width Actual distance Calculate the distance from the vehicle to the lane stop line and the speed of motor vehicles approaching the intersection ;

[0014] S8. Based on the distance of the vehicle from the lane stop line. and the speed of motor vehicles approaching the intersection Light status information management structure Vehicle Information Management Structure Determine whether a motor vehicle is speeding, parked abnormally, not stopping at a green light, or running a red light.

[0015] Furthermore, in step S1, video stream information from the intersection's electronic police camera is obtained via the ONVIF protocol.

[0016] Furthermore, step S2 includes the following steps:

[0017] S2-1. Use the cv::cvtcolor function in the OpenCV library to convert the video stream information from YUV-NV12 format to BGR format;

[0018] S2-2. Use the cv::equalizeHist function in the OpenCV library to extract the BGR format video stream information. The contrast of the intersection road image in frame 1 is enhanced to obtain the preprocessed image of frame 2. Frame intersection road image ,all The preprocessed intersection road images constitute an image set. .

[0019] Furthermore, step S3 includes the following steps:

[0020] S3-1. The first one after artificial labeling preprocessing Frame intersection road image The Middle Each lane area , , For the number of lanes, the first Each lane area It has a quadrilateral structure, the first Each lane area The edge at the front is the lane stop line, the first... Each lane area The domain is located at the back edge and the first Each lane area The edges located at the front end are parallel and the first Each lane area The edge located at the back end and the first Each lane area The distance between the edges at the front end is 40-60 meters. Each lane area The edge located on the left is the lane line on the left side of that lane. Each lane area The edge located at the right end is the lane line on the right side of that lane;

[0021] S3-2. The first one after artificial labeling preprocessing Frame intersection road image Red light violation monitoring area The red light violation monitoring area It is a quadrilateral structure, located in At the front end of each lane area, the red light violation monitoring area The width is , Lane width, In the formula For the first The width of each lane, the red light detection area The length of the actual lane stop line is 2-3 times the length of the vehicle. Using a GPS locator, a true far end line parallel to the lane stop line is marked 50-100 meters behind it. The width of the true far end line is equal to the width of the lane stop line. The actual distance between the lane stop line and the true far end line is measured. ;

[0022] S3-3. The first step after artificial labeling preprocessing Frame intersection road image Vehicle inspection area Vehicle inspection area It is a rectangular structure, all Each lane area and red light violation detection area Located in the vehicle inspection area Inside;

[0023] S3-4. Manually record the first... after preprocessing Frame intersection road image Central lamp status detection area Light status detection area To select the rectangular structure containing the traffic lights at the intersection;

[0024] S3-5. In the... Frame intersection road image A perspective transformation coordinate system is established with the lane stop line as the X-axis and the perpendicular line to the lane stop line as the Y-axis. The leftmost point of the lane stop line is marked as the nearest coordinate point A. , Let A be the X-axis coordinate of the nearest point A. Assuming the Y-axis coordinate of the nearest coordinate point A, mark the rightmost point of the lane stop line as the nearest coordinate point B. , Let B be the x-coordinate of the nearest point. Given the Y-axis coordinate of the nearby point B, mark the leftmost point of the actual farthest line as the distant point A. , Let A be the X-axis coordinate of a distant point. Given the Y-axis coordinate of distant point A, mark the rightmost point of the actual farthest line as distant point B. , Let A be the X-axis coordinate of a distant point. Let be the Y-coordinate of the distant point A.

[0025] Furthermore, step S4 includes the following steps:

[0026] S4-1. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed image. Frame intersection road image According to vehicle inspection area Cut the material to obtain the first... Frame vehicle detection area image ,all The vehicle detection region images constitute the vehicle detection region image set. , ;

[0027] S4-2. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed img... Frame intersection road image According to the light status detection area Cut to get the first Frame light state detection region image ,all The vehicle detection region images constitute the light state detection region image set. , .

[0028] Furthermore, step S5 includes the following steps:

[0029] S5-1. Set up the lamp status detection image set The Middle Frame light state detection region image The input is fed into the ResNet-18 model, and the first... List of frame light state detection results , ,in It is a traffic light type. A value of 0 indicates a dot signal light. A value of 1 indicates a left turn signal. A value of 2 indicates a straight-ahead signal. The results of the lamp status detection are as follows. A value of -1 indicates an invalid value. 0 indicates the lights are off. A value of 1 indicates a green light. A value of 2 indicates a red light. A value of 3 indicates a yellow light;

[0030] S5-2. According to the first List of frame light state detection results Establish the first Frame light state information management structure Light status information management structure This includes the light status parameter `LightInformation.Status` and the current light status duration parameter `LightInformation.Duration`. `LightInformation.Status` of -1 indicates an invalid value; `LightInformation.Status` of 0 indicates the dot signal light is off; `LightInformation.Status` of 1 indicates the dot signal light is green; `LightInformation.Status` of 2 indicates the dot signal light is red; and `LightInformation.Status` of 3 indicates the dot signal light is yellow. LightInformation.Status is 4 when the straight-ahead signal is green, 5 when it's red, 6 when it's yellow, 7 when it's green, 8 when it's red, and 9 when it's yellow. The LightInformation.Duration parameter records the duration of the current light status. Individual Light Status Information Management Structure Constructing the lamp status information management structure , .

[0031] Furthermore, step S6 includes the following steps:

[0032] S6-1. Set up vehicle detection area images The Middle Frame vehicle detection area image The data is input into a YOLOv11 model for vehicle target detection, and the results are obtained. The first box selects the rectangular frame containing the vehicle, constructing the second... Frame vehicle target detection list , ,in The first detection for the YOLOv11 model One motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing each motor vehicle. For the first The ordinate of the center point of the rectangular frame containing each motor vehicle. For the first The width of the rectangular frame of a motor vehicle. For the first The height of the rectangular frame of each motor vehicle;

[0033] S6-2. Using the Deepsort algorithm to sort the first... Frame vehicle detection area image In Track individual vehicles to obtain... The tracked motor vehicle, constructing the first Frame Motor Vehicle Target Tracking Information List , ,in For the first A tracked motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The ordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The width of the rectangle containing the tracked motor vehicle. For the first The height of the rectangle containing the tracked motor vehicle;

[0034] S6-3. Constructing the first Frame vehicle information management structure Vehicle information management structure Including the vehicle number being tracked The parameters Vehicle.k, the first Frame Motor Vehicle Target Tracking Information List Record No. The parameter Vehicle.LaneNum indicates which lane a tracked vehicle is located in. Time indicates the first The center point of the rectangle containing the tracked motor vehicle is located at the... Within each lane area When Vehicle.LaneNum is 0, it indicates that the first line is the first line of the first line. The tracked vehicle is not located in the lane area, all The vehicle information management structure consists of several vehicle information management structures. , .

[0035] Furthermore, step S7 includes the following steps:

[0036] S7-1. Set the corresponding points of the perspective transformation matrix image respectively. , , , , , , , ;

[0037] S7-2. Set the corresponding points of the real scene for the perspective transformation matrix respectively. , , , , , , , ;

[0038] S7-3. Use the `cv::getPerspectiveTransform(srcTri, dstTri)` function from the OpenCV library to transform the perspective transformation matrix image to the corresponding points. , , , and the corresponding points of the perspective transformation matrix in the real scene , , , Generate a 3×3 perspective transformation matrix ;

[0039] S7-4. Through formula Calculate the first one respectively The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding x-coordinate , No. The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate and perspective transformation coefficients ;

[0040] S7-5. The distance of a tracked vehicle from the lane stop line. ;

[0041] S7-6. Through formula

[0042] Calculate the first The speed at which a tracking vehicle approaches the intersection. In the formula, For the first The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate in the middle, The frame rate of the video stream information from the electronic police camera at the intersection.

[0043] Furthermore, step S8 includes the following steps:

[0044] S8-1. If the first The speed at which a tracking vehicle approaches the intersection. If the speed exceeds the overspeed threshold P times consecutively, then the first... The first tracked vehicle was found to be speeding, and the first... Frame vehicle information management structure Establish the abnormal vehicle behavior parameter Vehicle[k].Behavior in the middle, and set the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior to 1;

[0045] S8-2. If the first The speed at which a tracking vehicle approaches the intersection. If the value is less than the resting threshold for Q consecutive times, then the first value is determined to be... The tracked vehicle is stationary when the first... When the number of tracked motor vehicles that are stationary exceeds the abnormal parking threshold, the first vehicle is judged to be... For each tracked vehicle that stops abnormally, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 2.

[0046] S8-3. According to the... Frame vehicle information management structure The parameter Vehicle.LaneNum in the middle obtains the first... The tracked vehicle is located in which lane, and all vehicles in that lane... The distances of the tracked vehicles to the lane stop line are sorted from smallest to largest. If the distance of the tracked vehicle to the lane stop line is sorted from smallest to largest, then... The distance of a tracked vehicle from the lane stop line. If it is not the minimum value, it indicates that the first... If the vehicle being tracked has a car ahead, then... The distance of a tracked vehicle from the lane stop line. The minimum value indicates that the first The tracking vehicle had no car ahead, when the first... The first vehicle being tracked had no other vehicles ahead and the second... The tracked vehicle remained stationary for a longer period than the first Frame light state information management structure The current green light duration parameter, LightInformation.Duration, records the current green light duration and the current green light duration. If the duration of stationary position of the tracked motor vehicle exceeds the vehicle start-up reaction time threshold, then the first vehicle is determined to be stationary. If a tracked vehicle fails to pass a green light, set the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior to 3.

[0047] S8-4. According to the first Frame light state information management structure The LightInformation.Status parameter in the code determines the status of the first light. The tracked motor vehicle is in the first Each lane area Whether it is a red light or not, if the light is red in that lane area, use the vector cross multiplication method to determine the first... Whether the tracked motor vehicle has left the area. Each lane area If the first The tracked vehicle drove out of the first... Each lane area It then entered the red light monitoring area. Then determine the first If a tracked vehicle runs a red light, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 4.

[0048] Furthermore, it also includes sending the lamp status information management structure at a sending time interval SendTime after step S8. Vehicle Information Management Structure It can be sent to downstream applications in JSON format or stored in a database.

[0049] The beneficial effects of this invention are as follows: In real-time video footage from electronic traffic enforcement monitoring at urban road intersections, machine vision algorithms are used to detect traffic light status and vehicle spatial movement information; then, based on perspective transformation, a road traffic area model is created, obtaining a one-to-one mapping relationship between the video footage and real road coordinates; thereby determining whether vehicles are experiencing abnormal traffic events such as speeding, abnormal parking, failing to use a green light, or running a red light within the detection area. This invention, based on intersection sensing equipment for detecting and reporting abnormal traffic events, can provide more valuable road condition information for signal control and real-time dispatching by the command center; and under the development concept of smart city construction, it provides more diverse basic data for AI decision-making. Attached Figure Description

[0050] Figure 1 This is a flowchart of the method of the present invention;

[0051] Figure 2 This is a diagram of coordinate points calculated using the artificial calibration perspective transformation method of the present invention. Detailed Implementation

[0052] The following is in conjunction with the appendix Figure 1 Appendix Figure 2 The present invention will be further described below.

[0053] A method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling includes:

[0054] S1. Obtain video stream information from the intersection's electronic police camera.

[0055] S2. Preprocess the acquired video stream information to obtain... Image set of intersection road images , , For the preprocessed first Frame intersection road image, .

[0056] S3. Based on the image set Calibration yields the vehicle detection area , No. Each lane area Red light violation monitoring area Light status detection area In the Frame intersection road image The nearest and far coordinate points are determined, and the first point is calculated. Frame intersection road image Middle lane width Actual distance .

[0057] S4. Based on the vehicle inspection area Obtain the vehicle detection area image set According to the light status detection area Obtain the image set of the lamp state detection region .

[0058] S5. Light status detection image set Visual classification is performed to obtain the lamp status information management structure. .

[0059] S6. Image set of vehicle detection area Target detection and tracking are performed to obtain the vehicle information management structure. .

[0060] S7. Based on nearby coordinates, distant coordinates, and lane width Actual distance Calculate the distance from the vehicle to the lane stop line and the speed of motor vehicles approaching the intersection .

[0061] S8. Based on the distance of the vehicle from the lane stop line. and the speed of motor vehicles approaching the intersection Light status information management structure Vehicle Information Management Structure Determine whether a motor vehicle is speeding, parked abnormally, not stopping at a green light, or running a red light.

[0062] By using machine vision algorithms to detect traffic light status and vehicle spatial movement information in real-time video feeds from electronic traffic enforcement cameras at urban intersections, and establishing a one-to-one mapping between video images and real road coordinates through perspective transformation technology, this method accurately determines whether vehicles are engaging in abnormal behaviors such as speeding, abnormal parking, failing to use a green light, or running a red light within the detection area. This detection method not only provides more valuable traffic information for traffic light control and real-time dispatching by command centers, but also offers more comprehensive data support for AI-based decision-making in smart cities.

[0063] In one embodiment of the present invention, in step S1, video stream information of the intersection electronic police camera is obtained through the ONVIF protocol.

[0064] In one embodiment of the present invention, step S2 includes the following steps:

[0065] S2-1. Use the cv::cvtcolor function in the OpenCV library to convert the video stream information from YUV-NV12 format to BGR format.

[0066] S2-2. Use the cv::equalizeHist function in the OpenCV library to extract the BGR format video stream information. The contrast of the intersection road image in frame 1 is enhanced to obtain the preprocessed image of frame 2. Frame intersection road image ,all The preprocessed intersection road images constitute the image set. .

[0067] In one embodiment of the present invention, step S3 includes the following steps:

[0068] S3-1. The first one after artificial labeling preprocessing Frame intersection road image The Middle Each lane area , , For the number of lanes, the first Each lane area It has a quadrilateral structure, the first Each lane area The edge at the front is the lane stop line, the first... Each lane area The domain is located at the back edge and the first Each lane area The edges located at the front end are parallel and the first Each lane area The edge located at the back end and the first Each lane area The distance between the edges at the front end is 40-60 meters. Each lane area The edge located on the left is the lane line on the left side of that lane. Each lane area The edge located at the right end is the lane line on the right side of that lane.

[0069] S3-2. The first one after artificial labeling preprocessing Frame intersection road image Red light violation monitoring area The red light violation monitoring area It is a quadrilateral structure, located in At the front end of each lane area, the red light violation monitoring area The width is , Lane width, In the formula For the first The width of each lane (the width of a single lane can be set at 3.75 meters according to the standard for trunk highways), and the red light violation detection area. The length of the actual lane stop line is 2-3 times the length of the vehicle. Using a GPS locator, a true far end line parallel to the lane stop line is marked 50-100 meters behind it. The width of the true far end line is equal to the width of the lane stop line. The actual distance between the lane stop line and the true far end line is measured. .

[0070] S3-3. The first step after artificial labeling preprocessing Frame intersection road image Vehicle inspection area Vehicle inspection area It is a rectangular structure, all Each lane area and red light violation monitoring area Located in the vehicle inspection area Inside.

[0071] S3-4. Manually record the first... after preprocessing Frame intersection road image Central lamp status detection area Light status detection area The rectangular structure is used to select the traffic lights at the intersection.

[0072] S3-5. In the... Frame intersection road image A perspective transformation coordinate system is established with the lane stop line as the X-axis and the perpendicular line to the lane stop line as the Y-axis. The leftmost point of the lane stop line is marked as the nearest coordinate point A. , Let A be the X-axis coordinate of the nearest point A. Assuming the Y-axis coordinate of the nearest coordinate point A, mark the rightmost point of the lane stop line as the nearest coordinate point B. , Let B be the x-coordinate of the nearest point. Given the Y-axis coordinate of the nearby point B, mark the leftmost point of the actual farthest line as the distant point A. , Let A be the X-axis coordinate of a distant point A. Given the Y-axis coordinate of distant point A, mark the rightmost point of the actual farthest line as distant point B. , Let A be the X-axis coordinate of a distant point A. Let be the Y-coordinate of the distant point A.

[0073] In one embodiment of the present invention, step S4 includes the following steps:

[0074] S4-1. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed image. Frame intersection road image According to vehicle inspection area Cut to get the first Frame vehicle detection area image ,all The vehicle detection region images constitute the vehicle detection region image set. , .

[0075] S4-2. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed img... Frame intersection road image According to the light status detection area Cut to get the first Frame light state detection region image ,all The vehicle detection region images constitute the light state detection region image set. , .

[0076] In one embodiment of the present invention, step S5 includes the following steps:

[0077] S5-1. Set up the lamp status detection image set The Middle Frame light state detection region image The input is fed into the ResNet-18 model, and the first... List of frame light state detection results , ,in It is a traffic light type. A value of 0 indicates a dot signal light. A value of 1 indicates a left turn signal. A value of 2 indicates a straight-ahead signal. The results of the lamp status detection are as follows. A value of -1 indicates an invalid value. 0 indicates the lights are off. A value of 1 indicates a green light. A value of 2 indicates a red light. A value of 3 indicates a yellow light.

[0078] S5-2. According to the first List of frame light state detection results Establish the first Frame light state information management structure Light status information management structure This includes the light status parameter `LightInformation.Status` and the current light status duration parameter `LightInformation.Duration`. `LightInformation.Status` of -1 indicates an invalid value; `LightInformation.Status` of 0 indicates the dot signal light is off; `LightInformation.Status` of 1 indicates the dot signal light is green; `LightInformation.Status` of 2 indicates the dot signal light is red; and `LightInformation.Status` of 3 indicates the dot signal light is yellow. When `n.Status` is 4, the straight-ahead signal light is green; when `LightInformation.Status` is 5, the straight-ahead signal light is red; when `LightInformation.Status` is 6, the straight-ahead signal light is yellow; when `LightInformation.Status` is 7, the left-turn signal light is green; when `LightInformation.Status` is 8, the left-turn signal light is red; and when `LightInformation.Status` is 9, the left-turn signal light is yellow. The current light status duration parameter `LightInformation.Duration` records the current light status duration (in milliseconds). Individual Light Status Information Management Structure Constructing the lamp status information management structure , .

[0079] In one embodiment of the present invention, step S6 includes the following steps:

[0080] S6-1. Set up vehicle detection area images The Middle Frame vehicle detection area image The data is input into a YOLOv11 model for vehicle target detection, and the results are obtained. The first box selects the rectangular frame containing the vehicle, constructing the second... Frame vehicle target detection list , ,in The first detection for the YOLOv11 model One motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing each motor vehicle. For the first The ordinate of the center point of the rectangular frame containing each motor vehicle. For the first The width of the rectangular frame of a motor vehicle. For the first The height of the rectangular frame of a motor vehicle.

[0081] S6-2. Using the Deepsort algorithm to sort the first... Frame vehicle detection area image In Tracking a vehicle to obtain... The tracked motor vehicle, constructing the first Frame Motor Vehicle Target Tracking Information List , ,in For the first A tracked motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The ordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The width of the rectangle containing the tracked motor vehicle. For the first The height of the rectangle containing the tracked motor vehicle.

[0082] S6-3. Constructing the first Frame vehicle information management structure Vehicle information management structure Including the vehicle number being tracked The parameters Vehicle.k, the first Frame Motor Vehicle Target Tracking Information List Record No. The parameter Vehicle.LaneNum indicates which lane a tracked vehicle is located in. Time indicates the first The center point of the rectangle containing the tracked motor vehicle is located at the... Within each lane area When Vehicle.LaneNum is 0, it indicates that the first line is the first line of the first line. The tracked vehicle is not located in the lane area, all The vehicle information management structure consists of several vehicle information management structures. , .

[0083] In one embodiment of the present invention, step S7 includes the following steps:

[0084] S7-1. Set the corresponding points of the perspective transformation matrix image respectively. , , , , , , , .

[0085] S7-2. Set the corresponding points of the real scene for the perspective transformation matrix respectively. , , , , , , , .

[0086] S7-3. Use the `cv::getPerspectiveTransform(srcTri, dstTri)` function from the OpenCV library to transform the perspective transformation matrix image to the corresponding points. , , , and the corresponding points of the perspective transformation matrix in the real scene , , , Generate a 3×3 perspective transformation matrix .

[0087] S7-4. Through formula Calculate the first one respectively The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding x-coordinate , No. The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate and perspective transformation coefficients .

[0088] S7-5. The distance of a tracked vehicle from the lane stop line. .

[0089] S7-6. Through formula

[0090] Calculate the first The speed at which a tracking vehicle approaches the intersection. (Unit: meters per second), where, For the first The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate in the middle, The frame rate of the video stream information from the electronic police camera at the intersection.

[0091] In one embodiment of the present invention, step S8 includes the following steps:

[0092] S8-1. If the first The speed at which a tracking vehicle approaches the intersection. If the speed exceeds the overspeed threshold P times consecutively, then the first time is determined to be... The first tracked vehicle was found to be speeding, and the first... Frame vehicle information management structure A vehicle abnormal behavior parameter, Vehicle[k].Behavior, is established and its value is set to 1. In this preferred embodiment, P is set to 3, and the speeding threshold is set to 22 m / s, or 80 km / h.

[0093] S8-2. If the first The speed at which a tracking vehicle approaches the intersection. If the value is less than the resting threshold for Q consecutive times, then the first value is determined to be... The tracked vehicle is stationary when the first... When the number of tracked motor vehicles that are stationary exceeds the abnormal parking threshold, the first vehicle is judged to be... For each tracked vehicle that stops abnormally, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 2. In this preferred embodiment, Q is set to 3, the stationary threshold is set to 1.4 m / s (5 km / h), and the abnormal parking determination threshold is set to 180 seconds.

[0094] S8-3. According to the... Frame vehicle information management structure The parameter Vehicle.LaneNum in the middle obtains the first... The tracked vehicle is located in which lane, and all vehicles in that lane... The distances of the tracked vehicles to the lane stop line are sorted from smallest to largest. If the distance of the tracked vehicle to the lane stop line is sorted from smallest to largest, then... The distance of a tracked vehicle from the lane stop line. If it is not the minimum value, it indicates that the first... If the vehicle being tracked has a car ahead, then... The distance of a tracked vehicle from the lane stop line. The minimum value indicates that the first The tracking vehicle had no car ahead, when the first... The first vehicle being tracked had no other vehicles ahead and the second... The tracked vehicle remained stationary for a longer period than the first Frame light state information management structure The current green light duration parameter, LightInformation.Duration, records the current green light duration and the current green light duration. If the duration of stationary position of the tracked motor vehicle exceeds the vehicle start-up reaction time threshold, then the first vehicle is determined to be stationary. If a tracked vehicle fails to pass a green light, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 3. Preferably, in this embodiment, the vehicle start-up reaction time threshold is set to 5 seconds.

[0095] S8-4. According to the first Frame light state information management structure The LightInformation.Status parameter in the code determines the status of the first light. The tracked motor vehicle is in the first Each lane area Whether it is a red light or not, if the light is red in that lane area, use the vector cross multiplication method to determine the first... Whether the tracked motor vehicle has left the area. Each lane area If the first The tracked vehicle drove out of the first... Each lane area It then entered the red light monitoring area. Then determine the first If a tracked vehicle runs a red light, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 4.

[0096] In one embodiment of the present invention, the lamp status information management structure is further included after step S8 by sending a time interval SendTime. Vehicle Information Management Structure The data is sent to downstream applications in JSON format or stored in a database. By periodically reporting abnormal traffic events at the intersection, diverse data is provided for signal optimization and road planning. Preferably, in this embodiment, the sending interval SendTime is set to 1 second. Downstream applications can use abnormal vehicle traffic events to determine whether the lane division and signal timing scheme at the intersection are reasonable, and provide a reference for the evaluation, diagnosis, and management of traffic capacity.

[0097] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling, characterized in that, include: S1. Obtain video stream information from the intersection's electronic police camera; S2. Preprocess the acquired video stream information to obtain... Image set of intersection road images , , For the preprocessed first Frame intersection road image, ; S3. Based on the image set Calibration yields the vehicle detection area , No. Each lane area Red light violation monitoring area Light status detection area In the Frame intersection road image The nearest and far coordinate points are determined, and the first point is calculated. Frame intersection road image Middle lane width Actual distance ; S4. Based on the vehicle inspection area Obtain the vehicle detection area image set According to the light status detection area Obtain the image set of the lamp state detection region ; S5. Light status detection image set Visual classification is performed to obtain the lamp status information management structure. ; S6. Image set of vehicle detection area Target detection and tracking are performed to obtain the vehicle information management structure. ; S7. Based on nearby coordinates, distant coordinates, and lane width Actual distance Calculate the distance from the vehicle to the lane stop line and the speed of motor vehicles approaching the intersection ; S8. Based on the distance of the vehicle from the lane stop line. and the speed of motor vehicles approaching the intersection Light status information management structure Vehicle Information Management Structure Determine whether a motor vehicle is speeding, parked abnormally, not stopping at a green light, or running a red light.

2. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling as described in claim 1, characterized in that: In step S1, video stream information from the intersection's electronic police camera is obtained via the ONVIF protocol.

3. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling as described in claim 1, characterized in that, Step S2 includes the following steps: S2-1. Use the cv::cvtcolor function in the OpenCV library to convert the video stream information from YUV-NV12 format to BGR format; S2-2. Use the cv::equalizeHist function in the OpenCV library to extract the BGR format video stream information. The contrast of the intersection road image in frame 1 is enhanced to obtain the preprocessed image of frame 2. Frame intersection road image ,all The preprocessed intersection road images constitute the image set. .

4. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling according to claim 1, characterized in that, Step S3 includes the following steps: S3-1. The first one after artificial labeling preprocessing Frame intersection road image The Middle Each lane area , , For the number of lanes, the first Each lane area It has a quadrilateral structure, the first Each lane area The edge at the front is the lane stop line, the first... Each lane area The domain is located at the back edge and the first Each lane area The edges located at the front end are parallel and the first Each lane area The edge located at the back end and the first Each lane area The distance between the edges at the front end is 40-60 meters. Each lane area The edge located on the left is the lane line on the left side of that lane. Each lane area The edge located at the right end is the lane line on the right side of that lane; S3-2. The first one after artificial labeling preprocessing Frame intersection road image Red light violation monitoring area The red light violation monitoring area It is a quadrilateral structure, located in At the front end of each lane area, the red light violation monitoring area The width is , Lane width, In the formula For the first The width of each lane, the red light detection area The length of the actual lane stop line should be 2-3 times the length of the vehicle. Using a GPS locator, mark a true far end line parallel to the lane stop line 50-100 meters behind it. The width of the true far end line should be equal to the width of the lane stop line. Measure the actual distance between the lane stop line and the true far end line. ; S3-3. The first step after artificial labeling preprocessing Frame intersection road image Vehicle inspection area Vehicle inspection area It is a rectangular structure, all Each lane area and red light violation detection area Located in the vehicle inspection area Inside; S3-4. Manually record the first... after preprocessing Frame intersection road image Central lamp status detection area Light status detection area To select the rectangular structure containing the traffic lights at the intersection; S3-5. In the... Frame intersection road image A perspective transformation coordinate system is established with the lane stop line as the X-axis and the perpendicular line to the lane stop line as the Y-axis. The leftmost point of the lane stop line is marked as the nearest coordinate point A. , Let A be the X-axis coordinate of the nearest point A. Assuming the Y-axis coordinate of the nearest coordinate point A, mark the rightmost point of the lane stop line as the nearest coordinate point B. , Let B be the x-coordinate of the nearest point. Given the Y-axis coordinate of the nearby point B, mark the leftmost point of the actual farthest line as the distant point A. , Let A be the X-axis coordinate of a distant point. Given the Y-axis coordinate of distant point A, mark the rightmost point of the actual farthest line as distant point B. , Let A be the X-axis coordinate of a distant point. Let be the Y-coordinate of the distant point A.

5. The intersection vehicle abnormal driving detection method based on machine vision detection and road traffic area modeling according to claim 1, characterized in that, Step S4 includes the following steps: S4-1. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed image. Frame intersection road image According to vehicle inspection area Cut to get the first Frame vehicle detection area image ,all The vehicle detection region images constitute the vehicle detection region image set. , ; S4-2. Use the `cv::Mat crop_img = img(detect_rect)` function from the OpenCV library to crop the preprocessed img... Frame intersection road image According to the light status detection area Cut to get the first Frame light state detection region image ,all The vehicle detection region images constitute the light state detection region image set. , .

6. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling according to claim 5, characterized in that, Step S5 includes the following steps: S5-1. Set up the lamp status detection image set The Middle Frame light state detection region image The input is fed into the ResNet-18 model to identify the first... List of frame light state detection results , ,in It is a traffic light type. A value of 0 indicates a dot signal light. A value of 1 indicates a left turn signal. A value of 2 indicates a straight-ahead signal. The results of the lamp status detection are as follows. A value of -1 indicates an invalid value. 0 indicates the lights are off. A value of 1 indicates a green light. A value of 2 indicates a red light. A value of 3 indicates a yellow light; S5-2. According to the first List of frame light state detection results Establish the first Frame light state information management structure Light status information management structure This includes the light status parameter `LightInformation.Status` and the current light status duration parameter `LightInformation.Duration`. `LightInformation.Status` of -1 indicates an invalid value; `LightInformation.Status` of 0 indicates the dot signal light is off; `LightInformation.Status` of 1 indicates the dot signal light is green; `LightInformation.Status` of 2 indicates the dot signal light is red; and `LightInformation.Status` of 3 indicates the dot signal light is yellow. LightInformation.Status is 4 when the straight-ahead signal is green, 5 when it's red, 6 when it's yellow, 7 when it's green, 8 when it's red, and 9 when it's yellow. The LightInformation.Duration parameter records the duration of the current light status. Individual Light Status Information Management Structure Constructing the lamp status information management structure , .

7. The intersection vehicle abnormal driving detection method based on machine vision detection and road traffic area modeling according to claim 4, characterized in that, Step S6 includes the following steps: S6-1. Set up vehicle detection area images The Middle Frame vehicle detection area image The data is input into a YOLOv11 model for vehicle target detection, and the results are obtained. The first box selects the rectangular frame containing the vehicle, constructing the first... Frame vehicle target detection list , ,in The first detection for the YOLOv11 model One motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing each motor vehicle. For the first The ordinate of the center point of the rectangular frame containing each motor vehicle. For the first The width of the rectangular frame of a motor vehicle. For the first The height of the rectangular frame of each motor vehicle; S6-2. Using the Deepsort algorithm to sort the first... Frame vehicle detection area image In Track individual vehicles to obtain... The tracked motor vehicle, constructing the first Frame Motor Vehicle Target Tracking Information List , ,in For the first A tracked motor vehicle, , For the first The x-coordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The ordinate of the center point of the rectangle containing the tracked motor vehicle. For the first The width of the rectangle containing the tracked motor vehicle. For the first The height of the rectangle containing the tracked motor vehicle; S6-3. Constructing the first Frame vehicle information management structure Vehicle information management structure Including the vehicle number being tracked The parameters Vehicle.k, the first Frame Motor Vehicle Target Tracking Information List Record No. The parameter Vehicle.LaneNum indicates which lane a tracked vehicle is located in. Time indicates the first The center point of the rectangle containing the tracked motor vehicle is located at the... Within each lane area When Vehicle.LaneNum is 0, it indicates that the first line is the first line of the first line. The tracked vehicle is not located in the lane area, all The vehicle information management structure consists of several vehicle information management structures. , .

8. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling according to claim 7, characterized in that, Step S7 includes the following steps: S7-1. Set the corresponding points of the perspective transformation matrix image respectively. , , , , , , , ; S7-2. Set the corresponding points of the real scene for the perspective transformation matrix respectively. , , , , , , , ; S7-3. Use the `cv::getPerspectiveTransform(srcTri, dstTri)` function from the OpenCV library to transform the perspective transformation matrix image to the corresponding points. , , , and the corresponding points of the perspective transformation matrix in the real scene , , , Generate a 3×3 perspective transformation matrix ; S7-4. Through formula Calculate the first one respectively The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding x-coordinate , No. The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate and perspective transformation coefficients ; S7-5. The distance of a tracked vehicle from the lane stop line. ; S7-6. Through formula Calculate the first The speed at which a tracking vehicle approaches the intersection. In the formula, For the first The front of the tracked vehicle was in the... Frame vehicle detection area image The corresponding ordinate in the middle, The frame rate of the video stream information from the electronic police camera at the intersection.

9. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling according to claim 7, characterized in that, Step S8 includes the following steps: S8-1. If the first The speed at which a tracking vehicle approaches the intersection. If the speed exceeds the overspeed threshold P times consecutively, then the first time is determined to be... The first tracked vehicle was found to be speeding, and the first... Frame vehicle information management structure Establish the abnormal vehicle behavior parameter Vehicle[k].Behavior in the middle, and set the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior to 1; S8-2. If the first The speed at which a tracking vehicle approaches the intersection. If the value is less than the resting threshold for Q consecutive times, then the first value is determined to be... The tracked vehicle is stationary when the first... When the number of tracked motor vehicles that are stationary exceeds the abnormal parking threshold, the first vehicle is judged to be... For each tracked vehicle that stops abnormally, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 2. S8-3. According to the... Frame vehicle information management structure The parameter Vehicle.LaneNum in the middle obtains the first... The tracked vehicle is located in which lane, and all vehicles in that lane... The distances of the tracked vehicles to the lane stop line are sorted from smallest to largest. If the distance of the tracked vehicle to the lane stop line is sorted from smallest to largest, then... The distance of a tracked vehicle from the lane stop line. If it is not the minimum value, it indicates that the first... If the vehicle being tracked has a car ahead, then... The distance of a tracked vehicle from the lane stop line. The minimum value indicates that the first The tracking vehicle had no car ahead, when the first... The first vehicle being tracked had no other vehicles ahead and the second... The tracked vehicle remained stationary for a longer period than the first Frame light state information management structure The current green light duration parameter, LightInformation.Duration, records the current green light duration and the duration of the current green light. If the duration of stationary position of the tracked motor vehicle exceeds the vehicle start-up reaction time threshold, then the first vehicle is determined to be stationary. If a tracked vehicle fails to pass a green light, set the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior to 3. S8-4. According to the first Frame light state information management structure The LightInformation.Status parameter in the code determines the status of the light. The tracked motor vehicle is in the first Each lane area Whether it is a red light or not, if the light is red in that lane area, use the vector cross multiplication method to determine the first... Whether the tracked motor vehicle has left the area. Each lane area If the first The tracked vehicle drove out of the first... Each lane area It then entered the red light monitoring area. Then determine the first If a tracked vehicle runs a red light, the value of the abnormal vehicle behavior parameter Vehicle[k].Behavior is set to 4.

10. The method for detecting abnormal vehicle movement at intersections based on machine vision detection and road traffic area modeling according to claim 1, characterized in that: It also includes, after step S8, sending the lamp status information management structure at a time interval SendTime. Vehicle Information Management Structure It can be sent to downstream applications in JSON format or stored in a database.

Images