A vehicle line violation identification method and device

Abstract

The application discloses a kind of vehicle line pressing illegal behavior identification method and device, it is related to intelligent traffic monitoring technical field.The method includes: by unmanned aerial vehicle heterogeneous platform controls unmanned aerial vehicle to hover recording in target section airspace, obtains video stream data;Video frame is generated lane line binary mask after HSV color threshold filtering, then obtains candidate line segment set after Gaussian filtering, Canny edge detection and probability hough transform, after the double screening of angle constraint and spatial position constraint, topological fusion is extended, and lane center line is output;Vehicle is tracked using target detection and multi-target tracking algorithm;Lane center line is expressed as general equation as virtual boundary line, the boundary box center point coordinates of the current frame when vehicle first enters picture are substituted into the equation, when the product of two function values is less than zero, it is judged that line pressing illegal occurs.The application improves the anti-interference ability of lane line detection by geometric-space double constraint, realizes the automatic identification of line pressing illegal behavior.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic monitoring technology, specifically to a method and device for identifying vehicle lane violations. Background Technology

[0002] Vehicles illegally crossing solid lane lines is a common traffic violation on urban roads, posing a direct threat to driving safety. Currently, monitoring of such violations mainly relies on two methods: fixed-location electronic police cameras and traffic police patrols and on-site enforcement. Fixed cameras can only cover limited intersections or road sections, resulting in numerous blind spots across the vast area of ​​urban roads and intercity highways. Furthermore, large-scale deployment and maintenance are costly, and their fixed locations lack flexibility to address dynamic changes in high-incidence violation areas. Manual patrols, on the other hand, consume significant police resources, and officers face safety risks while working for extended periods in complex road environments. Therefore, existing monitoring methods are insufficient in terms of coverage, mobility, and automation, making it difficult to meet the needs of large-scale, dynamic traffic violation monitoring. Summary of the Invention

[0003] In view of this, the present invention aims to provide a method and device for identifying vehicle lane crossing violations, which utilizes the aerial perspective and mobile deployment capabilities of drones, combined with image processing and target tracking technology, to achieve automatic identification and judgment of vehicle lane crossing violations.

[0004] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A method for identifying vehicle lane violations includes the following steps: S1. Mission execution and video acquisition: Control the drone to perform automatic flight missions through the drone heterogeneous platform, hover over the target road section and record video, and acquire video stream data of the target road section; S2. Lane line recognition: For video frames in the video stream data, convert them from RGB color space to HSV color space and perform color threshold filtering to generate a lane line binary mask; perform Gaussian filtering and edge detection on the lane line binary mask in sequence to obtain an edge feature map; perform probabilistic Hough transform on the edge feature map to obtain a candidate line segment set; apply dual filtering of line segment angle constraints and line segment spatial position constraints to the candidate line segment set; perform topological fusion on the line segments that pass the filtering, and extend the fused line segments to span the entire screen to output the lane center line; S3. Vehicle Detection and Tracking: The target detection algorithm is used to detect vehicle targets in the video frame and output the bounding box of each vehicle target. The multi-target tracking algorithm is used to assign a unique identifier to each vehicle target, record its motion trajectory, and obtain the coordinates of the center point of the bounding box when each vehicle target first enters the frame and the coordinates of the center point of the bounding box in the current frame. S4. Determination of illegal behavior: The lane center line output in step S2 is used as a virtual boundary line and represented as a boundary equation. For any tracked vehicle target, the coordinates of the center point of the bounding box when it first enters the screen and the coordinates of the center point of the bounding box in the current frame are substituted into the boundary equation to obtain the first function value and the second function value. If the first function value and the second function value have opposite signs, it is determined that the vehicle target has committed a line crossing violation. S5. Result Output: Record the vehicles that are determined to have committed line violations and output alarm information.

[0005] As a preferred technical solution of the present invention, step S2 includes the following sub-steps: S21, Multispectral lane feature perception: Construct a dual-channel color threshold filter in the HSV color space, wherein the hue, saturation, and brightness threshold range of the first channel is adapted to yellow lane lines, and the hue, saturation, and brightness threshold range of the second channel is adapted to white lane lines. For each pixel of the video frame, determine whether its hue, saturation, and brightness fall within the threshold range of any channel. Pixels that fall within the threshold range are set as foreground, and the remaining pixels are set as background, generating the lane line binary mask; S22, Edge feature extraction: Perform Gaussian filtering on the lane line binary mask to smooth noise, and then use the Canny operator to sequentially perform gradient magnitude and direction calculation, non-maximum suppression, and dual-threshold edge connection to obtain the edge feature map; S23, Candidate line segment detection: Perform probabilistic Hough transform on the edge feature map, and aggregate edge pixels that meet the minimum line segment length and maximum allowable discontinuity into S24. Geometric-spatial dual constraint filtering: For each candidate line segment, calculate the angle between it and the perpendicular direction of the image, and remove line segments with an angle greater than a preset angle threshold. Then calculate the x-coordinate of its midpoint and remove line segments whose midpoints do not fall into the preset vertical region of interest. S25. Line segment topology fusion and main line segment selection: Line segments whose absolute value of the difference in slope is not greater than a preset slope threshold and whose minimum Euclidean distance between endpoints is not greater than a preset distance threshold are grouped into the same line segment cluster. The two endpoints that are farthest apart in the same line segment cluster are taken as the two endpoints of the fused line segment, and the fused line segment with the longest length is selected as the main lane line segment. S26. Edge extension and center line output: Calculate the equation of the straight line where the main lane line segment is located based on the coordinates of the two endpoints, calculate the intersection point of the straight line with the upper and lower boundaries of the image, and extend the main lane line segment to the two intersection points to obtain the lane center line that runs through the screen. This process effectively suppresses the interference of road surface texture, shadows, vehicles, and non-lane solid line targets such as pedestrian crossings, stop lines, and directional arrows on lane line recognition results through multispectral feature perception and geometric-spatial dual constraints.

[0006] As a preferred technical solution of the present invention, in sub-step S24, the angle threshold is ten degrees, that is, only line segments with an angle within plus or minus ten degrees to the perpendicular direction of the image are retained; the longitudinal region of interest is the middle one-third region in the width direction of the image.

[0007] As a preferred technical solution of the present invention, between sub-step S24 and sub-step S25, a sub-step is further included to distinguish between solid lane lines and dashed lane lines based on the line segment length characteristics: solid lane lines are represented as continuous long line segments in the lane line binary mask, while dashed lane lines are represented as a series of short line segments arranged at intervals. Therefore, the length of each line segment filtered by the double constraint is calculated, and line segments with a length less than a preset length threshold are determined as dashed lane line segments or interference line segments and filtered out. Only line segments with a length not less than the preset length threshold are retained as solid lane line segments to participate in the topology fusion of sub-step S25, so that subsequent processing is only performed on solid lane lines.

[0008] As a preferred embodiment of the present invention, the method further includes a periodic update step for lane line positions: setting an update time interval, and re-executing step S2 every time the update time interval is reached; if a lane center line that meets the conditions is obtained in this execution, the lane center line and its equation parameters in the cache are updated with the lane center line obtained in this execution; if a lane center line that meets the conditions is not obtained in this execution, the lane center line obtained successfully in the previous execution is used to cope with the slight displacement of the UAV caused by factors such as wind.

[0009] As a preferred embodiment of the present invention, in step S4, the boundary equation is a general form equation. ,in , These are the x-coordinate and y-coordinate in the image coordinate system, respectively. , , The coefficients of the straight line equation determined by the lane centerline; the process of determining the illegal behavior includes: obtaining the coordinates of the center point of the bounding box when the tracked vehicle target first enters the frame. and the center coordinates of the bounding box of the current frame Substitute each value into the boundary equation to calculate the first function value. Second function value ;when When established, it is determined that the vehicle target has committed a lane crossing violation. The geometric meaning of this determination condition is that the initial position and current position of the vehicle target are located on both sides of the lane center line.

[0010] As a preferred embodiment of the present invention, in sub-step S26, when there is another line segment in the fused line segment whose absolute value of the difference in slope with the main lane line segment is not greater than a preset slope threshold, and whose lateral distance from the main lane line segment falls within a preset lane solid line width range, the two line segments are respectively determined as the left boundary line and the right boundary line of the same lane solid line. The process of obtaining the lane center line includes: calculating the equation of the left boundary line based on the endpoint coordinates of the left boundary line and the right boundary line respectively. and the equation of the right boundary line ,in , These are the slope and intercept of the left boundary line, respectively. , Let be the slope and intercept of the right boundary line, respectively; calculate the average of the x-coordinates of the two boundary lines at the same y-coordinate to obtain the equation of the centerline. The centerline equation is converted into a general equation and then extended to span the entire screen to serve as the lane centerline.

[0011] A device for identifying vehicle lane violations, used to perform the above method, comprising: The UAV automatic task module is used to execute step S1, which controls the UAV to perform automatic flight, fixed-point hovering, recording and data transmission tasks through the UAV heterogeneous platform, and obtains video stream data of the target road segment. The lane line stability recognition module is used to execute step S2, which sequentially performs RGB color space to HSV color space conversion and color threshold filtering, Gaussian filtering and edge detection, probabilistic Hough transform, dual screening of line segment angle constraints and line segment spatial position constraints, and line segment topology fusion and extension on the video frames in the video stream data, outputs the lane center line that runs through the screen, and repeats the above processing at a preset update time interval to update the lane center line. The vehicle detection and tracking module is used to execute step S3. It uses a target detection algorithm to detect vehicle targets in the video frame and outputs the bounding box of each vehicle target. It uses a multi-target tracking algorithm to assign a unique identifier to each vehicle target, record its motion trajectory, and output the coordinates of the center point of the bounding box when each vehicle target first enters the frame and the coordinates of the center point of the bounding box in the current frame. The lane crossing behavior analysis module is used to execute steps S4 and S5. It takes the lane center line as a virtual boundary line and represents it as a boundary equation. It substitutes the coordinates of the center point of the bounding box when the vehicle first enters the screen and the coordinates of the center point of the bounding box in the current frame into the boundary equation to obtain a first function value and a second function value. When the signs of the first function value and the second function value are opposite, it determines that the vehicle target has committed a lane crossing violation, records the vehicle target, and outputs alarm information.

[0012] As a preferred technical solution of the present invention, when the lane line stability recognition module performs the dual screening, it retains only line segments whose angle with the vertical direction of the image is within ±10 degrees and whose midpoint x-coordinate falls within the middle one-third area of ​​the image width direction; and after the dual screening and before the line segment topology fusion, it filters out line segments whose length is less than a preset length threshold, and performs the line segment topology fusion and extension only with the retained solid lane line segments.

[0013] As a preferred embodiment of the present invention, the pressure line behavior analysis module expresses the boundary equation as a general equation. The product of the first function value and the second function value is less than zero as a condition for determining that a line crossing violation has occurred, and the unique identifier of the vehicle target that has been determined to have committed a line crossing violation is excluded from the determination process of subsequent video frames.

[0014] The beneficial effects of this invention are as follows: First, by utilizing the aerial perspective and mobile deployment capabilities of drones, it is not limited by the installation location of fixed monitoring equipment and can be quickly deployed to any target road segment to implement lane crossing violation monitoring. Second, through a dual screening strategy combining HSV color space multispectral feature perception with angle constraints and spatial position constraints, and supplemented by the distinction between solid and dashed lines based on length thresholds, the impact of road textures, shadows, vehicles, pedestrian crossings, stop lines, directional arrows, and other interference on lane line detection accuracy is effectively suppressed. Third, the periodic update mechanism for lane line positions addresses slight displacements during drone hovering, ensuring the continuous stability of the lane centerline reference. Fourth, the determination method based on the change in the function value sign of the boundary box center point of the current frame when the vehicle first enters the frame is in the same boundary equation has low computational load and clear geometric meaning, achieving accurate and automatic determination of lane crossing violations. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the heterogeneous platform system architecture and task flow of the unmanned aerial vehicle (UAV) according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the lane line recognition process according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the periodic update process of lane line positions according to an embodiment of the present invention. Figure 4 This is a schematic diagram illustrating the calculation principle of the left and right boundary lines of the solid lane line and the virtual center line according to an embodiment of the present invention; Figure 5 This is a diagram showing the result of HSV color space binarization processing as described in an embodiment of the present invention. Figure 6 This is a Canny edge detection result image as described in an embodiment of the present invention; Figure 7 This is a diagram showing the detection results of candidate line segments using the probabilistic Hough transform according to an embodiment of the present invention. Figure 8 This is a diagram showing the result of geometric-spatial dual constraint screening and line segment topology fusion as described in an embodiment of the present invention. Figure 9 This is a diagram showing the virtual boundary calibration results according to an embodiment of the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0017] This invention provides a method and device for identifying vehicle lane crossing violations. The core of the method is as follows: a drone hovers over a target road segment to collect road video, a multi-level image processing algorithm is used to identify solid lane lines from the video frames and calculate their center lines, the center lines of the lanes are used as virtual boundaries for violation determination, and then combined with vehicle target detection, multi-target tracking and analytical geometric determination, the violation of vehicles crossing solid lane lines is automatically identified.

[0018] To ensure consistency and clarity in the meaning of terms used in this document, the following conventions are adopted: Video stream data refers to a continuous sequence of images recorded and transmitted back by an UAV's onboard camera; Video frame refers to a single frame image in the video stream data; Solid lane line refers to the actual white or yellow solid line markings painted on the road surface; Lane line binary mask refers to a binary image with lane line pixels as the foreground obtained after color thresholding of the video frame; Edge feature map refers to the image obtained after edge detection of the lane line binary mask; Candidate line segment set refers to the set of all line segments obtained after performing a probabilistic Hough transform on the edge feature map; Main lane line segment refers to the line segment with the longest length after topological fusion; Lane center line refers to the straight line that runs through the image along its extension direction, calculated from the solid lane line; Virtual boundary line refers to the lane center line used for lane violation determination in step S4, and refers to the same straight line as the lane center line.

[0019] Step S1: Task execution and video capture. (e.g.) Figure 1 As shown, the backend server issues flight mission commands to the drone via a heterogeneous drone platform; the drone automatically executes the flight mission to hover over the target road segment, and the onboard camera continuously records video of the target road segment from a top-down perspective, obtaining video stream data; after the mission is completed, the drone automatically returns and transmits the video stream data back to the backend server. The algorithm program on the backend server executes steps S2 to S5 frame by frame on the transmitted video stream data. In this embodiment, the drone's hovering height is thirty to fifty meters, the optical axis of the onboard camera is approximately perpendicular to the road surface, and the recorded video has a resolution of 1920×1080 pixels and a frame rate of 25 frames per second.

[0020] Step S2: Lane Line Recognition. Lane line recognition is the foundational step in the entire violation determination process; its accuracy directly determines the reliability of subsequent lane crossing determinations. For example... Figure 2 As shown, this step is executed sequentially according to sub-steps S21 to S26, extracting solid lane lines from the complex road scene and calculating their center lines.

[0021] Sub-step S21: Multispectral Lane Feature Perception. The input RGB format video frame is converted to the HSV color space. The HSV color space decomposes color information into three independent channels: hue (H), saturation (S), and lightness (V), making it less sensitive to changes in illumination compared to the RGB color space. Let... , , To normalize to The three components of the interval are lightness. saturation (when Time to take ),tone The value is determined by the component that achieves the maximum value. After the conversion, a dual-channel color threshold filter is constructed based on the actual color features of lane lines in the road scene: the first channel is used to extract yellow lane lines, and the second channel is used to extract white lane lines. In this embodiment, hue, saturation, and brightness are respectively determined by... , , This indicates that the threshold range for the first channel is taken as... , , The threshold range of the second channel is taken as follows , , The video frame is processed pixel by pixel, and its HSV value is checked to see if it falls within any of the above-mentioned intervals. Pixels that fall within this interval are set to 255 in the output image (foreground), and the remaining pixels are set to 0 (background). This yields the lane line binary mask. Figure 5 As shown, after this sub-step, white or yellow markings such as solid lane lines, dashed lane lines, pedestrian crossing lines, stop lines, directional arrows, and road markings are highlighted as the foreground in the mask, while asphalt pavement, road shadows, green belts, and most vehicle body areas are suppressed as the background.

[0022] Sub-step S22: Edge feature extraction. First, a Gaussian filter is applied to the lane line binary mask to smooth edge burrs and isolated noise points. In this embodiment, the Gaussian kernel size is 5×5 and the standard deviation is [value missing]. The two-dimensional Gaussian kernel function is Subsequently, the Canny operator is used for edge detection, sequentially performing gradient magnitude and direction calculation, non-maximum suppression, and double-threshold edge connection. The gradient magnitude... gradient direction , , These are the first-order differences in the horizontal and vertical directions, respectively. In this embodiment, the high threshold is set to 150 and the low threshold to 50. Pixels with a gradient magnitude greater than the high threshold are identified as strong edges, while pixels with a gradient magnitude between the high and low thresholds and connected to strong edges are identified as weak edges and retained. The remaining pixels are discarded. This yields a pixel-level precision edge feature map, such as... Figure 6 As shown. It should be noted that, since the solid lane line has a certain width, it appears in the edge feature map as two approximately parallel edges located on the left and right sides of the solid lane line, respectively. These two edges are the left boundary line and the right boundary line mentioned later.

[0023] Sub-step S23: Candidate line segment detection. Perform a probabilistic Hough transform on the edge feature map: map the edge pixels in the image space to a line segment with a polar radius of 1 / 2. and polar angle In the parameter space of parameters, the polar coordinate expression of a straight line is: The peak value in the parameter space that exceeds the threshold corresponds to a straight line in the image space. Then, continuous edge pixels on this straight line that satisfy the minimum line segment length and maximum allowable discontinuity constraints are aggregated, and the coordinates of the two endpoints of the line segment are output. In this embodiment, the polar radius resolution is 1 pixel, and the polar angle resolution is... The transformation parameters are set to radians, with an accumulator threshold of 50, a minimum segment length of 50 pixels, and a maximum allowed discontinuity of 20 pixels on a straight line. The transformation result is a set of coordinates from the two endpoints. , The defined line segments form a set of candidate line segments. For example... Figure 7 As shown in the figure, the candidate line segment set detected in green is the set of candidate line segments obtained in this sub-step. This set includes not only the edge segments on both sides of the solid lane lines, but also interfering line segments generated by pedestrian crossings, stop lines, directional arrows, road markings, dashed lane lines, road edges, and green belt boundaries. The line segments generated by pedestrian crossings and stop lines extend in an approximately horizontal direction in the image, while the line segments generated by directional arrows and road markings are shorter and scattered in direction. These interfering line segments will be filtered out step by step in sub-step S24, the solid and dashed line area sub-step, and sub-step S25. Figure 7 The horizontal line segments and arrow outline segments that are still visible are intermediate results before filtering, rather than the final lane line recognition results.

[0024] Sub-step S24: Geometric-Spatial Dual Constraint Filtering. To accurately filter out the edge segments of solid lane lines from the candidate line segment set, this invention designs a dual filtering mechanism combining angle constraints and spatial position constraints based on prior geometric knowledge. Regarding angle constraints, considering that the drone captures the road from an approximately vertical overhead viewpoint, and the solid lane lines extend in an approximately vertical direction in the image, therefore... Calculate the angle between each candidate line segment and the vertical direction of the image. Only retain those that meet the requirements. The line segment, in this embodiment, has an angle threshold of 100°. Approximate horizontal line segments generated by pedestrian crossings, stop lines, etc. near Therefore, all of them were eliminated in this stage. Regarding spatial constraints, when the UAV performs its flight mission, the operator has already adjusted the target lane solid line to the center area of ​​the image through flight path planning. Therefore, the middle third of the image width is set as the longitudinal region of interest. Calculate the x-coordinate of the midpoint of the candidate line segment Only retain those that meet the requirements. The line segments, of which In this embodiment, the image width is... Pixels; line segments located on both sides of the image, generated by road edges, green belt boundaries, and adjacent lane markings, are discarded because their midpoints fall outside the region of interest. This dual constraint effectively filters out interfering line segments from both sides of the road surface and from non-perpendicular directions.

[0025] In a preferred embodiment, a sub-step involving solid and dashed line regions is performed between sub-steps S24 and S25. Considering that crossing dashed lane lines is generally considered a legal lane change under traffic regulations, while this invention aims to identify illegal crossings of solid lane lines, it is necessary to distinguish between solid and dashed lane lines. In the lane line binary mask and edge feature map, solid lane lines appear as continuous, long connected regions, while dashed lane lines appear as a series of spaced-out, short connected regions. Similarly, the outlines of directional arrows and road surface text are represented as short line segments. Therefore, according to... Calculate the length of each line segment after double constraint filtering. ,Will The line segments are identified as lane dashed lines or interfering line segments and filtered out, retaining only the lines that are not lane dashed lines or interfering line segments. The line segments are treated as solid lane lines, ensuring that subsequent topology fusion and centerline calculations are performed only on solid lane lines. In this embodiment, the length threshold is set to one-tenth of the image height, i.e. Pixel.

[0026] Sub-step S25: Line Segment Topology Fusion and Main Line Segment Selection. The same edge of a solid line in the same lane is often detected as multiple discrete line segments due to marking wear, vehicle obstruction, or edge breakage. Therefore, clustering and fusion of the retained line segments is necessary. Specifically, for any two line segments... and First press Calculate its slope respectively and When the condition of similar slopes is met simultaneously Adjacent to endpoints At that time, the two are classified into the same line segment family, among which The minimum Euclidean distance between the four pairs of endpoints of the two line segments; in this embodiment, the slope threshold is taken as... Distance threshold Pixels. For all line segments within the same line segment cluster, the two endpoints furthest apart from each other in their endpoint set are taken as the endpoints of the fused line segment, thus merging discrete short line segments belonging to the same edge but detected as multiple segments into a continuous long line segment. After fusion, based on the principle of optimal length, all fused line segments are sorted in descending order of length, and the longest one is selected as the main lane line segment; that is, the longest length is used as the sole quantification criterion for the main lane line segment. For example... Figure 8 As shown, after dual-constraint filtering and topology fusion, the edge segments on both sides of the solid lane line are completely preserved and connected into long line segments that run through the region of interest; the short green line segment located at the guide arrow in the figure is longer than [missing information]. However, residual interference segments that are significantly smaller than the main lane segment are discarded when selecting the main lane segment in this sub-step according to the principle of optimal length, and are not included in the subsequent centerline calculation.

[0027] Sub-step S26, Edge Extension and Centerline Output. First, determine if there exists another line segment in the merged line segment that forms the left and right edges of the solid lane line with the main lane line segment: if a merged line segment exists, the absolute value of the difference between its slope and the slope of the main lane line segment is not greater than... If the lateral distance between the midpoint of the line segment and the midpoint of the main lane line segment falls within the preset lane solid line width range (10 to 40 pixels in this embodiment), then the two line segments are determined to be the left and right boundary lines of the same lane solid line, and the virtual center line is calculated as follows: based on the endpoint coordinates of the two boundary lines, according to... and Find the equation of the left boundary line and the equation of the right boundary line ,in , These are the slope and intercept of the left boundary line, respectively. , These represent the slope and intercept of the right boundary line, respectively; then, using the same ordinate... The equation of the virtual centerline is obtained by averaging the x-coordinates of the two boundary lines. Its calculation principle is as follows Figure 4 As shown, Figure 4The two black lines represent the left and right boundary lines of the same lane, respectively, while the red line represents the virtual center line calculated from these two lines. If no other line segment satisfies the above conditions, the line containing the main lane segment is used directly as the virtual center line.

[0028] The virtual center line is then extended to span the entire screen: Let the equation of the virtual center line be... , respectively and ( In this embodiment, the height is the image height. (pixels) to find the intersection points of the line with the upper and lower boundaries of the image. and The line segment between the two intersection points is the final output lane centerline. Finally, the virtual centerline equation is simplified into a general equation. ,in , , This general equation serves as the virtual boundary equation in step S4. For example... Figure 9 As shown, the final virtual boundary line is indicated by a red straight line in the image, which is precisely located at the center of the target lane's solid line.

[0029] As a preferred implementation, to address the slight displacement of the drone during hovering caused by factors such as wind disturbance, this invention also designs a periodic lane line position update mechanism, such as... Figure 3 As shown. The system is set to update at specific intervals. In this embodiment, we take Seconds. After reading the current video frame, the system determines whether the frame is the first frame or whether the time elapsed since the last update has been reached. If not, skip the detection in this frame, directly read the last valid lane centerline from the cache and output it; if yes, re-execute the complete lane line detection process from sub-steps S21 to S26. When the detection successfully obtains a lane centerline that meets the conditions, update the lane centerline and its equation parameters in the cache with the result and record the current timestamp, then output the lane centerline; if the detection fails due to screen occlusion, drastic changes in lighting, or other reasons, discard the result and read the last valid lane centerline from the cache for output. This ensures that there is a usable lane centerline reference at any time.

[0030] Step S3: Vehicle Detection and Tracking. After obtaining a stable lane centerline reference, the system uses a deep learning object detection algorithm (in this embodiment, a YOLO series single-stage object detection network) to process each video frame, outputting the bounding boxes and confidence scores of all vehicle targets in the frame, and filtering out detection results with a confidence score lower than 0.5. For each vehicle target, based on the coordinates of its upper left corner of the bounding box... and the coordinates of the bottom right corner ,according to Calculate the coordinates of the vehicle target's bounding box center point in the current frame. Then, using a multi-target tracking algorithm (this embodiment employs a multi-target tracking method based on Kalman filter state prediction and Hungarian algorithm data association), match the current frame's detection box with the existing trajectory prediction box using the intersection-over-union ratio (IoU) as the association cost. Successfully matched detection boxes inherit the unique identifier of the original trajectory; unmatched detection boxes create new trajectories and are assigned new unique identifiers. Trajectories that fail to match for several consecutive frames are terminated. This assigns a unique identifier to each vehicle entering the frame and continuously tracks its position changes in subsequent frames, recording its complete motion trajectory from entering to leaving the frame, while simultaneously saving the bounding box center point coordinates of the vehicle target when it first enters the frame. Coordinates of the center point of the bounding box in the current frame .

[0031] Step S4: Determination of Violation. This step is based on analytical geometry principles. The system converts the lane centerline output in step S2 into a general equation. The form serves as a virtual boundary, in which , These are the x-coordinate and y-coordinate in the image coordinate system, respectively. , , The coefficients of the straight line equation are determined by the lane centerline. For any tracked vehicle target, the coordinates of the center point of the bounding box when it first enters the frame are... and the center coordinates of the bounding box of the current frame Substitute each value into the boundary equation and calculate the first function value. Second function value For a straight line, points on opposite sides of the line will produce function values ​​with opposite signs when substituted into the equation, while points on the same side will produce function values ​​with the same sign. Therefore, when... When established, it indicates that the initial position and current position of the vehicle target are on opposite sides of the lane center line, meaning that the vehicle target crossed the solid lane line during the tracking period, and the system determines that it constitutes a lane crossing violation.

[0032] Step S5: Result Output. Once a vehicle target corresponding to a unique identifier is determined to have committed a line-crossing violation, the system immediately records relevant information about the violation, including the vehicle target's unique identifier, the frame number and timestamp of the violation, the coordinates of the bounding box center point of the frame, and the corresponding video frame image, and outputs an alarm message. Simultaneously, the unique identifier is marked as processed and will no longer be involved in the judgment of subsequent video frames, thus preventing the same violation from being recorded repeatedly.

[0033] This invention also provides a vehicle lane-crossing violation identification device for executing the above-described method. The device includes an unmanned aerial vehicle (UAV) automatic task module, a lane stability recognition module, a vehicle detection and tracking module, and a lane-crossing behavior analysis module, each module corresponding one-to-one with the steps of the above-described method.

[0034] The UAV automatic mission module is used to execute step S1, which is to send flight mission commands to the UAV through the UAV heterogeneous platform, control the UAV to automatically fly to the target road segment, hover, record video and transmit data back, and obtain video stream data of the target road segment.

[0035] The lane line stabilization recognition module executes step S2, which involves sequentially performing sub-steps S21 (HSV color space conversion and dual-channel color threshold filtering), S22 (Gaussian filtering and Canny edge detection), S23 (probabilistic Hough transform candidate line segment detection), S24 (dual filtering based on angle and spatial position constraints), solid and dashed line differentiation, S25 (line segment topology fusion and main line segment selection), and S26 (edge ​​extension and centerline output), outputting the lane centerline and its general equation that spans the entire screen. This module also includes a built-in periodic update unit that re-executes the above detection process at set update intervals. When a detection is successful, the lane centerline and its equation parameters in the cache are updated; when a detection fails, the result of the last successful detection in the cache is used, thus ensuring the continuous stability of the output lane centerline.

[0036] The vehicle detection and tracking module is used to execute step S3, which is to use a target detection algorithm to detect vehicle targets in the video frame and output bounding boxes, use a multi-target tracking algorithm to assign a unique identifier to each vehicle target, record its motion trajectory, and output the coordinates of the center point of the bounding box when each vehicle target first enters the frame and the coordinates of the center point of the bounding box in the current frame.

[0037] The lane crossing behavior analysis module is used to execute steps S4 and S5. That is, the lane center line output by the lane line stability recognition module is used as a virtual boundary line in the form of a general equation. The coordinates of the center points of the two bounding boxes output by the vehicle detection and tracking module are substituted into the equation to obtain the first function value and the second function value. When the product of the two function values ​​is less than zero, it is determined that the vehicle target has committed a lane crossing violation, the violation event is recorded and an alarm message is output. At the same time, the unique identifier of the determined vehicle target is excluded from the judgment process of subsequent video frames.

[0038] Those skilled in the art will recognize that the units and method steps of the above embodiments can be implemented using electronic hardware, computer software, or a combination of both. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying vehicle lane-crossing violations, characterized in that, Includes the following steps: S1. Mission execution and video acquisition: Control the drone to perform automatic flight missions through the drone heterogeneous platform, hover over the target road section and record video, and acquire video stream data of the target road section; S2. Lane line recognition: For video frames in the video stream data, convert them from RGB color space to HSV color space and perform color threshold filtering to generate a lane line binary mask; perform Gaussian filtering and edge detection on the lane line binary mask in sequence to obtain an edge feature map; perform probabilistic Hough transform on the edge feature map to obtain a candidate line segment set; apply dual filtering of line segment angle constraints and line segment spatial position constraints to the candidate line segment set; Perform topology fusion on the selected line segments, and extend the fused line segments to span the entire screen to output the lane centerline; S3. Vehicle Detection and Tracking: The target detection algorithm is used to detect vehicle targets in the video frame and output the bounding box of each vehicle target. The multi-target tracking algorithm is used to assign a unique identifier to each vehicle target, record its motion trajectory, and obtain the coordinates of the center point of the bounding box when each vehicle target first enters the frame and the coordinates of the center point of the bounding box in the current frame. S4. Determination of illegal behavior: The lane center line output in step S2 is used as a virtual boundary line and represented as a boundary equation. For any tracked vehicle target, the coordinates of the center point of the bounding box when it first enters the screen and the coordinates of the center point of the bounding box in the current frame are substituted into the boundary equation to obtain the first function value and the second function value. If the first function value and the second function value have opposite signs, it is determined that the vehicle target has committed a line crossing violation. S5. Result Output: Record the vehicles that are determined to have committed line violations and output alarm information.

2. The method for identifying vehicle lane-crossing violations according to claim 1, characterized in that: Step S2 includes the following sub-steps: S21. Multispectral Lane Feature Perception: Construct a dual-channel color threshold filter in the HSV color space, wherein the hue, saturation, and brightness threshold range of the first channel is adapted to the yellow lane line, and the hue, saturation, and brightness threshold range of the second channel is adapted to the white lane line. For each pixel of the video frame, determine whether its hue, saturation, and brightness fall within the threshold range of any channel. Pixels that fall within the threshold range are set as foreground and the remaining pixels are set as background, thereby generating the lane line binary mask. S22. Edge feature extraction: Gaussian filtering is applied to the lane line binary mask to smooth noise, and then the Canny operator is used to calculate gradient magnitude and direction, suppress non-maximum values ​​and perform double threshold edge connection in sequence to obtain the edge feature map. S23. Candidate line segment detection: Perform a probabilistic Hough transform on the edge feature map, and aggregate the edge pixels that satisfy the minimum line segment length and the maximum allowable discontinuity into line segments to obtain the candidate line segment set defined by the coordinates of the two endpoints. S24. Geometric-Spatial Dual Constraint Screening: For each candidate line segment in the candidate line segment set, according to... Calculate the angle between it and the vertical direction of the image. Remove Greater than the preset angle threshold The line segments, of which and The coordinates of the two endpoints of the candidate line segment; then... Calculate the x-coordinate of its midpoint Remove Line segments that do not fall within the preset vertical region of interest; S25. Segment Topology Fusion and Main Segment Selection: For the filtered segment, according to... Calculate the slope of each line segment, ensuring that the absolute value of the difference between the slopes is not greater than a preset slope threshold. Furthermore, the minimum Euclidean distance between the endpoints is not greater than a preset distance threshold. The line segments are classified into the same line segment cluster; for all line segments in the same line segment cluster, the two endpoints that are farthest apart from each other in the set of their endpoints are taken as the two endpoints of the merged line segment to obtain the merged line segment. Select the longest merged line segment as the main lane line segment; S26. Edge Extension and Centerline Output: Based on the coordinates of the two endpoints of the main lane segment, according to... Find the equation of the line containing it, where the slope is... ,intercept ; respectively and Find the intersection points of the line with the upper and lower boundaries of the image. The image height is defined as follows: the main lane segment is extended to the area between the two intersection points to obtain the lane centerline that runs through the entire image.

3. The method for identifying vehicle lane-crossing violations according to claim 2, characterized in that: In sub-step S24, the angle threshold The angle is 10 degrees, meaning only line segments with an angle within ±10 degrees to the perpendicular direction of the image are retained; the vertical region of interest is the middle third of the image width, meaning only the midpoint x-coordinate is retained. The line segments, of which This represents the image width.

4. The method for identifying vehicle lane crossing violations according to claim 2, characterized in that: Between sub-steps S24 and S25, there is also a sub-step that distinguishes between solid lane lines and dashed lane lines based on the characteristics of line segment length: according to Calculate the length of each line segment filtered by double constraints. ,Will Less than the preset length threshold The line segments are identified as lane dashed lines or interfering line segments and filtered out, retaining only the lines that are not lane dashed lines or interfering line segments. Not less than The line segments are used as solid lane line segments to participate in the topology fusion of sub-step S25.

5. The method for identifying vehicle lane crossing violations according to claim 1, characterized in that: The method further includes a lane line position periodic update step: setting an update time interval, and re-executing step S2 every time the update time interval is reached; if a lane center line that meets the conditions is obtained in this execution, the lane center line and its equation parameters in the cache are updated with the lane center line obtained in this execution; if a lane center line that meets the conditions is not obtained in this execution, the lane center line obtained in the last successful execution is used in the cache.

6. The method for identifying vehicle lane-crossing violations according to claim 1, characterized in that: In step S4, the boundary equation is a general form equation. ,in , These are the x-coordinate and y-coordinate in the image coordinate system, respectively. , , The equation coefficients of the straight line determined by the lane centerline; The process for determining the illegal behavior includes: obtaining the coordinates of the center point of the bounding box when the tracked vehicle target first enters the frame. and the center point coordinates of the bounding box in the current frame Substitute each value into the boundary equation to calculate the first function value. Second function value ;when Upon establishment, it was determined that the vehicle had committed a violation by crossing the line.

7. The method for identifying vehicle lane-crossing violations according to claim 2, characterized in that: In sub-step S26, when the absolute value of the difference in slope between the fused segment and the main lane segment is not greater than... When another line segment whose lateral distance from the main lane segment falls within the preset lane solid line width range is identified, these two line segments are respectively determined as the left and right boundary lines of the same lane solid line. The process of obtaining the lane center line includes: calculating the equation of the left boundary line based on the endpoint coordinates of the left and right boundary lines respectively. and the equation of the right boundary line ,in , These are the slope and intercept of the left boundary line, respectively. , Let be the slope and intercept of the right boundary line, respectively; calculate the average of the x-coordinates of the two boundary lines at the same y-coordinate to obtain the equation of the centerline. Convert the centerline equation into a general form equation. It then extends across the entire screen, serving as the center line of the lane.

8. A device for identifying vehicle lane crossing violations, used to execute the method for identifying vehicle lane crossing violations as described in claim 1, characterized in that, include: The UAV automatic task module is used to execute step S1, which controls the UAV to perform automatic flight, fixed-point hovering, recording and data transmission tasks through the UAV heterogeneous platform, and obtains video stream data of the target road segment. The lane line stability recognition module is used to execute step S2, which sequentially performs RGB color space to HSV color space conversion and color threshold filtering, Gaussian filtering and edge detection, probabilistic Hough transform, dual screening of line segment angle constraints and line segment spatial position constraints, and line segment topology fusion and extension on the video frames in the video stream data, outputs the lane center line that runs through the screen, and repeats the above processing at a preset update time interval to update the lane center line. The vehicle detection and tracking module is used to execute step S3. It uses a target detection algorithm to detect vehicle targets in the video frame and outputs the bounding box of each vehicle target. It uses a multi-target tracking algorithm to assign a unique identifier to each vehicle target, record its motion trajectory, and output the coordinates of the center point of the bounding box when each vehicle target first enters the frame and the coordinates of the center point of the bounding box in the current frame. The lane crossing behavior analysis module is used to execute steps S4 and S5. It takes the lane center line as a virtual boundary line and represents it as a boundary equation. It substitutes the coordinates of the center point of the bounding box when the vehicle first enters the screen and the coordinates of the center point of the bounding box in the current frame into the boundary equation to obtain a first function value and a second function value. When the signs of the first function value and the second function value are opposite, it determines that the vehicle target has committed a lane crossing violation, records the vehicle target, and outputs alarm information.

9. A vehicle lane-crossing violation identification device according to claim 8, characterized in that: When performing the dual screening, the lane line stability recognition module retains only line segments whose angle with the vertical direction of the image is within ±10 degrees and whose midpoint x-coordinate falls within the middle third of the image width direction; and after the dual screening and before the line segment topology fusion, it filters out line segments with a length less than a preset length threshold, and performs the line segment topology fusion and extension only on the retained solid lane line segments.

10. A vehicle lane-crossing violation identification device according to claim 8, characterized in that: The boundary behavior analysis module expresses the boundary equation as a general equation. The product of the first function value and the second function value is less than zero as a condition for determining that a line crossing violation has occurred, and the unique identifier of the vehicle target that has been determined to have committed a line crossing violation is excluded from the determination process of subsequent video frames.

Images