A vehicle lane-changing behavior detection method and system for a highway monitoring camera

Abstract

The application provides a vehicle lane-changing behavior detection method and system for a highway monitoring camera, and the method comprises the following steps: a multi-model collaborative perception model is constructed; a plurality of monitoring video streams are acquired; the lane where a vehicle target is located is determined according to the vehicle target position and lane line information, and trajectory information is acquired through time sequence analysis; the trajectory information is cross-camera associated and matched, a globally unique vehicle identifier is generated, and cross-camera space-time state information is constructed; whether the vehicle lane state changes is judged according to the space-time state information, and if the condition is met, the vehicle lane-changing behavior is determined. The application constructs a multi-model collaborative perception mechanism, combines vehicle detection, lane line identification and cross-camera re-identification, realizes fine modeling of vehicle space-time behavior, and makes up for the problem of insufficient utilization of space-time correlation information of complex traffic scenes in traditional methods; a stable lane determination mechanism and a multi-target tracking strategy are introduced, so that the detection jitter and misjudgment rate are reduced, and the lane-changing recognition accuracy and robustness are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle lane change behavior detection technology, and in particular to a method and system for detecting vehicle lane change behavior for highway surveillance cameras. Background Technology

[0002] With the continuous development of artificial intelligence technology in the field of intelligent transportation, vehicle behavior prediction and real-time monitoring have become core aspects of safety decision-making. Deep learning-based vehicle behavior prediction models can provide key technical support for dynamic traffic risk assessment and decision support. Among these, illegal lane changing behavior, as the second leading cause of highway accidents, still presents a challenge for existing algorithms in its dynamic detection. Traditional visual algorithms suffer from poor real-time performance and high false alarm rates in complex environments, making it difficult to meet the needs of high-precision law enforcement and early warning. Therefore, in recent years, with the rapid development of deep learning technology in the field of computer vision, more and more scholars have introduced deep learning technology into vehicle lane changing behavior detection, aiming to improve safety monitoring efficiency and reduce accident rates.

[0003] Patent CN202211559485.9 discloses a vehicle lane change recognition method, which mainly relies on the nearest neighbor classification algorithm to distinguish between lane change and non-lane change behaviors. However, such methods depend on low-level features extracted manually and lack the ability to model complex spatiotemporal contexts. When processing high-dimensional features and nonlinear lane change trajectories, it is difficult to balance detection accuracy and response speed, which easily leads to false alarms. Patent CN202211363362.8 discloses a vehicle lane crossing recognition method and system based on an in-vehicle camera. This technology focuses on lane crossing recognition from the vehicle's perspective and uses a dashcam for real-time monitoring. However, in-vehicle cameras are limited by the field of view of a single vehicle and are subject to severe self-motion interference, causing frequent coordinate system shifts. Therefore, they cannot achieve regional-level, large-scale traffic flow monitoring and have poor versatility.

[0004] In summary, existing lane change detection solutions mostly rely on vehicle-mounted terminals or radar sensors, which not only have high hardware deployment costs but also struggle to simultaneously achieve real-time performance and high accuracy from a fixed monitoring perspective. Therefore, this patent proposes a detection method for fixed monitoring points on highways that possesses high timeliness and cross-scenario generalization capabilities. Summary of the Invention

[0005] This invention aims to at least solve the technical problems existing in the prior art, and innovatively proposes a method and system for detecting vehicle lane change behavior for highway surveillance cameras.

[0006] To achieve the above-mentioned objectives of the present invention, the present invention provides a method for detecting vehicle lane-changing behavior using a highway surveillance camera, the method comprising: S1. Construct a multi-model collaborative perception model for vehicle detection, lane line recognition, and vehicle re-recognition; S2. Acquire multiple surveillance video streams covering the target road area, preprocess the video streams, and analyze the video frames based on the multi-model collaborative perception model to obtain vehicle target information and corresponding lane line information. Before analyzing the video frames, a road geometric topology model is constructed based on the camera calibration parameters, and lane line information from different camera perspectives is mapped to a unified road coordinate system to form a consistent lane topology expression across cameras. S3. Based on the vehicle target location and lane line information, determine the lane where the vehicle is located, and obtain the vehicle's trajectory information through time series analysis; The determination of the vehicle's lane is based on the accessibility constraints of the vehicle's position and the road geometric topology model of the corresponding lane, and is corrected by combining the temporal stability of the trajectory of consecutive multiple frames to obtain stable lane attributes. S4. Based on the vehicle's trajectory information, perform cross-camera association matching to generate a globally unique identifier for the vehicle and construct cross-camera spatiotemporal state information of the vehicle. In the cross-camera association matching process, a joint consistency constraint function based on trajectory speed continuity, motion direction continuity and lane topology reachability is introduced to verify the candidate matching results, and the optimal matching result is determined by a global trajectory cost minimization strategy. S5. Based on the spatiotemporal state information, determine whether the vehicle lane state has changed, and determine it as a lane change behavior when the preset conditions are met. The determination of the preset conditions is based on a temporal attention lane change probability prediction model constructed from historical trajectory sequences, which outputs the vehicle lane change probability and fuses this probability with the lane state change results for judgment.

[0007] As an optional embodiment of the present invention, the multi-model collaborative perception model may include a vehicle target detection model, a lane line key point detection and fitting model, and a re-identification model for extracting vehicle appearance features.

[0008] As an optional embodiment of the present invention, the preprocessing may optionally include video decoding, image scaling, and region of interest filtering.

[0009] As an optional embodiment of the present invention, optionally, determining the lane where the vehicle is located in step S3 includes: S301. Initial lane markings are obtained based on the vehicle position and lane line fitting results; S302. Construct a sliding window based on the initial lane markings and perform statistics on the lane markings of multiple consecutive frames; S303. Determine the stable lane attributes of the vehicle based on the statistical results.

[0010] As an optional embodiment of the present invention, the trajectory information of the vehicle is generated by a multi-target tracking algorithm, and the effective targets are filtered based on the number of consecutive occurrence frames of the targets.

[0011] As an optional embodiment of the present invention, the cross-camera association matching in step S4 may include: S401. Extract the vehicle appearance feature vector; S402. Match the feature vector with the feature library; S403. Assign or update the globally unique identifier of the vehicle based on the matching results.

[0012] As an optional embodiment of the present invention, in step S402, during the matching process, if a candidate identifier already exists in a valid record in the current camera, it is determined to be a matching conflict, and the matching result is abandoned, and a new identifier is reassigned.

[0013] As an optional embodiment of the present invention, the determination of lane-changing behavior in step S5 may include: Lane change detection across cameras is based on changes in vehicle lane status between different cameras; and / or Lane change determination is based on changes in vehicle lane status within the same camera; When a vehicle changes its stable lane within the same camera and the duration exceeds a preset threshold, it is judged as a lane-changing behavior; When a vehicle's entry lane and exit lane are inconsistent in adjacent cameras, it is judged as a lane change.

[0014] In another aspect, the present invention also provides a vehicle lane change behavior detection system for highway surveillance cameras, the system comprising: processor; Memory used to store processor-executable instructions; The processor is configured to implement the vehicle lane change behavior detection method for highway surveillance cameras when executing the executable instructions.

[0015] The beneficial effects of this invention are as follows: By constructing a multi-model collaborative perception mechanism, this invention organically combines vehicle detection, lane line recognition, and cross-camera re-recognition, achieving refined modeling of vehicle spatiotemporal behavior and effectively compensating for the insufficient utilization of spatiotemporal correlation information in complex traffic scenarios by traditional methods. Simultaneously, by introducing a stable lane determination mechanism based on sliding windows and a multi-target tracking strategy, detection jitter and false positive rates are significantly reduced, improving the accuracy and robustness of lane change recognition. Furthermore, through cross-camera vehicle correlation and global state management, unified modeling of long-distance continuous vehicle trajectories is achieved, breaking through the technical bottleneck of limited monitoring range from a single perspective. In addition, under fixed monitoring point conditions, this invention can achieve high-precision, real-time lane change behavior detection without relying on onboard equipment or high-cost sensors, possessing good engineering application value and cross-scenario generalization ability, and can effectively improve the level of highway traffic safety monitoring and intelligent early warning.

[0016] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0017] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a vehicle lane change behavior detection method for highway surveillance cameras according to the present invention; Figure 2 This is a schematic diagram of the judgment process of a vehicle lane change behavior detection method for highway monitoring cameras according to the present invention. Detailed Implementation

[0018] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0019] Example 1 like Figure 1 and 2 As shown, a method for detecting vehicle lane-changing behavior using highway surveillance cameras is provided, the method comprising: S1. Construct a multi-model collaborative perception model for vehicle detection, lane line recognition, and vehicle re-recognition; the multi-model collaborative perception model includes a vehicle target detection model, a lane line key point detection and fitting model, and a re-recognition model for vehicle appearance feature extraction.

[0020] This embodiment uses an improved YOLOv5s network as the backbone detection model for vehicle targets. To balance inference speed and accuracy, FP16 half-precision inference is enabled in GPU mode, and a warmup mechanism is configured to eliminate first-frame latency. The model output categories are strictly filtered to motor vehicle-related categories (car, bus, truck, tanker) to exclude interference from non-motorized vehicles.

[0021] The lane line key point detection and fitting model adopts the UnLanedet segmentation regression model based on ResNetBackbone for end-to-end lane line key point extraction. Combined with the LaneFit fitting algorithm, cubic spline interpolation or polynomial fitting is used to transform discrete points into smooth lane curve equations, which are then persistently stored in the database to form a static lane mapping table.

[0022] The re-identification model for vehicle appearance feature extraction constructs a ReID feature extractor based on a deep convolutional network and initializes a feature library to store the appearance feature vectors of identified vehicles and their corresponding globally unique identifiers (UIDs).

[0023] The vehicle target detection model is based on the YOLOv5s architecture, with lightweight optimization of its neck network and retention of the SPPF module to enhance feature fusion capabilities. During the training phase, a public highway dataset is used, and data augmentation strategies such as Mosaic stitching, random brightness and contrast adjustment, and horizontal flipping are employed to expand sample diversity. The loss function combines CIoU loss and class cross-entropy loss, using the AdamW optimizer with an initial learning rate of 2e-3, dynamically adjusted using a cosine annealing strategy. The training period is 120 epochs, and the final result on the validation set is a recognition accuracy of no less than 92.5%, with a single-frame inference time of ≤20ms. The lane line key point detection and fitting model is based on the UnLanedet framework. The backbone network is ResNet18 to balance accuracy and speed. The training dataset includes CULane, TuSimple, and a self-made highway lane line dataset. The input image size is uniformly 1280×720. The loss function is a weighted combination of Dice loss and L1 loss. The optimizer is Adam, with an initial learning rate of 1e-4, which decays by a factor of 10 every 30 epochs. After outliers are removed from the lane line key points output by the model using the existing RANSAC algorithm, a quadratic polynomial is used to fit and generate the lane line equation. The fitting error is controlled within ±3 pixels. The re-identification model for vehicle appearance feature extraction uses ResNet50 as its backbone and embeds a CBAM attention module to enhance feature representation capabilities. The training dataset covers VeRi-776, VehicleID, and real highway vehicle images. It adopts triplet loss combined with cross-entropy loss. During the training process, enhancement techniques such as random erasure, color jitter, and affine transformation are introduced. The feature vector dimension is set to 512 dimensions. The accuracy on the test set is no less than 92%, and the feature extraction time is ≤15ms / frame.

[0024] S2. Acquire multiple surveillance video streams covering the target road area, preprocess the video streams, and analyze the video frames based on the multi-model collaborative perception model to obtain vehicle target information and corresponding lane line information; the preprocessing includes video decoding, image scaling, and region of interest filtering.

[0025] Export real-time video streams from multiple surveillance cameras (Cam_1 to Cam_N) covering the detection section and perform the following preprocessing: Decoding and scaling: The video stream is read and decoded into a frame sequence using the OpenCV library, and the image size is scaled to 640x640 to match the model input requirements.

[0026] ROI filtering: Define a polygonal region of interest (ROI) and use the cv2.pointPolygonTest algorithm to filter out invalid vehicle targets in non-main road areas or distant backgrounds, and only process vehicles that enter the ROI.

[0027] Regarding step S2, it's also necessary to explain image enhancement optimization: For common adverse environments in surveillance scenarios such as low light, rain, and fog, an adaptive image enhancement strategy is introduced. The CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm is used to enhance the local contrast of low-light frames, improving the edge clarity of vehicles and lane lines. For rainy scenes, Gaussian blur is used to remove raindrop noise, followed by guided filtering to preserve edge information. For foggy scenes, a dark channel prior algorithm is used for defogging to ensure the accuracy of feature extraction in subsequent models.

[0028] Multi-channel video stream synchronization and parallel processing: NTP (Network Time Protocol) is used to achieve time synchronization of multiple surveillance cameras with an error controlled within ±50ms; a multi-threaded asynchronous processing framework is adopted, with independent preprocessing and model inference threads allocated to each video stream, and result aggregation is achieved through message queues to ensure the real-time performance of the system under multiple video inputs (the frame rate of single video processing is not less than 25FPS).

[0029] Model collaborative inference output: For each preprocessed image frame, firstly, the lane line key point detection and fitting model is executed, outputting the polynomial curve parameters of the lane line (such as the quadratic polynomial y=ax2+bx+c) and the lane ID; then, the vehicle target detection model is run, outputting the vehicle's bounding box, confidence score (≥0.5) and vehicle category; finally, each detected vehicle region is cropped, input into the re-identification model to extract a 512-dimensional appearance feature vector, and temporarily stored in a temporary feature cache.

[0030] S3. Based on the vehicle target location and lane line information, determine the lane where the vehicle is located, and obtain the vehicle's trajectory information through time series analysis; the vehicle's trajectory information is generated by a multi-target tracking algorithm, and effective targets are filtered based on the number of consecutive frames of target appearance.

[0031] The detection model in S1 is invoked to perform inference on each frame of the image, generating vehicle bounding boxes and confidence scores, and the following tracking logic is executed: Multi-target association. The ByteTrack tracking algorithm is integrated to perform inter-frame target association using the overlap of detection boxes (IOU). For low-score detection boxes, secondary matching is performed to solve the problems of missed detections and ID jumps caused by changes in lighting or occlusion within the tunnel.

[0032] Temporal stability filtering. A lifecycle determination mechanism is introduced, setting a frame threshold γ (γ=50 in this embodiment). Only when a target is continuously tracked for more than 50 frames is it marked as a valid target and the capture logic is triggered, capturing a vehicle image as input to the ReID module.

[0033] In step S3, it should also be noted that for each valid target, a trajectory data structure including timestamp, camera ID, vehicle bounding box center coordinates, lane ID, and appearance feature vector is established and stored in a time-series database, supporting fast querying by time range or vehicle UID. Trajectory completion in occlusion scenarios: When a vehicle temporarily disappears (≤3 frames) due to occlusion by a large vehicle in front, the existing Kalman filter algorithm is used to predict the vehicle's position and speed, combined with the historical trajectory movement trend for completion, ensuring trajectory continuity. If the occlusion time exceeds a preset threshold (e.g., 5 frames), it is marked as temporarily lost, and re-detected and re-recovered using ReID features. Real-time lane status update: During vehicle tracking, each frame calculates the vehicle's lane based on the vehicle's center coordinates and the current lane line equation. Combined with the lane ID statistics within a sliding window (window size set to 10 frames), the vehicle's stable lane attributes are dynamically updated. When more than 70% of the frames within the window belong to the same lane, the lane is confirmed as the current stable lane, reducing lane determination jitter caused by slight vehicle deviations. In addition, for multi-lane parallel scenarios on highways, a lane priority determination rule is introduced. When a vehicle crosses two lane lines at the same time, its lane is determined based on the main area where the vehicle's center of gravity is located, avoiding ambiguous determination.

[0034] S4. Based on the vehicle's trajectory information, perform cross-camera association matching to generate a globally unique identifier for the vehicle and construct cross-camera spatiotemporal state information of the vehicle. By combining the lane mapping table with vehicle coordinates, the vehicle's lane position is calculated in real time. Coordinate mapping. Obtain the coordinates of the center point of the vehicle detection box, project it onto the area defined by the lane line fitting equation, and initially determine the physical lane ID where the vehicle is currently located.

[0035] A sliding window voting mechanism is used. To eliminate lane misjudgments caused by detection jitter, a sliding window queue of length L (L=15 in this embodiment) is designed. Initial lane IDs are recorded within L consecutive frames, and a counter is used to count the mode within the window. The vehicle's stable lane attribute (stable_lane) is updated only when a particular lane ID has a dominant proportion within the window.

[0036] Lane change determination within the camera's field of view. If a vehicle's stable lane changes within the same camera's field of view, and the new lane remains stable for more than the number of confirmed frames (e.g., 10 frames), it is directly determined as a lane change event within the camera's field of view.

[0037] S5. Based on the spatiotemporal state information, determine whether the vehicle lane state has changed, and determine it as a lane change behavior when the preset conditions are met.

[0038] Design a centralized controller to receive vehicle slice data uploaded by each camera detection thread and perform cross-camera ReID matching using an asynchronous algorithm: First-camera priority matching strategy. When a vehicle appears at the entrance camera (Cam_1), the system first searches the Gallery for matching features. If a match is found, the existing UID is used (to handle cases of reverse driving or out-of-order arrival); if no match is found, the ID manager is called to assign a new UID, and the features are added to the database.

[0039] Intermediate camera conflict circuit breaker mechanism. When a vehicle appears in the intermediate camera, its features are extracted and compared with the Gallery to obtain a candidate UID. The system further checks the state cache table (state_by_uid) to determine if the candidate UID already has an active record in the current camera. If a record exists (i.e., the same UID appears in different locations in the same camera at the same time), it is determined to be an ID conflict, the matching result is forcibly abandoned, the current target is treated as a new vehicle and a new ID is assigned, preventing erroneous trajectory merging.

[0040] Last camera closed-loop and ghost car filtering. When a vehicle arrives at the exit camera, if no existing UID can be matched, it is marked as a "ghost car (Unknown)" and filtered out, and no lane change record is generated to avoid false alarms in a single frame.

[0041] Lane change logic determination based on spatiotemporal state machine Maintain a global state machine to record the final and initial lane states of each UID in real time at each camera, and determine lane changes by constructing "edge" relationships: Construct a spatiotemporal boundary. When a vehicle is captured by the current camera (Cam_Cur), search backwards for the record of that UID in the previous valid camera (Cam_Pre). Support skip comparison logic, that is, if the vehicle is missed by an intermediate camera, automatically backtrack to the most recent valid record.

[0042] Lane change condition calculation. Compare whether the lane exiting Cam_Pre and the lane entering Cam_Cur are consistent. The determination formula is: If the above conditions are met, then a cross-camera lane change record is generated by combining the timestamp and camera location information.

[0043] Redundancy filtering. The generated lane change records are post-processed to merge consecutive lane change alarms for the same UID within a short period of time, ensuring that only one complete record is generated for each lane change behavior.

[0044] Regarding step S5, it should also be noted that the system has developed a targeted handling strategy for abnormal scenarios during the lane change determination process: when a vehicle's trajectory across the camera is interrupted due to occlusion (more than 5 frames) or signal loss, a feature re-matching mechanism is triggered. This mechanism combines the motion direction, velocity vector, and appearance feature similarity (threshold set to 0.85) of the historical trajectory for secondary association. If the match is successful, the trajectory is restored and the lane change determination process continues; if the match fails, it is marked as a trajectory interruption, and no lane change record is generated. Simultaneously, lane change behavior is classified in multiple dimensions: based on the direction of the lane change, it is divided into lane changes in adjacent lanes in the same direction and lane changes across multiple lanes; based on compliance, it is divided into normal lane changes and illegal lane changes (such as lane changes occupying the emergency lane, lane changes crossing solid lines, etc.). Illegal lane changes will additionally record the violation type and the corresponding lane line attribute (such as solid line / dashed line). The lane change results include fields such as vehicle UID, starting camera ID, starting lane ID, ending camera ID, ending lane ID, lane change time interval, lane change type, and compliance status. It supports data reporting to traffic management platforms. For visualization, the lane change process is presented as a dynamic trajectory line on the monitoring screen. Normal lane change trajectories are marked in green, while illegal lane changes are marked in red. Users can click on the trajectory to view detailed vehicle information and lane change frame sequences. To ensure system performance, the feature library uses a hierarchical index optimization, dividing the feature sub-library according to camera region and time window, controlling cross-camera matching time to ≤30ms / time. Simultaneously, an automatic feature library cleanup mechanism is enabled, periodically deleting vehicle features with no active records within 7 days to release storage resources and ensure long-term stable system operation.

[0045] As an optional embodiment of the present invention, optionally, determining the lane where the vehicle is located in step S3 includes: S301. Obtain the initial lane label based on the vehicle position and lane line fitting results; In S301, the determination of the initial lane label needs to combine the coordinates of the center point of the vehicle detection box and the lane line fitting equation: For the fitted lane lines in each frame image, calculate the vertical distance from the vehicle center point to the two adjacent lane lines on the left and right, and the lane corresponding to the side with the smaller distance is the initial lane label of that frame; If the difference in distance from the center point to the two adjacent lane lines is less than a preset threshold (such as 2 pixels), the initial lane label of that frame is marked as "to be confirmed" and will not be included in subsequent statistics.

[0046] S302. Construct a sliding window based on the initial lane markings to statistically analyze lane markings across multiple consecutive frames. In S302, the sliding window adopts a first-in-first-out (FIFO) queue structure, and the window length is set to 10 frames (which can be adjusted according to the actual scenario). For each frame of image processed, the initial lane markings (not in the "pending confirmation" state) of that frame are added to the tail of the queue, and the earliest frame data at the head of the queue is removed. At the same time, the frequency of lane markings in the queue is statistically analyzed, and the number of times each lane marking appears and the maximum number of consecutive frames are recorded.

[0047] S303. Determine the stable lane attribute of the vehicle based on the statistical results. In S303, the determination rule for the stable lane attribute is as follows: when the appearance rate of a certain lane marker within the sliding window is ≥70% (i.e., at least 7 frames), that lane marker is determined as the stable lane attribute of the vehicle; if there is no lane marker with an appearance rate ≥70% within the window, the stable lane attribute of the previous frame is maintained until subsequent windows meet the condition; frames marked as "pending confirmation" are not included in the valid samples during statistics to avoid fluctuations in stable lane determination due to slight offsets near lane boundaries. Furthermore, if the initial lane markers of 3 consecutive frames are all in the same non-"pending confirmation" state, the stable lane attribute update can be triggered in advance, improving the real-time performance of lane determination.

[0048] As an optional embodiment of the present invention, the cross-camera association matching in step S4 may include: S401. Extracting Vehicle Appearance Feature Vectors: In S401, the extraction of vehicle appearance feature vectors is based on a lightweight ReID model (such as MobileNetV3-Large as the backbone network). This model completes transfer learning on a highway vehicle dataset (containing samples with different lighting, weather, and vehicle types) to ensure the discriminativeness and robustness of the features. The extraction process is as follows: When a vehicle is marked as a valid target (continuous tracking ≥50 frames), the ROI region of the vehicle detection box is extracted, and it is subjected to standardized preprocessing (including resizing to 224×224 pixels, RGB channel mean subtraction, and pixel value normalization to the range of [-1,1]). After inputting into the model, a 512-dimensional floating-point feature vector is output, and then the vector is mapped to a unit hypersphere through L2 normalization.

[0049] S402. Match the feature vector with the feature library. In S402, feature matching adopts a two-layer retrieval strategy of "local priority + global supplement": First, perform an approximate nearest neighbor search in the local feature sub-library of the current camera (using the IVF_FLAT index of the FAISS library). If a candidate feature with a similarity ≥ 0.85 is found, the match is initially determined to be successful. If there are no valid candidates in the local sub-library, the search is expanded to the global feature library for cross-regional retrieval. At this time, the similarity threshold is adjusted to 0.8 to balance the recall and precision. At the same time, spatiotemporal constraints are introduced to filter invalid candidates: if the vehicle trajectory corresponding to the candidate feature deviates from the current vehicle's movement direction by more than 30°, or the timestamp difference is greater than 15 seconds, the candidate is directly excluded to reduce the probability of false matching.

[0050] S403. Allocate or update the globally unique identifier (UID) of the vehicle based on the matching result. In S403, the allocation and update logic for the UID is as follows: If the feature matching is successful, the current vehicle's UID is synchronized to the UID corresponding to the candidate feature, and the trajectory data of this UID (including the current camera ID, stable lane ID, latest timestamp, etc.) is updated to the global time-series database; if the matching fails, the ID manager is triggered to generate a new globally unique identifier, and the new UID is bound to the current vehicle's feature vector and initial trajectory information before being written to the feature library and state cache table. Furthermore, for multi-threaded concurrent scenarios, the ID manager uses a distributed lock mechanism to ensure the atomicity of UID generation and avoid duplicate allocation; for vehicles that are temporarily lost and then re-detected, if the original UID is matched through the ReID feature, the active state of that UID is restored, and the trajectory during the interruption period is completed (based on Kalman filter prediction results).

[0051] As an optional embodiment of the present invention, in step S402, during the matching process, if a candidate identifier already exists in a valid record in the current camera, it is determined to be a matching conflict, and the matching result is abandoned, and a new identifier is reassigned.

[0052] As an optional embodiment of the present invention, the determination of lane-changing behavior in step S5 may include: Lane change detection across cameras is based on changes in vehicle lane status between different cameras; and / or Lane change determination is based on changes in vehicle lane status within the same camera; When a vehicle changes its stable lane within the same camera and the duration exceeds a preset threshold, it is judged as a lane-changing behavior; When a vehicle's entry lane and exit lane are inconsistent in adjacent cameras, it is judged as a lane change.

[0053] To ensure stable 24 / 7 system operation, a multi-level cleanup strategy is designed: Passive cleanup upon timeout: A daemon thread is started to scan all active UIDs every 60 seconds. If the last active time of a UID is more than a threshold away from the current time... (In this embodiment, the time is set to 600 seconds). Once it is determined that the vehicle has left or stopped, its memory and video memory resources are forcibly released.

[0054] Export proactive delayed data clearing: When a vehicle is confirmed to have departed by the last camera, the data is not immediately deleted, but a delay period is initiated. A delay timer (set to 300 seconds in this embodiment) is used. This window period is used to receive intermediate camera data that is delayed due to network latency, and after ensuring the integrity of the trajectory, the data is uniformly deleted by the physical cleanup program.

[0055] The beneficial effects of this invention are as follows: High-precision lane-level positioning: By introducing a deep learning model for lane line detection, combined with LaneFit curve fitting technology, and supplemented by a sliding window voting mechanism, the problem of inaccurate lane line recognition and lane misjudgment caused by vehicle detection frame jitter under complex lighting conditions in tunnels, which is effectively solved by traditional methods.

[0056] Robust cross-camera tracking capability: The innovative "conflict breaker mechanism" and "first-camera priority strategy" of this invention effectively solve the problems of ID error merging and data disorder caused by similar vehicle appearance in multi-camera environments, and significantly improve the integrity and accuracy of the evidence chain.

[0057] Lane change detection with full scene coverage: Unlike detection technologies that rely solely on a single viewpoint, this invention combines two logics: "cross-camera state comparison" and "intra-camera sliding window judgment". It can detect both long-distance lane changes across lines and short-distance lane changes across solid lines, achieving full scene coverage of lane change behavior.

[0058] Example 2 A method for detecting vehicle lane-changing behavior using a highway surveillance camera, the method comprising: S1. Construct a multi-model collaborative perception model for vehicle detection, lane line recognition, and vehicle re-recognition; the specific construction method is the same as in Example 1, and will not be elaborated further here.

[0059] S2. Acquire multiple surveillance video streams covering the target road area, preprocess the video streams, and analyze the video frames based on the multi-model collaborative perception model to obtain vehicle target information and corresponding lane line information. Before analyzing the video frames, a road geometric topology model is constructed based on the camera calibration parameters, and lane line information from different camera perspectives is mapped to a unified road coordinate system to form a consistent lane topology expression across cameras. The method for constructing the road geometric topology model in this embodiment 2 is as follows: The first step is to calibrate each surveillance camera covering the target highway section, obtain the intrinsic parameter matrix, distortion coefficient, rotation matrix and translation vector of a single camera, establish the transformation relationship from pixel coordinate system to unified road world coordinate system, and convert the lane line feature points detected from each camera's viewpoint from pixel coordinates to global road world coordinates.

[0060] The second step involves iteratively registering the nearest points of the lane line point clouds after multi-camera conversion based on the preset design lane parameters of the highway. This eliminates the system bias introduced by the calibration of different cameras and merges the overlapping lane lines in the overlapping fields of view of adjacent cameras to obtain a continuous global lane network covering the entire target road segment. At the same time, a unique global lane ID is assigned to each global lane, and a node-edge topology is constructed: the merging and diverging points of the road segment, intersections, and the endpoints of the camera coverage area are set as topology nodes, and individual lanes between adjacent nodes are set as topology edges. The topology edges store basic information such as the global ID of the corresponding lane, the camera coverage area to which it belongs, the lane line attributes (solid line / dashed line), and the lane type (regular driving lane / emergency lane / acceleration and deceleration lane).

[0061] The third step involves a manual verification and correction process for special road sections with complex lighting conditions and where calibration deviations are prone to accumulate, such as tunnel entrances / exits and bridge connections. This involves manually adjusting the coordinates of key lane line vertices and correcting topological connectivity to ensure that the final generated road geometry model is completely consistent with the actual road conditions. The completed topology model enables a unified mapping from local lane IDs to global lane IDs across different cameras, providing a unified and accurate spatial reference for determining cross-camera lane change behavior and avoiding misjudgments of lane changes due to inconsistent lane numbers across different cameras.

[0062] S3. Based on the vehicle target location and lane line information, determine the lane where the vehicle is located, and obtain the vehicle's trajectory information through time series analysis; The determination of the vehicle's lane is based on the accessibility constraints of the vehicle's position and the road geometric topology model of the corresponding lane, and is corrected by combining the temporal stability of the trajectory of consecutive multiple frames to obtain stable lane attributes. The part about obtaining stable lane attributes needs to be explained in detail here. The global road world coordinates obtained by converting the vehicle from pixel coordinates are spatially matched with the topological edge regions in the road geometric topology model: if the vehicle position falls within the lane range corresponding to a single topological edge, the global lane ID bound to that topological edge is directly used as the initial candidate lane; if the vehicle position falls into the overlapping area of ​​two adjacent topological edges due to detection errors, the Euclidean distance from the vehicle's center point to the center lines of the lanes corresponding to the two topological edges is calculated, and the lane with the shorter distance is selected as the initial candidate lane.

[0063] Based on this, stability correction is performed by combining the temporal consistency of continuous multi-frame trajectories: This embodiment adopts an adaptive sliding window statistical mechanism. The sliding window length is dynamically adjusted according to the camera frame rate. The window length is set to 8 frames at a frame rate of 25fps and 10 frames at a frame rate of 30fps, ensuring that the time interval corresponding to the statistical window is stable at around 300ms, adapting to the output parameters of different monitoring equipment models. For each frame of image processed, the obtained initial candidate lanes are added to the sliding window queue, historical data exceeding the window range are removed, and the occurrence frequency of each candidate lane is counted. When the occurrence ratio of a certain global lane ID in the window is ≥65%, the ID is directly determined as the current stable lane attribute of the vehicle. If there is no candidate lane in the current sliding window that meets the ratio requirement, the stable lane determination result of the previous frame is maintained. If two consecutive sliding windows fail to meet the ratio requirement, the current position of the vehicle is predicted by Kalman filtering based on the motion parameters of the vehicle's historical trajectory to assist in correcting the selection result of the initial candidate lane. If there is still no candidate that meets the ratio requirement after five consecutive sliding windows, the vehicle is marked as having an uncertain lane state and is temporarily not connected to the subsequent lane change determination process until a stable lane attribute is obtained before resuming the process.

[0064] This correction method combines the spatial constraints of global topology with the stability constraints of temporal statistics, which can effectively suppress lane misjudgment caused by single-frame detection errors, image blurring, and local occlusion. Compared with lane determination methods that rely solely on single-frame positions, this method can improve lane determination accuracy by more than 3%, providing a reliable spatial benchmark for the accurate determination of subsequent cross-camera lane change behavior.

[0065] S4. Based on the vehicle's trajectory information, perform cross-camera association matching to generate a globally unique identifier for the vehicle and construct cross-camera spatiotemporal state information of the vehicle. In the cross-camera association matching process, a joint consistency constraint function based on trajectory speed continuity, motion direction continuity and lane topology reachability is introduced to verify the candidate matching results, and the optimal matching result is determined by a global trajectory cost minimization strategy. In this embodiment, the cross-camera association matching problem is formalized as a constrained trajectory consistency optimization problem. For a vehicle target detected in the current camera, its candidate matching set is defined as the vehicle trajectory set in the historical global feature library, and a joint consistency constraint function is constructed, including speed consistency constraint, direction consistency constraint, and lane topology reachability constraint. The speed consistency constraint measures the deviation between the current trajectory speed and the historical average speed of the candidate trajectory; the direction consistency constraint calculates the angle difference between the vehicle motion direction vectors; and the lane topology reachability constraint determines whether the candidate vehicle satisfies the physical connectivity relationship between adjacent cameras. By weighted fusion of the above three types of constraints, a comprehensive consistency score for the candidate matching is obtained, which characterizes the association credibility between the currently detected target and the historical trajectory.

[0066] Building upon the aforementioned consistency scoring, this embodiment further constructs a global trajectory cost function for global optimization of cross-camera association results. The cost function consists of three parts: an appearance feature distance term, a spatiotemporal consistency penalty term, and an ID switching penalty term. The appearance feature distance term measures the Euclidean or cosine distance between ReID feature vectors; the spatiotemporal consistency penalty term constrains the time interval and spatial displacement of the vehicle between different cameras to meet reasonable kinematic ranges; and the ID switching penalty term additionally penalizes frequent ID changes of the same vehicle to suppress trajectory fragmentation caused by erroneous associations. By linearly weighting and combining these three cost terms, a global trajectory matching cost function is obtained, and an optimization problem is constructed with minimizing this function as the objective.

[0067] In the solution process, this embodiment employs the Hungarian algorithm or a greedy optimization strategy to solve the global trajectory cost matrix, selecting the matching relationship with the minimum cost from the candidate matching set as the final cross-camera association result. Once the optimal matching result is determined, the current vehicle target is bound to its corresponding globally unique identifier, and its cross-camera spatiotemporal state information is updated synchronously, including the camera sequence in which the vehicle is located, entry and exit timestamps, stable lane status, and historical trajectory path. Simultaneously, candidate matching results that fail to meet the constraints are removed from the candidate set and added to the re-identification cache queue, continuing to participate in matching in subsequent camera frame sequences. This improves the trajectory recovery capability under long-term occlusion or missed detection conditions, thereby enhancing the overall stability and robustness of cross-camera association.

[0068] S5. Based on the spatiotemporal state information, determine whether the vehicle lane state has changed, and determine it as a lane change behavior when the preset conditions are met. The determination of the preset conditions is based on a temporal attention lane change probability prediction model constructed from historical trajectory sequences, which outputs the vehicle lane change probability and fuses this probability with the lane state change results for judgment.

[0069] It should be noted that, firstly, the trajectory features of the target vehicle in the most recent consecutive T frames (T is 20 in this embodiment) are extracted, including the vehicle's global position coordinates, instantaneous speed, heading angle, distance of the vehicle's center point from the center line of the adjacent lane, and the attribute of the current lane (solid line / dashed line), totaling 5 basic features, which constitute a temporal feature sequence input model of length T.

[0070] The model backbone employs a stacked one-dimensional convolutional layer + multi-head temporal attention block structure: the one-dimensional convolutional layer is used to extract local motion features of the trajectory over short distances, capturing the lateral displacement trend of the vehicle during lane changing; the multi-head temporal attention block learns the attention weights of trajectory features from different frames, strengthening the contribution of motion features in the current frame while weakening the interference of irrelevant historical trajectories in the early stages, thus accurately capturing the change in motion trend after the lane change is initiated. At the model output, two fully connected layers are connected to finally output the probability value of the current vehicle's lane change behavior, ranging from [0,1].

[0071] During the training phase, 100,000 labeled highway vehicle driving samples (including 40,000 positive lane-changing samples and 60,000 negative normal driving samples) were used to train the model parameters end-to-end. The binary cross-entropy loss was used as the optimization objective. After training, the model's inference time per frame was less than 1ms, which can meet the real-time detection requirements of multiple highway monitoring videos.

[0072] After outputting the lane change probability, the probability result is fused with the lane state change detection result from the previous step. The specific rules are as follows: First, if a clear lane state change has been detected (stable lane change within the same camera / inconsistent lanes across cameras), when the model outputs a lane change probability ≥ 0.3, it is judged as a real lane change behavior, and low-probability false detection results are filtered out. Second, if no clear lane state change is detected, but the model outputs a lane change probability ≥ 0.7 for 3 consecutive frames, it means that the vehicle has initiated a lane change but has not yet completed a stable lane change. At this time, it is marked as a suspected lane change, and the trajectory is continuously tracked until the lane state stabilizes before a final judgment is made. Third, if the lane change probability is below 0.2 for 5 consecutive frames, even if there are brief fluctuations in lane candidates, it is judged as detection noise and no lane change judgment is triggered.

[0073] By mining the motion trend features of vehicles before lane changes using a temporal attention model and combining them with hard judgment results of lane state changes, the false judgment rate of lane changes in scenarios such as single-frame detection error and partial lane line occlusion is effectively reduced. Compared with the judgment method based solely on lane state changes, the F1 score of lane change detection in this invention is improved, further enhancing the detection accuracy and robustness of the system.

[0074] Example 3 A vehicle lane change behavior detection system for highway surveillance cameras includes: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to perform vehicle lane change behavior detection for highway surveillance cameras when executing the executable instructions. It should be noted that the computer device includes a processor and memory, and may also include one or more of the following: multimedia components, input / output (I / O) interfaces, and communication components. The processor controls the overall operation of the computer device, completing all or part of the steps of vehicle lane change behavior detection for highway surveillance cameras. The memory stores various types of data to support device operation and can be implemented by volatile or non-volatile storage devices or combinations thereof, such as SRAM, EEPROM, etc. The multimedia components include a screen (such as a touch screen) and an audio component. The audio component has a microphone to receive external audio signals and includes at least one speaker to output audio signals. The I / O interface provides an interface for the processor and other interface modules (such as a keyboard, mouse, buttons, etc.). The communication component is used for wired or wireless communication between devices. Wireless communication includes Wi-Fi, Bluetooth, etc., and the communication component includes a Wi-Fi module, etc. As a preferred option, the computer device can be implemented using electronic components such as ASICs and DSPs to execute a vehicle lane change behavior detection method for highway surveillance cameras.

[0075] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting vehicle lane-changing behavior using highway surveillance cameras, characterized in that, The method includes: S1. Construct a multi-model collaborative perception model for vehicle detection, lane line recognition, and vehicle re-recognition; S2. Acquire multiple surveillance video streams covering the target road area, preprocess the video streams, and analyze the video frames based on the multi-model collaborative perception model to obtain vehicle target information and corresponding lane line information. Before analyzing the video frames, a road geometric topology model is constructed based on the camera calibration parameters, and lane line information from different camera perspectives is mapped to a unified road coordinate system to form a consistent lane topology expression across cameras. S3. Based on the vehicle target location and lane line information, determine the lane where the vehicle is located, and obtain the vehicle's trajectory information through time series analysis; The determination of the vehicle's lane is based on the accessibility constraints of the vehicle's position and the road geometric topology model of the corresponding lane, and is corrected by combining the temporal stability of the trajectory of consecutive multiple frames to obtain stable lane attributes. S4. Based on the vehicle's trajectory information, perform cross-camera association matching to generate a globally unique identifier for the vehicle and construct cross-camera spatiotemporal state information of the vehicle. In the cross-camera association matching process, a joint consistency constraint function based on trajectory speed continuity, motion direction continuity and lane topology reachability is introduced to verify the candidate matching results, and the optimal matching result is determined by a global trajectory cost minimization strategy. S5. Based on the spatiotemporal state information, determine whether the vehicle lane state has changed, and determine it as a lane change behavior when the preset conditions are met. The determination of the preset conditions is based on a temporal attention lane change probability prediction model constructed from historical trajectory sequences, which outputs the vehicle lane change probability and fuses this probability with the lane state change results for judgment.

2. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, The multi-model collaborative perception model includes a vehicle target detection model, a lane line key point detection and fitting model, and a re-identification model for extracting vehicle appearance features.

3. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, The preprocessing includes video decoding, image scaling, and region of interest filtering.

4. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, Determining the vehicle's lane in step S3 includes: S301. Initial lane markings are obtained based on the vehicle position and lane line fitting results; S302. Construct a sliding window based on the initial lane markings and perform statistics on the lane markings of multiple consecutive frames; S303. Determine the stable lane attributes of the vehicle based on the statistical results.

5. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, The vehicle's trajectory information is generated using a multi-target tracking algorithm, and valid targets are filtered based on the number of consecutive frames in which the targets appear.

6. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, The cross-camera association matching in step S4 includes: S401. Extract the vehicle appearance feature vector; S402. Match the feature vector with the feature library; S403. Assign or update the globally unique identifier of the vehicle based on the matching results.

7. A method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 6, characterized in that, During the matching process in step S402, if a candidate identifier already exists in a valid record in the current camera, it is determined to be a matching conflict, and the matching result is abandoned and a new identifier is reassigned.

8. The method for detecting vehicle lane-changing behavior for highway surveillance cameras as described in claim 1, characterized in that, Determining lane-changing behavior in step S5 includes: Lane change detection across cameras is based on changes in vehicle lane status between different cameras; and / or Lane change determination is based on changes in vehicle lane status within the same camera; When a vehicle changes its stable lane within the same camera and the duration exceeds a preset threshold, it is judged as a lane-changing behavior; When a vehicle's entry lane and exit lane are inconsistent in adjacent cameras, it is judged as a lane change.

9. A vehicle lane change behavior detection system for highway surveillance cameras, characterized in that, The system includes: processor; Memory used to store processor-executable instructions; The processor is configured to implement, when executing the executable instructions, the vehicle lane change behavior detection method for highway surveillance cameras as described in any one of claims 1 to 8.

Images