Vehicle track splicing method for long road section cross-camera target association matching

Abstract

The invention provides a vehicle track splicing method for long road section cross-camera target association matching. The method comprises the following steps: acquiring an image including a target vehicle, and constructing a training image data set and a verification image data set; capturing a single-camera vehicle track by using a YOLOV5 model and a Kalman filtering algorithm; and splicing the cross-camera vehicle tracks by using a vehicle re-recognition algorithm. According to the invention, vehicle positioning and track splicing can be realized more accurately, and support is provided for vehicle motion mode analysis, traffic flow monitoring and abnormal behavior detection; a target vehicle can be quickly and accurately identified through a YOLOV5 model and a Kalman filtering algorithm, and target tracking is carried out on the vehicle; through coordinate conversion, the difference of visual angles is eliminated through a unified spatial reference system, and the error or inconsistency of image features is avoided, so that the positioning and the track of the vehicle can be more accurately spliced; and the vehicle is re-identified through a vehicle re-identification algorithm, so that the identification accuracy is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic image splicing, and in particular to a vehicle track splicing method for cross-camera target association matching over a long road section. Background Art

[0002] As current road construction continues to accelerate, high-speed traffic flow states with safety risks have always been the focus of supervision. In long-distance high-speed tunnel groups, it is often necessary to achieve vehicle supervision over large spans, so the demand for cross-camera vehicle trajectory capture and splicing in traffic flow supervision is increasing. This technology is mainly used to accurately identify and track vehicles in each surveillance camera in large-span scenarios, and associate the trajectories of the same vehicle in multiple surveillance cameras to splice and obtain the full trajectory. This technology can obtain vehicle trajectories over long spans through visual methods, which is of great significance in applications such as vehicle anomaly detection and accident replay.

[0003] Existing cross-camera association methods are often achieved by switching cameras by determining the current camera position of the vehicle. This association only considers the image coordinate system and cannot obtain the entire trajectory of the vehicle in the actual world coordinate system. It is difficult to apply it to real-world tasks such as anomaly detection and actual positioning of anomalies. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a vehicle trajectory splicing method for cross-camera target association matching over a long road section.

[0005] The technical solution of the present invention is: a vehicle track splicing method for long road section cross-camera target association matching, comprising the following steps:

[0006] S1), acquiring an image including a target vehicle and constructing a training image dataset and a verification image dataset;

[0007] S2), using the YOLOV5 model and Kalman filter algorithm to capture the single camera vehicle trajectory;

[0008] S3) Use the vehicle re-identification algorithm to stitch the vehicle trajectories across cameras.

[0009] Preferably, in step S2), capturing the vehicle trajectory with a single camera specifically includes the following steps:

[0010] S21), fine-tune the YOLOV5 model using the training image dataset;

[0011] S22), then using the fine-tuned YOLOV5 model to identify the vehicle in the monitoring image, and obtain the identified vehicle bounding box;

[0012] S23), using a Kalman filter algorithm to track the vehicle under a single camera to obtain an image coordinate trajectory of the vehicle within the field of view of the single camera;

[0013] S24), according to the camera parameters, the coordinates in each camera image coordinate system are converted to the same world coordinate system, so as to obtain the world coordinate trajectory of the vehicle.

[0014] Preferably, in step S3), the cross-camera vehicle trajectory stitching specifically includes the following steps:

[0015] S31), using a vehicle re-identification algorithm to associate target vehicles under multiple cameras to ensure that multiple vehicle tracks identified by a single camera belong to the same vehicle;

[0016] S32), splicing the vehicle trajectories identified by multiple single cameras in the same coordinate system and performing smoothing filtering, and outputting the vehicle trajectory of the entire process in the multi-camera scene.

[0017] The beneficial effects of the present invention are:

[0018] 1. The present invention can more accurately realize the positioning of vehicles and the splicing of trajectories, and provide support for vehicle movement pattern analysis, traffic flow monitoring and abnormal behavior detection;

[0019] 2. The present invention can quickly and accurately identify the target vehicle through the YOLOV5 model;

[0020] 3. The present invention uses Kalman filter algorithm to track the vehicle target;

[0021] 4. The present invention eliminates the difference in viewing angles through coordinate transformation and a unified spatial reference system, and provides consistency in physical space, thereby avoiding errors or inconsistencies in image features, so that the positioning and trajectory of the vehicle can be spliced ​​more accurately;

[0022] 5. The present invention re-identifies the vehicle through a vehicle re-identification algorithm to ensure the accuracy of identification, thereby improving the accuracy of monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 It is a schematic diagram of the process of the present invention.

[0024] Figure 2 This is a technical effect diagram of the identification and tracking method of the present invention. DETAILED DESCRIPTION

[0025] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings:

[0026] like Figure 1 and 2As shown, this embodiment provides a vehicle trajectory splicing method for cross-camera target association matching over a long road section, comprising the following steps:

[0027] S1), acquiring an image including a target vehicle and constructing a training image dataset and a verification image dataset;

[0028] S2), capturing vehicle trajectory with a single camera;

[0029] S3), cross-camera vehicle trajectory stitching.

[0030] As a preferred embodiment of the present invention, in step S1), the image of the target vehicle is obtained by using Opencv to loop through each frame of the surveillance camera video stream. In the present embodiment, in order to complete the vehicle re-identification task in a multi-camera scene, firstly, single-camera vehicle detection and tracking is used to identify all vehicles in adjacent single-camera scenes, thereby obtaining all candidate set images of the vehicle.

[0031] As a preferred embodiment of this invention, in step S2), capturing the vehicle trajectory with a single camera specifically includes the following steps:

[0032] S21), fine-tune the YOLOV5 model using the training image dataset;

[0033] S22), then using the fine-tuned YOLOV5 model to identify the vehicle in the monitoring image, and obtain the identified vehicle bounding box;

[0034] S23), using a Kalman filter algorithm to track the vehicle under a single camera to obtain an image coordinate trajectory of the vehicle within the field of view of the single camera;

[0035] S24), according to the camera parameters, the coordinates in each camera image coordinate system are converted to the same world coordinate system, so as to obtain the world coordinate trajectory of the vehicle.

[0036] As a preferred embodiment of the present invention, step S21) fine-tuning the YOLOV5 model using the training image data set specifically includes the following steps:

[0037] S211), using LabelImg to label the images in the training image dataset in a format recognizable by the YOLOv5 model, including the category label and the bounding box coordinates of the target;

[0038] S212), loading the pre-trained YOLOv5 model weights as a starting point for fine-tuning to accelerate the model training process and improve model performance;

[0039] S213), fine-tune the YOLOv5 model using the labeled training image dataset, and optimize the model parameters through the back propagation algorithm until the YOLOv5 model meets the preset requirements on the verification image dataset.

[0040] As a preferred embodiment of this invention, in step S22), the vehicle in the monitoring image is identified using the fine-tuned YOLOV5 model to obtain an identified vehicle boundary box, which specifically includes the following steps:

[0041] S221), using Opencv to loop and read each frame of the surveillance camera video stream;

[0042] S222), input the read image into the fine-tuned YOLOV5 model to obtain the bounding box coordinates of the identified vehicle.

[0043] As a preferred embodiment of the present invention, step S23) uses a Kalman filter algorithm to track the vehicle under a single camera to obtain an image coordinate trajectory of the vehicle within the field of view of the single camera; specifically includes the following steps:

[0044] S231), initializing the Kalman filter, setting the vehicle coordinates as the state variables of the vehicle, and setting the initial position state of the vehicle and the corresponding covariance matrix;

[0045] S232), using the prediction step of the Kalman filter to predict the current state and covariance matrix according to the previous state of the vehicle;

[0046] S233), in the new image frame, using the vehicle recognition result to update the state estimation and covariance matrix of the Kalman filter, calculate the Kalman gain, and update the state estimation and error covariance matrix;

[0047] S234), the Kalman filter continuously outputs the state variable vehicle coordinates, thereby outputting the coordinate trajectory of the vehicle.

[0048] As a preferred embodiment of the present invention, in step S24), the coordinates in each camera image coordinate system are converted to the same world coordinate system according to the camera parameters, so as to obtain the world coordinate trajectory of the vehicle, specifically:

[0049] First, convert the image coordinate system to the camera coordinate system, that is:

[0050]

[0051] Among them, x c ,y c 、z c They represent the x-axis coordinate, y-axis coordinate, and z-axis coordinate in the camera coordinate system respectively; f is the focal length of the camera; x f ,y fRepresents image coordinates;

[0052] The camera coordinate system is then translated and rotated to obtain the world coordinate system:

[0053]

[0054] In the formula, [X c ,Y c ,Z c ] represents the camera coordinates; [X w ,Y w ,Z w ] represents the world coordinate; R=R1R2R3 is the rotation matrix; T is the translation matrix.

[0055] in,

[0056]

[0057] Where R1 represents the coordinate system around Z c Axis rotation angle θ; R2 represents the coordinate system around X c Axis rotation angle R3 represents the coordinate system around Y c The axis rotates by an angle ω.

[0058] As a preferred embodiment of this embodiment, in step S3), the cross-camera vehicle trajectory splicing specifically includes the following steps:

[0059] S31), using a vehicle re-identification algorithm to associate target vehicles under multiple cameras to ensure that multiple vehicle tracks identified by a single camera belong to the same vehicle;

[0060] S32), splicing the vehicle trajectories identified by multiple single cameras in the same coordinate system and performing smoothing filtering, and outputting the vehicle trajectory of the entire process in the multi-camera scene.

[0061] As a preferred embodiment of the present invention, in step S31), the vehicle re-identification algorithm is used to associate the target vehicles under multiple cameras to ensure that multiple vehicle tracks identified by a single camera belong to the same vehicle; specifically, the following steps are included:

[0062] S311), adding the identified vehicles to the candidate set and storing them in the database;

[0063] S312), feature matching the currently identified vehicle in real time with the vehicles in the candidate set;

[0064] S313), according to the matching similarity, retrieve the ID of the vehicle for real-time identification.

[0065] As a preferred embodiment of the present invention, in step S32), the acquisition of the vehicle trajectory during the whole process specifically includes the following steps:

[0066] S321), extracting vehicle trajectories with the same ID from different cameras according to the vehicle re-identification algorithm;

[0067] S322), the vehicle trajectory is filtered using a smoothing filter;

[0068] This embodiment uses the smoothing capability of the Kalman filter to smooth the trajectory of the vehicle to reduce the impact of noise and prediction errors;

[0069] S323) According to the sequence of the cameras, an orderly spliced ​​vehicle trajectory across multiple cameras is obtained.

[0070] As a preferred embodiment of the present invention, the database stores vehicle ID, vehicle feature information, and vehicle track information.

[0071] The above embodiments and descriptions are only for illustrating the principles and best embodiments of the present invention. Without departing from the spirit and scope of the present invention, the present invention may be subject to various changes and improvements, all of which fall within the scope of the present invention to be protected.

Claims

1. A vehicle trajectory splicing method for cross-camera target association matching over a long road section, characterized in that: The following steps are involved: S1), acquiring an image including a target vehicle and constructing a training image dataset and a verification image dataset; S2), using the YOLOV5 model and Kalman filter algorithm to capture the single camera vehicle trajectory; S3) Use the vehicle re-identification algorithm to stitch the vehicle trajectories across cameras.

2. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 1 is characterized by: In step S2), capturing the single-camera vehicle trajectory specifically includes the following steps: S21), fine-tune the YOLOV5 model using the training image dataset; S22), using the fine-tuned YOLOV5 model to identify the vehicle in the monitoring image and obtain the identified vehicle bounding box; S23), using a Kalman filter algorithm to track the vehicle under a single camera to obtain an image coordinate trajectory of the vehicle within the field of view of the single camera; S24), according to the camera parameters, the coordinates in each camera image coordinate system are converted to the same world coordinate system, so as to obtain the world coordinate trajectory of the vehicle.

3. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 1 is characterized by: In step S3), the cross-camera vehicle trajectory splicing specifically includes the following steps: S31), using a vehicle re-identification algorithm to associate target vehicles under multiple cameras to ensure that multiple vehicle tracks identified by a single camera belong to the same vehicle; S32), splicing the vehicle trajectories identified by multiple single cameras in the same coordinate system and performing smoothing filtering, and outputting the vehicle trajectory of the entire process in the multi-camera scene.

4. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 1 is characterized by: In step S1), the image of the target vehicle is obtained by using Opencv to cyclically read each frame of the surveillance camera video stream.

5. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 2 is characterized in that: Step S21) specifically includes the following steps: S211), using LabelImg to label the images in the training image dataset in a format recognizable by the YOLOv5 model, including the category label and the bounding box coordinates of the target; S212), loading the pre-trained YOLOv5 model weights as a starting point for fine-tuning to accelerate the model training process and improve model performance; S213), fine-tune the YOLOv5 model using the labeled training image dataset, and optimize the model parameters through the back propagation algorithm until the YOLOv5 model meets the preset requirements on the verification image dataset.

6. The vehicle trajectory splicing method for long-section cross-camera target association matching according to claim 2 is characterized by: Step S23), using the Kalman filter algorithm to track the vehicle under a single camera to obtain the image coordinate trajectory of the vehicle within the field of view of the single camera; specifically comprising the following steps: S231), initializing the Kalman filter, setting the vehicle coordinates as the state variables of the vehicle, and setting the initial position state of the vehicle and the corresponding covariance matrix; S232), using the prediction step of the Kalman filter to predict the current state and covariance matrix according to the previous state of the vehicle; S233), in the new image frame, using the vehicle recognition result to update the state estimation and covariance matrix of the Kalman filter, calculate the Kalman gain, and update the state estimation and error covariance matrix; S234), the Kalman filter continuously outputs the state variable vehicle coordinates, thereby outputting the coordinate trajectory of the vehicle.

7. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 2 is characterized by: In step S24), the coordinates of each camera image coordinate system are converted to the same world coordinate system according to the camera parameters, so as to obtain the world coordinate trajectory of the vehicle, specifically: First, convert the image coordinate system to the camera coordinate system, that is: Among them, x c ,y c 、z c They represent the x-axis coordinate, y-axis coordinate, and z-axis coordinate in the camera coordinate system respectively; f is the focal length of the camera; x f ,y f Represents image coordinates; The camera coordinate system is then translated and rotated to obtain the world coordinate system: In the formula, [X c ,Y c ,Z c ] represents the camera coordinates; [X w ,Y w ,Z w ] represents the world coordinate; R=R1R2R3 is the rotation matrix; T is the translation matrix. in, Where R1 represents the coordinate system around Z c Axis rotation angle θ; R2 represents the coordinate system around X c Axis rotation angle R3 represents the coordinate system around Y c The axis rotates by an angle ω.

8. The vehicle trajectory splicing method for long-section cross-camera target association matching according to claim 3 is characterized by: Step S31) specifically includes the following steps: S311), adding the identified vehicles to the candidate set and storing them in the database; S312), feature matching the vehicle currently identified in real time with the vehicles in the candidate set; S313), according to the similarity of the matching, retrieve the ID for real-time identification of the vehicle.

9. The vehicle trajectory splicing method for long-section cross-camera target association matching according to claim 8 is characterized by: Step S32) specifically includes the following steps: S321), extracting vehicle trajectories with the same ID from different cameras according to the vehicle re-identification algorithm; S322), the vehicle trajectory is filtered using a smoothing filter; S323) According to the sequence of the cameras, an orderly spliced ​​vehicle trajectory across multiple cameras is obtained.

10. The vehicle trajectory splicing method for long road section cross-camera target association matching according to claim 8, characterized in that: The database stores vehicle ID, vehicle feature information, and vehicle track information.

Images