Vehicle track extraction method based on roadside fixed camera
Through the vehicle trajectory extraction method based on the roadside fixed camera, the YOLOv8 network is used to perform multi-object detection and image calibration, which solves the problem of complex and low accuracy of vehicle trajectory extraction methods in the prior art, and realizes the accurate visualization of vehicle trajectory and speed and real-time monitoring.
Patent Information
- Application Number
- CN202411468489.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-21
- Publication Date
- 2025-02-18
AI Technical Summary
In the prior art, the vehicle trajectory extraction method is too complex, the hardware requirements are high, and the robustness is poor, resulting in low accuracy.
Using a vehicle track extraction method based on a roadside fixed camera, vehicle images are collected through multiple angles in multiple time periods and under multiple weather conditions, pre-processing, frame extraction and annotation, and converted to YOLOv8 network input data format, using YOLOv8 The network performs multi-object detection and visualizes the vehicle trajectory and speed through image calibration.
It realizes accurate identification and multi-object detection of vehicle trajectory, improves extraction efficiency and accuracy, can predict vehicle position and speed in real time, and provides important traffic behavior research data.
Smart Images
Figure CN119478430A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of vehicle trajectory extraction methods, and in particular to a vehicle trajectory extraction method based on a roadside fixed camera. Background Art
[0002] In recent years, a large amount of background vehicle data is needed in the training of smart cars and vehicle-road collaboration algorithms, among which vehicle trajectory data is particularly important. At present, roadside cameras are widely distributed, and it is of great significance to automatically extract vehicle motion trajectories based on roadside fixed cameras. Most of the previous vehicle trajectory extraction methods are too complicated and difficult to implement, and require certain hardware conditions. General image processing is not commonly used, and the robustness is poor, resulting in low accuracy. Summary of the invention
[0003] The present invention provides a vehicle trajectory extraction method based on a roadside fixed camera to solve the problems existing in the vehicle trajectory extraction method in the prior art.
[0004] In order to achieve the above object, the technical solution adopted by the present invention is: The vehicle trajectory extraction method based on a roadside fixed camera includes the following steps: Step 1: Using a roadside fixed camera to collect vehicle images from multiple angles under multiple time periods and multiple weather conditions, and pre-processing the collected vehicle images; Step 2: Extract the vehicle images collected in step 1 to obtain multiple images; then annotate each image to select the outer contour of the vehicle on each image; finally, convert each image into a format file that meets the input data requirements of the YOLOv8 network, thereby obtaining a data set; Step 3: Use the images in the data set of step 2 to train the YOLOv8 network, and then use the trained YOLOv8 network to process the vehicle images collected in step 1. The trained YOLOv8 network is used to select and identify the outer contour of the vehicle from the vehicle images collected in step 1, thereby completing multi-target detection; Step 4: Image calibration. The process is as follows: First, from the images of the data set obtained in step 2, select a background image with relatively clear lane lines and no obstructions, import the selected background image into Photoshop, and create a blank layer with the same resolution as the background image. Then, draw the outline of the lane lines in the background layer, transfer the outline of the lane lines to the blank layer and color it as pure black, then invert the color, and thus extract the lane line detection base map; Then, take the lane detection base map as the source image, import the modules import OpenCV, NumPy and Matplotlib in Python and load the source image, specify the corner points of the lane lines in the source image, and map the extracted corner points to the fixed points of the target image, create two NumPy arrays to store the input and output corner point coordinates, and then pass the two NumPy arrays to the cv2.getPerspectiveTransform function, and use the perspective transformation to generate the function and the transformation matrix M to describe the transformation from the lane detection base map to the target image; then use cv2.warpPerspective to apply the generated transformation matrix M to the source image, output the transformed image, and use Matplotlib to display the transformed image; finally, save the result, use the output_filename variable to save the transformed image, and output the file name and path, so that the image calibration is completed; Step 5: According to the multi-target detection results obtained in step 3, the center position of the selected rectangular box is calculated, and then the coordinate point of the center position of the rectangular box is printed in the video, and the speed is printed. The speed is printed in the upper right corner of each rectangular box, thereby realizing the visualization of the vehicle speed; Since the trajectory points of the vehicles are all the central pixel coordinate points of the vehicle rectangular frame, the motion trajectory of each vehicle in the video is presented as a continuous line composed of discontinuities. Each discontinuity is the center point position of the rectangular frame captured in each frame of the video, thereby realizing the visualization of the vehicle trajectory. After the vehicle trajectory speed is visualized, the vehicle trajectory within the specified calibration range of this stage is obtained. The coordinate system conversion is realized according to the image calibration in step 4, and the trajectory result is displayed in the position coordinate system. At the same time, the vehicle speed within the calibrated range is counted to obtain the frequency distribution histogram of the vehicle speed. The distribution of the vehicle speed in the corresponding time period can be seen through the histogram.
[0005] In the further step 1, the preprocessing includes grayscale processing, noise reduction processing, binarization processing and morphological processing.
[0006] In further steps 2 and 3, the YOLOv8 network is YOLOv8n.
[0007] In the further step 2, each image is symmetric, rotated, and brightness adjusted to expand the number of images, and then each expanded image is labeled and transformed to obtain a data set.
[0008] Compared with the prior art, the present invention has the following advantages: The present invention combines deep learning and image processing technology to establish a vehicle trajectory automatic extraction method based on the YOLO algorithm. The algorithm is flexible and simple, can achieve real-time position prediction and multi-target detection, and has accurate recognition and high efficiency. The vehicle trajectory data obtained by the present invention can also be uploaded to the cloud to make an online cloud database, and the vehicle trajectory information can be extracted in real time, which can be used as an important data source for traffic behavior research. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION
[0010] The present invention is further described below in conjunction with the accompanying drawings and embodiments.
[0011] like Figure 1 As shown, this embodiment discloses a vehicle trajectory extraction method based on a roadside fixed camera, comprising the following steps: Step 1: Image acquisition and preprocessing.
[0012] Using fixed cameras placed on the roadside, multiple sets of vehicle images were collected at different times and in different weather conditions to provide data sources under different lighting conditions for the training of the YOLO dataset. In view of the characteristics of the YOLO dataset, to prevent overfitting, images were collected from multiple angles, including the front, side, back, and oblique side of the vehicle, to increase data diversity.
[0013] Then, each group of vehicle images collected is subjected to image preprocessing, including grayscale processing, noise reduction processing, binarization processing, and morphological processing. Among them: Grayscale processing: Converting images from color to black and white or grayscale to simplify image analysis and processing and reduce the amount of data stored and transmitted.
[0014] Noise reduction processing: removes noise in the image caused by lighting, sensors, etc. This embodiment uses Gaussian filtering to smooth the image in order to better preserve the edge details of the image.
[0015] Binarization: Convert a grayscale image to a black-and-white image, that is, divide each pixel in the image into two categories (black and white) and assign a binary value (0 or 1) as the new value of the pixel.
[0016] Morphological processing: involves the extraction, transformation and recognition of morphological features of binary images and grayscale images. Morphological processing includes dilation, erosion, opening operation, closing operation and gradient operation. In this embodiment, the convolution kernel is continuously adjusted to obtain a suitable convolution kernel size, and then the image is opened, that is, the image is eroded first and then dilated.
[0017] Step 2: Dataset creation.
[0018] The vehicle images collected in step 1 are processed by frame extraction to obtain multiple pictures; each picture is then labeled to select the outer contour of the vehicle on each picture; finally, each picture is converted into a format file that meets the input data requirements of the YOLOv8 network, thereby obtaining a data set; Specifically, in this embodiment, because of the need for individual labeling, a Python algorithm is used to batch extract frames from the collected vehicle images. Most of the collected vehicle images are 2-3 minutes long, and the video frame rate is 30fps. It is set to extract a frame every 5 seconds, and all the captured videos are extracted to obtain several pictures.
[0019] Then, each image is preprocessed by symmetry, rotation, and brightness change to prevent overfitting. At this time, the number of images increases significantly.
[0020] Next, use the lambing tool to annotate all the expanded images. After annotation, an xml file is obtained. However, the YOLOv8 dataset needs to convert the xml file into a txt file. Therefore, file conversion is performed to convert the xml format images into a txt format that meets the YOLOv8 network input data format requirements. In this way, the dataset for target detection training and the annotation information of the images therein are obtained, which can be directly used as the input for YOLOv8 training.
[0021] Step 3: Vehicle inspection.
[0022] The detection part of YOLOv8 provides five models by default, namely YOLO8n, YOLOv8s, YOLOv8m, YOLOv8l, and YOLOv8x. Different models are suitable for different fields. This embodiment adopts the first YOLOv8n network. The YOLOv8n network is trained using the pictures in the data set in step 2 to obtain a trained YOLOv8n network.
[0023] During training in this embodiment, the YOLOv8n network parameters are set to: 100 epochs and 2 branches. Environment setup: Download and install Pycharm directly, then add an interpreter in Pycharm and install the ultralytics package. The built-in Pytorch and Pytorch-related packages are CPU versions, which need to be uninstalled using pip. Then, go to the Pytorch official website to install the GPU version to increase the training speed. The environment uses the virtualenv environment that comes with Python.
[0024] Then, the trained YOLOv8 network is used to process the vehicle image collected in step 1. The trained YOLOv8 network uses a rectangular frame to select and identify the outer contour of the vehicle from the vehicle image collected in step 1, thereby completing multi-target detection.
[0025] Step 4: Image calibration.
[0026] Image calibration is to convert the lane line image from the roadside fixed camera perspective to a bird's-eye view, to achieve inverse perspective transformation, that is, the conversion of the coordinate system, so that the vehicle trajectory map and speed analysis histogram in the subsequent step 5 are more universal and convenient for data processing and calling. This embodiment uses a roadside fixed camera, and the camera position remains unchanged. The calibration principle is as follows: Homography is a commonly used technology in computer vision, which is used to map an image on one plane to another plane. When processing road images, homography is usually used for inverse perspective transformation, which can convert the fixed camera shooting angle into a bird's-eye view, realize the establishment of road coordinate axes, and facilitate data analysis. This design uses Python code and OpenCV library to complete the inverse perspective transformation through homography transformation and realize image calibration.
[0027] The calibration process of this embodiment is as follows: (1) Preliminary processing of the base map. In the image of the data set in step 2, select a background image with relatively clear lane lines and no obstructions. Import the selected background image into Photoshop and create a blank layer with the same resolution as the background image. Then use the pen tool to draw the outline of the lane line in the background layer. Transfer the outline of the lane line to the blank layer and color it pure black, then invert the color to extract the lane line detection base map. Then binarize the image and save it in an array for subsequent processing.
[0028] (2) Apply the homography transformation to calculate the transformation matrix.
[0029] Take the lane detection base map as the source image, import the modules in Python: import OpenCV, NumPy and Matplotlib and load the source image (i.e. the lane detection base map), specify the corner points of the lane lines in the source image, and map the extracted corner points to the fixed points of the target image, and create two NumPy arrays to store the input and output corner point coordinates (one-to-one correspondence between the front and rear corner points). Next, pass the two NumPy arrays to the cv2.getPerspectiveTransform function, and use the perspective transformation to generate the function and the transformation matrix M (the transformation matrix is a 3x3 homography matrix) to describe the transformation from the lane detection base map to the target image. Then use cv2.warpPerspective to apply the generated transformation matrix M to the source image, output the transformed image, and use Matplotlib to display the transformed image.
[0030] Finally, save the result and use the output_filename variable to save the transformed image and the name and path of the output file, thus completing the image calibration.
[0031] Step 5: Data visualization. The process is as follows: (S1) According to the multi-target detection results obtained in step 3, the center position of the selected rectangular box is calculated, and then the coordinate point of the center position of the rectangular box is printed in the video, and the speed is printed. The speed is printed in the upper right corner of each rectangular box, thereby realizing the visualization of the vehicle speed.
[0032] Because the frame rate changes too quickly, the frame rate of the captured video is small, and because of the calibration error, the printed speed changes too much, resulting in the identified vehicle speed changing too quickly, and the printed speed value jumps and becomes unclear.
[0033] (S2) Since the trajectory points of the vehicles are all the central pixel coordinate points of the vehicle rectangular frame, the motion trajectory of each vehicle in the video is expressed as a continuous line composed of discontinuous points. Each discontinuity point is the center point position of the rectangular frame captured in each frame of the video, thereby realizing the visualization of the vehicle trajectory.
[0034] (S3) After the vehicle trajectory speed is visualized, the vehicle trajectory within the prescribed calibration range of this stage is obtained, and the coordinate system conversion is implemented according to the image calibration in step 4, and the trajectory result is displayed in the position coordinate system (i.e., the bird's-eye view after conversion, the x direction represents the horizontal direction of the road, and the y direction represents the vertical direction of the road).
[0035] It can be seen that the calibration effect of the vehicle trajectory and speed is relatively accurate when the y coordinate is small. As the y coordinate gradually increases, the component in the y direction gradually becomes close to the component in the x direction, and the x direction cannot be simply measured by uniform changes. However, the vehicle's speed component in the x direction is low. From this dimension, if only the vehicle's speed change is observed, the x direction coordinate can be ignored.
[0036] At the same time, the vehicle speeds within the calibrated range are counted to obtain a frequency distribution histogram of the vehicle speeds, through which the distribution of the vehicle speeds in the corresponding time period can be seen.
[0037] The main purpose of this embodiment is to provide an important data source for the study of traffic behavior, which includes the original trajectory, speed data, and the visualized trajectory map and speed analysis histogram. All the data obtained from the experiment are saved in a .csv file.
[0038] The preferred embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. The embodiments described in the present invention are merely descriptions of the preferred embodiments of the present invention, and do not limit the concept and scope of the present invention. The various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction, and such combinations should also be regarded as the contents disclosed in the present disclosure as long as they do not violate the concept of the present invention. In order to avoid unnecessary repetition, the present invention will not further describe various possible combinations.
[0039] The present invention is not limited to the specific details of the above-mentioned embodiments. Within the scope of the technical concept of the present invention and without departing from the design concept of the present invention, various modifications and improvements made to the technical solution of the present invention by technical personnel in this field should fall within the protection scope of the present invention. The technical contents for which protection is sought in the present invention have been fully recorded in the claims.
Claims
1. A vehicle trajectory extraction method based on a roadside fixed camera, characterized in that: The following steps are involved: Step 1: Using a roadside fixed camera to collect vehicle images from multiple angles under multiple time periods and multiple weather conditions, and pre-processing the collected vehicle images; Step 2: Extract the vehicle images collected in step 1 to obtain multiple images; then annotate each image to select the outer contour of the vehicle on each image; finally, convert each image into a format file that meets the input data requirements of the YOLOv8 network, thereby obtaining a data set; Step 3: Use the images in the data set of step 2 to train the YOLOv8 network, and then use the trained YOLOv8 network to process the vehicle images collected in step 1. The trained YOLOv8 network is used to select and identify the outer contour of the vehicle from the vehicle images collected in step 1, thereby completing multi-target detection; Step 4: Image calibration. The process is as follows: First, from the images of the data set obtained in step 2, select a background image with relatively clear lane lines and no obstructions, import the selected background image into Photoshop, and create a blank layer with the same resolution as the background image. Then, draw the outline of the lane lines in the background layer, transfer the outline of the lane lines to the blank layer and color it as pure black, then invert the color, and thus extract the lane line detection base map; Then, take the lane detection base map as the source image, import the modules import OpenCV, NumPy and Matplotlib in Python and load the source image, specify the corner points of the lane lines in the source image, and map the extracted corner points to the fixed points of the target image, create two NumPy arrays to store the input and output corner point coordinates, and then pass the two NumPy arrays to the cv2.getPerspectiveTransform function, and use the perspective transformation to generate the function and the transformation matrix M to describe the transformation from the lane detection base map to the target image; Then use cv2.warpPerspective to apply the generated transformation matrix M to the source image, output the transformed image, and use Matplotlib to display the transformed image; finally save the result, use the output_filename variable to save the transformed image, output the file name and path, and the image calibration is completed; Step 5: According to the multi-target detection results obtained in step 3, the center position of the selected rectangular box is calculated, and then the coordinate point of the center position of the rectangular box is printed in the video, and the speed is printed. The speed is printed in the upper right corner of each rectangular box, thereby realizing the visualization of the vehicle speed; Since the trajectory points of the vehicles are all the central pixel coordinate points of the vehicle rectangular frame, the motion trajectory of each vehicle in the video is presented as a continuous line composed of discontinuities. Each discontinuity is the center point position of the rectangular frame captured in each frame of the video, thereby realizing the visualization of the vehicle trajectory. After the vehicle trajectory speed is visualized, the vehicle trajectory within the specified calibration range of this stage is obtained. The coordinate system conversion is realized according to the image calibration in step 4, and the trajectory result is displayed in the position coordinate system. At the same time, the vehicle speed within the calibrated range is counted to obtain the frequency distribution histogram of the vehicle speed. The distribution of the vehicle speed in the corresponding time period can be seen through the histogram.
2. The vehicle trajectory extraction method based on a roadside fixed camera according to claim 1 is characterized in that: In step 1, preprocessing includes grayscale processing, noise reduction processing, binarization processing and morphological processing.
3. The vehicle trajectory extraction method based on a roadside fixed camera according to claim 1 is characterized in that: In steps 2 and 3, the YOLOv8 network is YOLOv8n.
4. The vehicle trajectory extraction method based on a roadside fixed camera according to claim 1 is characterized in that: In step 2, each image is symmetric, rotated, and brightness adjusted to expand the number of images, and then each expanded image is labeled and transformed to obtain a data set.