Vehicle track extraction method and device based on rotation detection frame, and storage medium

Abstract

The invention discloses a vehicle track extraction method and device based on a rotation detection frame and a storage medium, and belongs to the technical field of vehicle track recognition, and the method comprises the steps: obtaining a road vehicle image, and classifying the road vehicle image, and obtaining a training set; training a pre-constructed rotating target detection model by using the training set to obtain a trained rotating target detection model; acquiring a traffic flow video acquired by the unmanned aerial vehicle, and performing image stabilization processing on the traffic flow video; the trained rotating target detection model and a target tracking algorithm are used to process the traffic flow video after image stabilization processing to obtain vehicle trajectory data, and the vehicle trajectory data comprise vehicle image coordinates; converting the vehicle image coordinates into vehicle geographic coordinates through a homography matrix; and smoothing the vehicle trajectory time sequence data containing the vehicle geographic coordinates to obtain the vehicle trajectory curve, thereby solving the problem of low vehicle trajectory data extraction precision in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle trajectory recognition, and particularly relates to a vehicle trajectory extraction method, device and storage medium based on a rotation detection frame. Background Art

[0002] Vehicle target detection and trajectory extraction are core tasks in the field of traffic engineering research. They can be used to analyze traffic flow characteristics, explore complex traffic operation rules of road sections and important nodes (such as intersections, interchanges, merging and diverging points, etc.), clarify the occurrence mechanism of traffic congestion, identify traffic safety conflicts, and are of great significance for guiding traffic organization optimization, formulating traffic guidance and active control strategies, etc. Due to the characteristics of wide video monitoring range, large number of targets collected at one time, and continuous observation of unmanned aerial vehicles (UAVs) equipped with cameras, they are often used to collect traffic images and realize vehicle detection and trajectory tracking based on computer vision technology.

[0003] However, unstable factors such as wind direction and air flow will cause random translation and rotation of frame images during the process of the UAV shooting videos, resulting in phenomena such as vehicle motion overlap and coordinate drift in the videos, which pose challenges to vehicle detection and trajectory tracking. Common traditional vehicle detection and trajectory tracking methods include background extraction method, continuous video frame difference method, optical flow method, etc. However, the background extraction method is easily interfered by environmental light intensity and background changes. The continuous video frame difference method is not sensitive to slow motion, is easily affected by light changes, has a high false detection rate and poor segmentation accuracy. The optical flow method has poor anti-noise performance and a large amount of calculation. Compared with traditional object detection methods, deep learning methods are more sensitive to scale changes. However, currently, vehicle detection and trajectory tracking based on deep learning often use rectangular non-rotating detection frames. Compared with the pictures in the video, such detection frames do not detect along the vehicle contour, easily resulting in the situation of vehicle contour overlap within the target detection frame, thus reducing the accuracy of vehicle detection and trajectory tracking. Summary of the Invention

[0004] The present invention aims to provide a vehicle trajectory extraction method, device and storage medium based on a rotation detection frame. By first performing anti-shake processing on the traffic flow video captured by the UAV and using a detection model including a rotation angle prediction branch to predict target vehicles for the anti-shake processed video, and then matching and updating the vehicle trajectory according to the target prediction result, vehicle trajectory data with higher accuracy and heading angle can be obtained.

[0005] To achieve the above object, the present invention is implemented by the following technical solutions: In a first aspect, the present invention provides a vehicle trajectory extraction method based on a rotation detection frame, including: Obtaining a traffic flow video collected by a UAV and performing video stabilization processing on the traffic flow video; Process the traffic flow video after image stabilization using the trained rotated object detection model and object tracking algorithm to obtain vehicle trajectory data in the traffic flow, where the vehicle trajectory data includes vehicle image coordinates. The method for obtaining the trained rotated object detection model includes: Obtain road vehicle images and classify the road vehicle images to obtain a training set; Use the training set to train a pre-constructed rotated object detection model, and obtain the trained rotated object detection model. Convert the vehicle image coordinates to vehicle geographical coordinates through a homography matrix; Smooth the vehicle trajectory time series data containing vehicle geographical coordinates to obtain a vehicle trajectory curve.

[0006] Optionally, the step of obtaining road vehicle images and classifying the road vehicle images to obtain a training set includes: Obtain historical road vehicle images taken by a drone to obtain a historical data set; Crop the image data in the historical data set to a uniformly sized specification; Classify and label the cropped image data according to a preset target category using a labeling tool to obtain a training set( x 1 , y 1 , x 2 , y 2 , x 3 , y 3 , x 4 , y 4 , label , cls ), where,( x 1 , y 1 )、( x 2 , y 2 )、( x 3 , y 3 )、( x 4 , y 4 ) respectively represent the image coordinates of the four corner points of the rotated detection box, label represents the label, cls represents the vehicle category.

[0007] Optionally, the pre - constructed rotational object detection model includes a fusion model of the YOLOv8 network and the OBB model, wherein the CloU loss function in the existing YOLOv8 network is replaced with the ProbIoU loss function.

[0008] Optionally, training the pre - constructed rotational object detection model using the training set to obtain a trained rotational object detection model includes: Performing feature extraction on the road vehicle image data in the input training set through the YOLOv8 network to obtain predicted vehicle target bounding boxes; Predicting the rotation angle of the vehicle target bounding boxes through the OBB model to obtain rotated target boxes ( x , y , w , h , θ ), where ( x , y ) are the center point coordinates, w and h are the width and height, θ is the rotation angle of the rotated target box, representing the vehicle heading angle; Calculating the intersection - over - union IoU and the ProbIoU loss function between pairs of rotated target boxes in the prediction results, and using the NMS algorithm to filter overlapping boxes according to the intersection - over - union IoU and the ProbIoU loss function; Adjusting the parameters of the rotational object detection model according to the filtered rotated target box results.

[0009] Optionally, acquiring the traffic flow video collected by the drone and performing video stabilization processing on the traffic flow video includes: Extracting the first frame and the last frame of the traffic flow video, and using a mask to cover the roadside area in the first frame; Respectively extracting the static feature points of the first frame and the last frame and performing feature point matching to obtain a feature point template; Applying the feature point template to each frame between the first frame and the last frame to align all frames of the traffic flow video with the first frame, obtaining a stabilized traffic flow video.

[0010] Optionally, using the trained rotational object detection model and the object tracking algorithm to process the stabilized traffic flow video to obtain vehicle trajectory data in the traffic flow includes: Inputting the stabilized traffic flow video into the trained rotational object detection model to obtain the prediction results of rotated target boxes in the traffic flow; According to the rotation angle of the predicted rotated target boxesθ Perform confidence scoring; Perform the first-stage matching on the rotation target boxes with high confidence and the existing vehicle trajectories according to the intersection over union (IoU); Eliminate the background detection results of the rotation target boxes with low confidence, and perform the second-stage matching on the rotation target boxes after elimination and the unmatched existing vehicles; Update the existing vehicle trajectories for the rotation target boxes that are successfully matched in the two matching stages respectively, and initialize the rotation target boxes that are mismatched in both matching stages as new vehicle trajectories.

[0011] Optionally, the conversion of the vehicle image coordinates to vehicle geographic coordinates through the homography matrix includes: Extract the first frame of the de-shaken traffic flow video as a picture, and select several anchor points to be evenly distributed in the picture; Obtain the screen image coordinates of the anchor points ( x i , y j ) and the geographic coordinates (X i , Y j ); Solve the homography matrix H between the image coordinate system and the geographic coordinate system. The conversion formula based on the homography matrix H is: ,

[0012] where H is the homography matrix to be obtained, ( x i , y j ) are the feature point coordinates of the first frame, (X i , Y j ) are the feature point coordinates of the selected video frame; Convert the image coordinate system of the obtained vehicle trajectories to the corresponding geographic coordinate system according to the homography matrix H.

[0013] Optionally, the smoothing process of the vehicle trajectory time series data containing vehicle geographic coordinates to obtain a vehicle trajectory curve includes: Eliminate invalid trajectories and remove error detection data, and use linear interpolation to fill in some missing vehicle trajectory data; Use a Gaussian filter to perform noise reduction processing on the geographic coordinates, vehicle speeds, and vehicle heading angles of each vehicle to obtain a smooth trajectory curve.

[0014] In a second aspect, the present invention provides a vehicle trajectory extraction device based on a rotation detection frame, including: A video anti-shake module: configured to obtain a traffic flow video collected by a drone and perform image stabilization processing on the traffic flow video; A vehicle trajectory data acquisition module: configured to use a trained rotation target detection model and a target tracking algorithm to process the traffic flow video after image stabilization processing to obtain vehicle trajectory data in the traffic flow, where the vehicle trajectory data includes vehicle image coordinates, and the acquisition method of the trained rotation target detection model includes: Obtain road vehicle images and classify the road vehicle images to obtain a training set; Use the training set to train a pre-constructed rotation target detection model to obtain a trained rotation target detection model; A vehicle geographic coordinate acquisition module: configured to convert the vehicle image coordinates into vehicle geographic coordinates through a homography matrix; A vehicle trajectory curve acquisition module: configured to perform smoothing processing on the vehicle trajectory time series data including vehicle geographic coordinates to obtain a vehicle trajectory curve.

[0015] In a third aspect, the present invention provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the vehicle trajectory extraction method based on a rotation detection frame as described in any step of the first aspect.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention first performs anti-shake processing on the aerial video of the drone to eliminate the error caused by camera shake, and by adopting a target detection model based on a rotation detection frame, the rotation detection frame is closer to the edge contour line of the vehicle, reducing the redundancy of the background and the overlapping area, making the vehicle detection accuracy higher, and thus improving the extraction accuracy of vehicle trajectory parameters, providing a basis for accurately analyzing traffic flow characteristics; on the basis of the anchor-free algorithm YOLOv8, an angle prediction branch OBB is added. Adopting the anchor-free mechanism can better adapt to objects of different scales and shapes, and obtain information such as vehicle length and width that cannot be obtained by traditional target detection algorithms. The ProbIoU loss function is used to realize the recognition of targets in any angular direction, and directly predict the center point and size of the object, so that the model has better generalization ability and faster inference speed while maintaining high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The following shows a flowchart of a vehicle trajectory extraction method based on a rotation detection frame in an embodiment of the present invention;

[0018] Figure 2The following is a flowchart of a vehicle trajectory extraction method based on a rotation detection box in another embodiment of the present invention;

[0019] Figure 3 The following is a flowchart of a traffic flow video steady image processing method in an embodiment of the present invention;

[0020] Figure 4 The following is a flowchart of a method for tracking vehicle trajectories using Bytetrack in an embodiment of the present invention;

[0021] Figure 5 The following is a comparison schematic diagram of the speed-time curve and the vehicle initial speed-time curve after using a smaller standard deviation as the smoothing coefficient in an embodiment of the present invention. Detailed implementation manners

[0022] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and cannot be used to limit the protection scope of the present invention.

[0023] Embodiment 1

[0024] As Figure 1 shown, this embodiment provides a vehicle trajectory extraction method based on a rotation detection box, including:

[0025] Obtain the traffic flow video collected by the drone and perform steady image processing on the traffic flow video;

[0026] Use the trained rotation target detection model and the target tracking algorithm to process the traffic flow video after steady image processing to obtain vehicle trajectory data in the traffic flow. Among them, the vehicle trajectory data includes vehicle image coordinates. The acquisition method of the trained rotation target detection model includes:

[0027] Obtain road vehicle images and classify the road vehicle images to obtain a training set;

[0028] Use the training set to train the pre-constructed rotation target detection model to obtain the trained rotation target detection model;

[0029] Convert the vehicle image coordinates to vehicle geographical coordinates through the homography matrix;

[0030] Perform smoothing processing on the vehicle trajectory time series data including vehicle geographical coordinates to obtain a vehicle trajectory curve.

[0031] By pre - processing the aerial video of the drone to eliminate jitter, and using a detection model based on a rotating detection box to detect vehicles in the video, the boundaries of object detection are adapted to the vehicle contour, improving the accuracy of vehicle object detection. Then, using an object tracking algorithm to perform trajectory matching and updating on the detected target vehicles, a high - precision vehicle trajectory curve is obtained, which is of great significance for analyzing traffic flow characteristics.

[0032] Embodiment 2

[0033] Based on Embodiment 1, the following design is also made in this embodiment.

[0034] As Figure 2 shown, the vehicle trajectory extraction method based on eliminating drone jitter interference and a rotating detection box is divided into the following specific steps.

[0035] Step 1: Use a drone to collect traffic flow video data in the weaving area of the expressway, and perform video stabilization processing on the jittery video to eliminate the jitter caused by the drone's high - altitude shooting.

[0036] Step 2: Divide the obtained road vehicle targets into different categories; make them into a dataset available for model training; based on the pytorch deep - learning framework, use the rotating object detection model YOLOv8 - OBB for training, and obtain the YOLOv8.pt model file after training is completed; when the training result enters the convergence interval, if the mean average precision (mAP50%) under the 50% intersection - over - union threshold reaches more than 90%, the training is completed.

[0037] Step 3: Perform object detection and tracking on each frame of the video data after video stabilization processing in Step 1, and use an image - processing algorithm combining YOLOv8 - OBB and ByteTrack to extract vehicle trajectories; the data information obtained includes the planar position x, y coordinates of the vehicle, vehicle category, vehicle speed, body size, vehicle heading angle, and vehicle ID number. Save this information to different.csv files in the order of vehicle ID.

[0038] Step 4: Extract the first frame of the stabilized drone traffic flow video in Step 1 as a single picture, and then select 4 anchor points evenly distributed in the picture; use the anchor points as control points for coordinate conversion, regard the road surface within the video shooting range as a plane, mark the anchor points through a labeling tool, and call OpenCV to obtain the screen image coordinates of the anchor points( x i , y j ), and obtain the geographical coordinates of the anchor points through RTK (X i , Yj );Call the findHomography algorithm of OpenCV to solve the transformation matrix between the image coordinate system and the geographic coordinate system. This transformation matrix is called the homography matrix H; calculate the corresponding geographic coordinates from the image coordinates of the obtained vehicle trajectory through H, and store the calculation results in a new.csv file according to the vehicle ID number.

[0039] Step 5: Perform preprocessing such as data cleaning, smoothing, and noise reduction on the vehicle trajectory and heading angle data obtained in Step 3.

[0040] (1) Data cleaning

[0041] Eliminate invalid trajectories and remove error detection data, and use linear interpolation to fill in part of the missing vehicle trajectory data.

[0042] (2) Smoothing and noise reduction

[0043] Regarding problems such as the possible offset, rotation, and deformation of the target detection box, resulting in deviations between the vehicle detection center point and the detection box, use the discrete-time smoothing algorithm to smooth and denoise the trajectory data. Smooth the data at each discrete time point such as the coordinates or speeds of each vehicle to obtain a smoother trajectory curve.

[0044] When the drone shoots a video, due to influencing factors such as wind direction and air flow, the random translation, rotation, and scaling of the frame image overlap with the object movement in the video, and the extracted data is inaccurate. Therefore, it is necessary to perform video stabilization on the collected video.

[0045] As Figure 3 shown, the implementation method of video stabilization in Step 1:

[0046] (1) First, extract the first frame and the last frame of the video. Use a mask to cover the roadside building and tree areas in the first frame; then extract the static feature points of the first frame and the last frame respectively, perform feature point matching, and retain the intersection part to obtain the key points of the static objects in the frame, that is, the feature point template; then apply the feature template to each frame between the first frame and the last frame to align all frames of the video to the first frame and achieve video shake elimination.

[0047] (2) Extraction of feature points: Use the ORB feature point extraction algorithm and call the OpenCV ORB module to extract feature points in the video frame in real time. For feature point matching between two frames, use the BFMatcher algorithm to compare and match the feature points one by one. The module selects those points that fall on static objects as key points. The static feature points are further screened and optimized using the Lower's algorithm to obtain at least 4 pairs of high-quality static feature matching points. The feature template is used to screen out the static feature points that match the first frame, thereby generating a static feature point template. Call the OpenCV findHomography algorithm to calculate the homography matrix H. Use H to perform perspective transformation between the two frames. Merge each processed frame and output it as a new video.

[0048] In step 2:

[0049] Use drones to obtain vehicle trajectory data and create a data set. After cropping the original image data to a uniform size, use the labeling tool to annotate the image according to the divided target category, that is, use a polygonal box to select the location of the target in the image and specify its category name for model training. The model training .txt label file format is: ( x 1 , y 1 , x 2 , y 2 , x 3 , y 3 , x 4 , y 4 , label , cls ),in,( x 1 , y 1 )、( x 2 , y 2 )、( x 3 , y 3 )、( x 4 , y 4 ) represent the image coordinates of the four corner points of the rotation detection box, label Indicates the label, cls Indicates the vehicle category.

[0050] In Step 3:

[0051] YOLOv8-OBB Rotated Object Detection Algorithm:

[0052] Based on YOLOv8 with the Anchor-Free algorithm added, a new angle prediction branch is added, and the original CIoU loss function is replaced with the ProbIoU (Probabilistic Intersection over Union) loss function to achieve the recognition of objects in any angular direction. Compared with other object detection networks using the Anchor-Based mechanism, this model no longer relies on predefined anchor boxes but directly predicts the center point and size of the object, thus better adapting to objects of different scales and shapes, enabling the model to have better generalization ability and faster inference speed while maintaining high accuracy. The non-rotated detection boxes do not frame along the edge contour of the vehicle, while the rotated detection boxes are closer to the edge contour of the vehicle, with less redundant background in the box and fewer overlapping regions in the image, resulting in higher extraction accuracy for vehicle detection.

[0053] The rotated object detection model includes a data preprocessing module, a network module, and a post-processing module, which will be introduced in detail one by one below.

[0054] 1. Data Preprocessing Module: First, input the vehicle video image data, perform operations such as scaling and normalization on the image, and output a processed image of 640*640.

[0055] 2. Network Module: Load the weights of the model, use the preprocessed vehicle trajectory image as the input, then send the trajectory image into the trained YOLOv8 network, perform feature extraction through the network layer, output the prediction of the target bounding box, and then output the rotation angle prediction of the target bounding box through the OBB model to obtain the rotated target box. The rotated target box includes the vehicle center point coordinates, width, height, rotation angle, as well as the class and confidence. The format of the YOLOv8-OBB rotated target box is expressed as ( x 1 , y 1 , x 2 , y 2 , x 3 , y 3 , x 4 , y 4 ).

[0056] 3. Post - processing module: Calculate the intersection area of the rotation detection boxes of two vehicles. Since the rotation boxes contain rotation information, it is necessary to find the distance between the centers of the two rotation boxes and the difference in their respective angles to calculate their intersection area more precisely. Then, divide the intersection area by the difference between the sum of their respective total areas and the intersection area to obtain the Intersection over Union (IoU). Use the Non - Maximum Suppression (NMS) algorithm to filter overlapping boxes during testing. Sort a set of rotation box predictions according to probability. Starting from the rotation box with the highest probability, if its IoU score is greater than a pre - set threshold, remove other rotation boxes that overlap highly with it. This process continues until the remaining rotation boxes no longer meet the IoU threshold. During the NMS algorithm, calculate information such as the parameter variance of the bounding box to represent the uncertainty of the rotation box direction, calculate the ProbIoU loss function of two rotation boxes, and use the overlapping area of the Gaussian distribution to more accurately reflect the overlap degree between the rotation bounding boxes and determine whether to retain one of the bounding boxes.

[0057] 4. In the output data of the model, the bounding box can determine the position and size of the vehicle in the image, and the rotation angle provided by the OBB model can be used to estimate the orientation of the vehicle; the class information can identify the type of the vehicle; the trajectory information can analyze the motion state of the vehicle, such as the speed and driving direction of the vehicle; the confidence information reflects the accuracy rate of the model's prediction. Through the detected information, the vehicle trajectory can be further extracted.

[0058] ByteTrack Rotating Object Tracking Algorithm:

[0059] Since most object tracking methods set a threshold before the Kalman filter predicts the bounding box, and obtain the target ID by associating detection boxes with a score higher than the threshold, while detection boxes with a score lower than the threshold are discarded. However, low - score boxes may be boxes generated when an object is occluded. Directly discarding low - score boxes will cause problems such as missed detections or fragmented trajectories. As Figure 4 shown, adopt the ByteTrack data association method to track the vehicle trajectory through a phased data association strategy. Each rotating target box output by YOLOv8 - OBB ( x , y , w , h , θ ), where ( x , y ) is the center point coordinate, w and h are the width and height, θConfidence scores are assigned to the rotation angles, and the rotation target boxes with high confidence are matched with the existing vehicle trajectories in the first stage. The matching basis includes the spatial position and shape features of the rotation target boxes. The IoU described above is used for matching and association to update the vehicle trajectories. For the low-confidence detection boxes that fail to match in the first stage, ByteTrack does not directly discard them but enters the second stage. Combining the prediction results of the Kalman filter, these low-confidence boxes are further associated with the remaining unmatched trajectories. The historical information of the tracked trajectories is used to determine whether these low-confidence boxes may be occluded or partially blurred vehicle targets.

[0060] To reduce false detections, ByteTrack eliminates the low-confidence background detection results based on the distinguishability between the rotation boxes and the background. For vehicle detection results that may be occluded or in a motion-blurred state in complex scenarios, ByteTrack can mine these difficult samples through data association, thus effectively reducing the missed detection rate.

[0061] After two stages of matching and association processing, ByteTrack updates the corresponding trajectories for each successfully matched detection box, including updating parameters such as the rotation angle, center position, vehicle width, and height of the trajectory. For the mismatched detection boxes, the algorithm initializes them as new vehicle trajectories. Through this processing flow, ByteTrack can not only effectively retain the high-confidence targets in vehicle trajectory tracking but also capture complex vehicle behaviors caused by occlusion, blurring, etc. through low-confidence detection boxes, thereby improving the coherence of vehicle trajectories and the overall accuracy of tracking.

[0062] .csv file save format is as follows: Veh_ID, Center_X, Center_Y, Box_length, Box_width, Box_theta, Vehicle_Class; where Veh_ID is the vehicle number, Center_X and Center_Y are the horizontal and vertical coordinates of the center point of the rotation detection box in the image respectively, Box_length, Box_width, and Box_theta are the length, width, and rotation angle of the rotation detection box in pixels respectively; Vehicle_Class is the vehicle class.

[0063] In step 4:

[0064] The homography matrix is used to describe the transformation relationship between two images or the transformation relationship between a plane in the three-dimensional world and the image plane. In camera calibration, the geographic coordinate system is a three-dimensional coordinate system, and the image coordinate system is a two-dimensional coordinate system. We define the calibration board to be on the plane where the Z-axis of the geographic coordinate system is 0, with the origin located at a corner of the calibration board. The homography matrix H is used to describe the conversion relationship between the two planes. Since the coordinates of the corresponding points on the two planes are known, the H matrix can be solved. Denote the coordinates of the points on the image plane as ( x i , y j ), and the coordinates of the corresponding points in the geographic coordinate system are (X i , Y j ). Then the conversion formula based on the homography matrix H is: ,

[0065] In the formula: H is the homography matrix to be solved; ( x i , y j ) are the coordinates of the feature points in the first frame; (X i , Y j ) are the coordinates of the feature points in the selected video frame. The solution of the matrix H requires at least four pairs of known image coordinates and corresponding geographic coordinates. In the program, the findHomography function of OpenCV can be used to obtain the matrix H . After obtaining the matrix H , the conversion from the image coordinate system to the geographic coordinate system can be realized, thus ensuring that the vehicle trajectory data extracted from the UAV aerial video accurately reflects in the real-world geographic coordinate system.

[0066] In step 5:

[0067] The trajectory data obtained by detection and tracking will still have errors, resulting in fluctuations and irregularities in the trajectory curve. Therefore, Gaussian filtering is used to denoise the obtained vehicle trajectory time series data. The specific formula is as follows: ,

[0068] where, x is the data point, σ is the standard deviation, which determines the width of the Gaussian distribution. The larger the standard deviation, the more obvious the smoothing effect of the filter.

[0069] The Gaussian filter achieves noise reduction by performing a convolution operation on the image. Specifically, the convolution operation is used to apply the Gaussian function to each set of data, and the weighted average method is used to smooth the data. The calculation formula is as follows: ,

[0070] Among them, y ( i ) is the filtered data, i is the original signal point, j represents the signal points within the neighborhood, x ( i + j ) is the original data, G ( j ) is the value of the Gaussian function, k is the radius of the filter.

[0071] The specific steps are as follows:

[0072] (1)Determine the Gaussian filter parameters

[0073] Select appropriate standard deviation and the window size of the filter. The window size is taken as about 6 σ or so to ensure the effective range of the Gaussian function.

[0074] (2)Construct the Gaussian filter

[0075] According to the selected standard deviation and window size, calculate the weights of the Gaussian filter.

[0076] (3)Perform the convolution operation

[0077] Apply the Gaussian filter to the trajectory data and obtain the smoothed trajectory through the convolution operation.

[0078] However, excessive smoothing may lead to the loss of some key information (such as vehicle speed, acceleration, etc.), making the data incomplete. Therefore, in this study, a smaller standard deviation is selected as the smoothing coefficient for trajectory noise reduction. The comparison results of the speed-time curve after trajectory noise reduction and the initial speed-time curve are as Figure 5 shown.

[0079] Example 3

[0080] This example provides a vehicle trajectory extraction device based on a rotating detection frame, including:

[0081] Video anti-shake module: used to obtain the traffic flow video collected by the drone and perform image stabilization processing on the traffic flow video;

[0082] Vehicle trajectory data acquisition module: It is used to process the traffic flow video after image stabilization processing by using the trained rotation object detection model and object tracking algorithm to obtain the vehicle trajectory data in the traffic flow. Among them, the vehicle trajectory data includes vehicle image coordinates. The acquisition method of the trained rotation object detection model includes:

[0083] Obtain road vehicle images and classify the road vehicle images to obtain a training set;

[0084] Use the training set to train a pre-constructed rotation object detection model to obtain a trained rotation object detection model;

[0085] Vehicle geographic coordinate acquisition module: It is used to convert the vehicle image coordinates into vehicle geographic coordinates through a homography matrix;

[0086] Vehicle trajectory curve acquisition module: It is used to smooth the vehicle trajectory time series data containing vehicle geographic coordinates to obtain a vehicle trajectory curve.

[0087] Embodiment 4

[0088] This embodiment provides a computer storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the vehicle trajectory extraction method based on a rotated detection box described in any step of Embodiment 2.

[0089] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0090] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0091] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one or more of the processes Figure 1 one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the processes Figure 1 one or more of the processes and / or blocks Figure 1 specified in the block or blocks.

[0093] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit and scope of the present invention as protected by the claims. All of these are within the protection scope of the present invention.

Claims

1. A vehicle trajectory extraction method based on a rotating detection frame, characterized in that: include: Acquire a traffic flow video collected by a drone, and perform image stabilization processing on the traffic flow video; The traffic flow video after image stabilization is processed using the trained rotating target detection model and the target tracking algorithm to obtain vehicle trajectory data in the traffic flow, wherein the vehicle trajectory data includes vehicle image coordinates, and the method for obtaining the trained rotating target detection model includes: Acquire road vehicle images and classify the road vehicle images to obtain a training set; Using the training set to train the pre-built rotating object detection model, to obtain a trained rotating object detection model; Converting the vehicle image coordinates into vehicle geographic coordinates through a homography matrix; The vehicle trajectory time series data including the vehicle geographic coordinates is smoothed to obtain a vehicle trajectory curve.

2. The vehicle trajectory extraction method based on the rotating detection frame according to claim 1 is characterized in that: The step of acquiring road vehicle images and classifying the road vehicle images to obtain a training set includes: Obtain historical road vehicle images taken by drones from high altitudes to obtain historical data sets; Cutting the image data in the historical data set into a uniform size; Use the labeling tool to classify and label the cropped image data according to the preset target category to obtain the training set ( x 1, y 1, x 2, y 2, x 3, y 3, x 4, y 4, label , cls ),in,( x 1, y 1) ( x 2, y 2) ( x 3, y 3) ( x 4, y 4) represent the image coordinates of the four corner points of the rotation detection frame, label Indicates the label, cls Indicates the vehicle category.

3. The vehicle trajectory extraction method based on the rotating detection frame according to claim 2 is characterized in that: The pre-built rotation target detection model includes a fusion model of a YOLOv8 network and an OBB model, wherein the CloU loss function in the existing YOLOv8 network is replaced with a ProbIoU loss function.

4. The vehicle trajectory extraction method based on a rotating detection frame according to claim 3 is characterized in that: The method of using the training set to train the pre-built rotating object detection model to obtain a trained rotating object detection model includes: Performing feature extraction on the road vehicle image data in the input training set through the YOLOv8 network to obtain a predicted vehicle target bounding box; The rotation angle of the vehicle target bounding box is predicted by the OBB model to obtain the rotated target box ( x , y , w , h , θ ),in,( x , y ) are the center point coordinates, w and h for width and height, θ The rotation angle of the rotating target frame represents the vehicle heading angle; Calculate the intersection over union (IoU) and ProbIoU loss function between the two rotated target frames in the prediction results, and use the NMS algorithm to filter the overlapping frames according to the intersection over union (IoU) and ProbIoU loss function; The parameters of the rotating target detection model are adjusted according to the filtered rotating target frame result.

5. The vehicle trajectory extraction method based on a rotating detection frame according to claim 4 is characterized in that: The step of obtaining the traffic flow video collected by the drone and performing image stabilization processing on the traffic flow video includes: Extracting the first frame and the last frame of the traffic flow video, and using a mask to cover the roadside area in the first frame; Extracting static feature points of the first frame and the last frame respectively and performing feature point matching to obtain a feature point template; The feature point template is applied to each frame between the first frame and the last frame, so that all frames of the traffic flow video are aligned with the first frame, and the traffic flow video after de-jittering is obtained.

6. The vehicle trajectory extraction method based on a rotating detection frame according to claim 5 is characterized in that: The trained rotating target detection model and target tracking algorithm are used to process the traffic flow video after image stabilization to obtain vehicle trajectory data in the traffic flow, including: The de-jittered traffic flow video is input into the trained rotating target detection model to obtain the prediction result of the rotating target frame in the traffic flow; According to the predicted rotation angle of the rotating target box θ Conduct confidence scoring; The high-confidence rotated target box is matched with the existing vehicle trajectory according to the intersection over union (IoU) ratio in the first stage; The background detection results of the rotating target frame with low confidence are eliminated, and the eliminated rotating target frame is matched with the unmatched existing vehicles in the second stage; The rotated target frames that are successfully matched in the two matching stages are respectively used to update the existing vehicle trajectory, and the rotated target frames that are mismatched in both matching stages are initialized as new vehicle trajectories.

7. The vehicle trajectory extraction method based on a rotating detection frame according to claim 6 is characterized in that: The converting the vehicle image coordinates into vehicle geographic coordinates by using a homography matrix comprises: Extract the first frame of the traffic flow video after de-jittering as a picture, and select a number of anchor points evenly distributed in the picture; Get the screen image coordinates of the anchor point ( x i ,y j ) and geographic coordinates (X i ,Y j ); Solve the homography matrix H between the image coordinate system and the geographic coordinate system. The conversion formula based on the homography matrix H is: ,in, H is the required homography matrix, ( x i ,y j ) is the coordinate of the feature point of the first frame, (X i ,Y j ) are the coordinates of the feature points of the selected video frame; The image coordinate system of the acquired vehicle trajectory is converted into a corresponding geographic coordinate system according to the homography matrix H.

8. The vehicle trajectory extraction method based on a rotating detection frame according to claim 7 is characterized in that: The step of smoothing the vehicle trajectory time series data including the vehicle geographic coordinates to obtain a vehicle trajectory curve includes: Eliminate invalid trajectories and remove erroneous detection data, and use linear interpolation to fill in some missed vehicle trajectory data; A Gaussian filter is used to reduce the noise of each vehicle's geographic coordinates, vehicle speed and vehicle heading angle to obtain a smooth trajectory curve.

9. A vehicle trajectory extraction device based on a rotating detection frame, characterized in that: include: Video de-shaking module: used to obtain the traffic flow video collected by the drone and perform image stabilization processing on the traffic flow video; The vehicle trajectory data acquisition module is used to process the traffic flow video after image stabilization using the trained rotating target detection model and the target tracking algorithm to obtain the vehicle trajectory data in the traffic flow, wherein the vehicle trajectory data includes the vehicle image coordinates, and the method for acquiring the trained rotating target detection model includes: Acquire road vehicle images and classify the road vehicle images to obtain a training set; Using the training set to train the pre-built rotating object detection model, to obtain a trained rotating object detection model; Vehicle geographic coordinate acquisition module: used for converting the vehicle image coordinates into vehicle geographic coordinates through a homography matrix; Vehicle trajectory curve acquisition module: used to smooth the vehicle trajectory time series data containing the vehicle geographic coordinates to obtain the vehicle trajectory curve.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle trajectory extraction method based on a rotating detection frame as described in any one of claims 1 to 8 is implemented.

Images