Vehicle target detection tracking and trajectory data extraction method based on deep learning

Abstract

The invention discloses a vehicle target detection tracking and trajectory data extraction method based on deep learning, and relates to the technical field of unmanned aerial vehicle aerial photography. The method comprises the following steps: carrying out stable frame processing on an unmanned aerial vehicle video, extracting feature points and feature vectors through an SURF algorithm, matching and screening reliable matching pairs through an FLANN algorithm, calculating a homography matrix through an RANSAC algorithm when a condition is met, carrying out perspective transformation to eliminate jitter, and outputting a stable video sequence; vehicle target detection: introducing an AIFI module to construct an improved YOLOv5OBB model, and outputting vehicle rotation bounding box parameters and categories after training; vehicle tracking and trajectory extraction are carried out, cross-frame tracking is realized based on a DeepSORT model, original trajectory data are preprocessed, and the speed, the acceleration, the additional lane number and the ID of an adjacent vehicle are calculated. According to the method, the problems of video jitter and insufficient detection precision are effectively solved, and the accuracy and continuity of track data extraction in the highway scene are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of unmanned aerial vehicle aerial photography, and particularly relates to a vehicle target detection tracking and trajectory data extraction method based on deep learning. BACKGROUND

[0002] With the rapid development of intelligent transportation and automatic driving technology, vehicle trajectory data, as the core basis for analyzing traffic flow characteristics, optimizing road control strategies, and verifying automatic driving algorithms, its accuracy and reliability are increasingly critical. As a typical representative of high-speed and high-density traffic scenes, the driving state of vehicles on highways is complex, with different sizes of vehicles such as large trucks and small cars, and various target forms such as dense arrangement and arbitrary direction distribution due to lane changing and following driving behaviors, which poses strict requirements for high-precision trajectory data extraction.

[0003] Currently, the trajectory extraction method based on unmanned aerial vehicle aerial photography combined with deep learning has become the mainstream technical path, which can capture the complete traffic flow state of the entire highway due to its high-altitude perspective and large coverage. However, the existing technology still faces two major bottlenecks in practical application: on the one hand, the unmanned aerial vehicle aerial photography process is easily affected by air flow disturbance, mechanical vibration, and maneuvering flight, resulting in high-frequency jitter in the collected video sequences, causing unstable inter-frame geometric correspondence and image blur, which seriously damages the quality of the basic data for target detection and tracking, and reduces the temporal continuity and spatial accuracy of trajectory extraction; on the other hand, the existing target detection model uses horizontal bounding boxes (HBB) for target positioning, which is difficult to adapt to the arbitrary rotation posture and dense arrangement characteristics of vehicles in the highway scene, easily introduces a large amount of background noise, causes target position deviation and direction information loss, and dense vehicles are easily misjudged and excluded due to bounding box overlap, with high miss and false detection rates. Even if the horizontal rectangular box is replaced by the rotated bounding box (OBB), there are still problems such as angle prediction ambiguity and unstable detection of small targets, which cannot meet the high-precision extraction requirements of trajectory data for target spatial position and direction information. In addition, the local vehicle trajectory dataset for the highway scene is relatively scarce, and the existing technology is mostly adapted to urban roads or foreign road scenes, further limiting the optimization and landing application of the highway trajectory extraction technology. SUMMARY

[0004] In view of the defects in the prior art, the present application provides a vehicle target detection tracking and trajectory data extraction method based on deep learning, comprising the following steps: S1, UAV video frame stabilization processing: obtain the highway vehicle driving video of the UAV aerial photography, take the first frame of the video as the initial reference frame, convert the gray scale of the initial reference frame, and then extract the feature points and feature vectors by the SURF algorithm; read the subsequent frames one by one, perform the same gray scale conversion and SURF feature extraction operation on the frames, match the feature points of the reference frame and the frames to be processed by the FLANN algorithm, and select the reliable matching pairs; determine whether the number of reliable matching pairs is not less than 4 pairs, if yes, calculate the homography matrix by the RANSAC algorithm, perform perspective transformation on the frames to be processed based on the matrix to eliminate jitter, and update the transformed frames as new reference frames; if not, directly retain the frames to be processed as stable frames; loop the above steps to output a stable video sequence; S2, vehicle target detection: introduce an AIFI module, construct an improved YOLOv5_OBB rotating target detection model; collect highway vehicle images containing different weather conditions and different vehicle densities, construct a vehicle training set, and train the improved YOLOv5_OBB model using the vehicle training set until the model converges; input each frame of the stable video sequence into the trained improved YOLOv5_OBB model, and output the rotating bounding box parameters of the vehicle and the vehicle category, the rotating bounding box parameters including the center point coordinates , length , width , and azimuth angle ; S3, vehicle tracking and trajectory extraction: construct a DeepSORT multi-target tracking model, construct a training set based on the rotating bounding box and train the model to obtain tracking weights; realize cross-frame vehicle tracking by the trained model, and output original trajectory data; preprocess the original trajectory data, calculate the vehicle speed and acceleration, and add lane number and adjacent vehicle ID information to obtain final vehicle trajectory data.

[0005] Further, the step of extracting feature points by the SURF algorithm in S1 includes: S101, calculate the integral image of the gray image: first, construct the integral image, take the first pixel position in the original gray image as the coordinate origin O, establish a coordinate system with the right as the positive x-axis and the downward as the positive y-axis, and the sum of the gray values of all points in the rectangular region from the top left corner of the image to any point is the value of the pixel I(x, y).

[0006] S102, constructing a scale space based on a Hessian matrix and a box filter: the scale space is constructed by multiple groups and multiple layers, and different scale response images are generated by gradually increasing the template size of the box filter. Specifically, different groups keep the original resolution of the input image unchanged, but the basic template size of the box filter is gradually expanded from group to group. In the same group, each layer uses a gradually increasing box filter template size (such as 9x9, 15x15, 21x21...) to realize multi-scale response. For each point in the scale space, the determinant approximation value of the Hessian matrix of the point is calculated using a box filter of a specified size, and the value represents the feature intensity of the point at the current scale.

[0007] S103, comparing each pixel with 26 feature points in the neighborhood of the two-dimensional space and the scale space, and if it is greater than the other 26 points, it is positioned as a preliminary key point, and then filtered by a set threshold to obtain a final strong feature point; S104, taking the strong feature point as the center and 6s (s is the feature point scale) as the radius to draw a circle, and defining a 60-degree sector region in the circular field, and by rotating the sector region, the total sum of Haar wavelet features in each sector region is counted, and finally the direction with the maximum Haar wavelet feature sum is selected as the main direction of the feature point; S105, selecting a square frame with a side length of 20s x 20s along the main direction, dividing it into 4x4 sub-blocks, and calculating the response value of each sub-block using a 2s size Haar template, and then counting the response value to form a 64-dimensional feature vector.

[0008] Further, the step of RANSAC feature point matching in S1 comprises: S111, random sampling: randomly sampling 4 pairs of non-collinear feature points from reliable matching pairs for calculating an initial projection transformation matrix; S112, model assumption: calculating a candidate homography matrix based on the 4 pairs of matching points; S113, inlier detection: distinguishing inliers and outliers based on the criterion that the re-projection error is less than a preset threshold; S114, model evaluation: comparing the number of inliers of the current model with that of the historical optimal model, and recording the model parameters when the number of inliers is the most; S115, iteration termination: when the preset iteration number is reached or the inlier ratio meets the requirement, the iteration is terminated, and the matrix corresponding to the optimal model is taken as the final homography matrix.

[0009] Further, in the S1 unmanned aerial vehicle video frame stabilization processing, the FLANN algorithm matches the feature points by using a k-nearest neighbor algorithm (K=2) to find the best match, and filters reliable matching pairs through distance ratio testing; and the reference frame update adopts an incremental strategy, where after processing a frame of stable frame, it is set as a new reference frame, and the cache gray image and SURF feature points and feature vectors are implemented for incremental processing.

[0010] Further, the AIFI module includes a global attention mechanism, which can model global pixel dependencies through a Transformer Encoder, better understand the overall shape and direction features of the rotating target, and thus improve the accuracy of vehicle heading angle extraction.

[0011] Further, the training set of the DeepSORT model is constructed in the following manner: the rotating bounding box is converted into a PascalVOC format label, the vehicle image dataset is cut based on the label, and the training set and test set are divided in a ratio of 8:2; the DeepSORT model introduces a deep appearance feature extraction network and a cascaded matching strategy, and adopts motion information and appearance information fusion matching.

[0012] Further, the preprocessing of the original trajectory data includes the following steps: S31, the coordinates of the top point of the bounding box are converted into the corresponding center point coordinates through geometric transformation, and the information is rearranged in the order of video frame sequence number, vehicle ID, center point X-axis coordinate, center point Y-axis coordinate, vehicle length, and vehicle width. The relationship of the above coordinate conversion is shown in equations (1) and (2): ; ; In the formula, is the center point X-axis coordinate; is the bounding box top point X coordinate; is the vehicle length; is the center point Y-axis coordinate; is the bounding box top point Y coordinate; is the vehicle width; S32, abnormal data rejection, for non-target road vehicle and short-term invalid trajectory, a double cleaning mechanism is adopted: first, the target road boundary is calibrated by manual calibration, and the out-of-bound trajectory points are removed; secondly, based on the trajectory continuity, a minimum trajectory point number threshold is set to filter out the trajectory segments with too few frames, and the spatial correlation and time sequence integrity of the data are ensured; S33, edge data repair, based on the average length of the vehicle, dynamically corrects the vehicles close to the left and right boundaries of the image. When the distance between the vehicle center and the boundary is less than half of the original vehicle length, the vehicle length is calibrated to the average value and the center point position is adjusted symmetrically. The left boundary is 0 pixels and the right boundary is 5120 pixels. S34, coordinate transformation, the transformation ratio is determined by the ratio of the actual road width D to the corresponding number of image pixels d, and the pixel coordinates are converted into real geographic coordinates. The transformation ratio r is calculated as shown in equation (3): ; Where D is taken as 3.75m and d as 44.25 pixels, r is calculated to be 0.084745762712; S35, Data smoothing: The trajectory data is smoothed using the Simple Moving Average (SMA) method, and its calculation formula is shown in (4): ; Among them, S t Let be the predicted value (or smoothed value) at time t; w is the window size, i.e., the average time span of the calculation; A i This represents the actual observed value at time i. S36 downsamples the original vehicle trajectory data by taking odd-numbered frames, reducing the 50Hz data to 25Hz, and extracting vehicle motion features, including speed, acceleration, lane number, and adjacent lane vehicle ID data. The vehicle speed is based on the time difference between video frames. The coordinate difference between consecutive frames is taken as the displacement and substituted into formulas (5) and (6) to calculate the speed. The speed data is then smoothed using the simple moving average method described in S35. ; ; Among them, v x,t The center point of the target border in frame t. Directional velocity; x c,t f is the x-coordinate of the center point of the target border in frame t; t v is the sequence number of the t-th frame; FPS is the video frame rate; y,t Let y be the velocity of the center point of the target bounding box in the t-th frame in the y-th direction; c,t Let be the ordinate of the center point of the target border in frame t.

[0013] The acceleration calculation formulas are shown in (7) and (8), and the acceleration data is then smoothed using the simple moving average method described in S35: ; ; wherein, a x,t is the x-direction acceleration of the target bounding box center point of the t frame; a y,t is the y-direction acceleration of the target bounding box center point of the t frame.

[0014] S37, additional lane and adjacent vehicle information, divide the road into multiple lane areas and match the lane number for each data, and assign 0 if the matching is unsuccessful; record the nearest front and rear vehicle ID in the same lane and adjacent lane for each vehicle, and calculate the distance between vehicles; if the distance exceeds the set threshold, the corresponding ID is recorded as 0, otherwise the adjacent vehicle ID is retained.

[0015] Compared with the prior art, the application has the advantages that: 1. The application introduces the random sample consensus algorithm (RANSAC) on the basis of the SURF algorithm, which can quickly and stably extract high-quality feature points that are not sensitive to scale, rotation and illumination changes, provide a reliable basis for motion estimation, effectively eliminate false matches, improve matching accuracy, and ensure accurate estimation of the motion transformation model.

[0016] 2. On the basis of retaining the advantages of YOLOv5_OBB rotating frame, the application innovatively introduces an AIFI module, reconstructs the feature interaction mechanism, further improves the position accuracy of the rotating frame, eliminates angle prediction ambiguity, and enhances the robustness of small target detection.

[0017] 3. The application realizes cross-frame vehicle target association based on the detection result through DeepSORT model training, generates a continuous trajectory; and improves the tracking robustness in occlusion and dense scenes by training and optimizing the tracking weight. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flow chart of a vehicle target detection, tracking and trajectory data extraction method based on deep learning; Figure 2 is a flow chart of a video stabilization technology based on SURF and RANSAC; Figure 3 is a data set extraction flow chart based on the DeepSORT model. DETAILED DESCRIPTION

[0019] To make the purpose, advantages and characteristics of the application more obvious, the application will be further described in detail below in combination with the drawings and specific embodiments. It should be emphasized that the above drawings and the following description are only exemplary and are not intended to limit the scope and application of the application.

[0020] The core objective of the application is to solve the problems of shaking of unmanned aerial vehicle aerial video, insufficient vehicle detection accuracy and discontinuous trajectory extraction in the expressway scene, and through an end-to-end technical scheme of "unmanned aerial vehicle video frame stabilization processing -> vehicle target detection -> vehicle tracking and trajectory extraction", high-precision vehicle trajectory data extraction is realized. The whole process takes the unmanned aerial vehicle aerial video as the input, after frame stabilization optimization, accurate detection and tracking preprocessing, the trajectory data containing key information such as vehicle position, speed, acceleration and lane number are output, which is suitable for intelligent traffic analysis, automatic driving algorithm verification and other scenes. The specific process of the application is as follows.

[0021] I. Unmanned aerial vehicle video frame stabilization processing During the unmanned aerial vehicle aerial process, video shaking is easily caused by airflow disturbance and mechanical vibration, which leads to unstable inter-frame geometric relationship. In this step, feature points are extracted by SURF algorithm and matching is optimized by RANSAC algorithm to realize video frame stabilization. The specific implementation is as follows: First, the basic parameters of the video are analyzed, including frame rate, resolution (width x height) and codec format, and an output video file container is created to ensure that the processed video parameters are consistent with the original file. The first frame of the video stream is set as the initial reference frame, which serves as the basis for subsequent frame processing. The reference frame is converted to grayscale, and the RGB three-channel weighted algorithm is used to calculate the grayscale value. Then the SURF (Speeded-Up Robust Features) feature extraction process of the reference frame is performed: (1) Key point detection: based on Hessian matrix to locate the significant feature points of the image; (2) Feature description: generate a 64-dimensional feature vector to describe the neighborhood of the feature point; (3) Direction assignment: determine the main direction of the feature point to ensure rotation invariance.

[0022] After completing the feature extraction of the reference frame, the following processing flow is executed in a frame-by-frame loop before the end of the video: (1) Read the current frame, sequentially read the next frame as the frame to be processed; (2) Grayscale conversion: use the same algorithm as the reference frame to perform grayscale processing on the current frame; (3) SURF feature extraction: perform the same SURF feature extraction on the current frame as the reference frame; (4) FLANN feature matching: use the FLANN (Fast Library for Approximate Nearest Neighbors) algorithm to match the feature points of the reference frame and the current frame, use the k-nearest neighbor algorithm (K=2) to find the best match, and apply the distance ratio test (threshold 0.7) to filter reliable matching pairs; (5) Pairing point quantity decision: determine whether the number of reliable matching points is not less than 4 pairs. If the condition is met (homography matrix calculation requires at least 4 point pairs matching points), steps (6) to (7) are continued; otherwise, the original frame is directly written into the output video; (6) RANSAC calculation: calculate the homography matrix through the RANSAC (RANdom Sample Consensus) algorithm. Randomly extract 4 pairs of matching points to calculate the initial projection transformation matrix, evaluate the inliers within the re-projection error threshold, and obtain the optimal inlier set after iterative optimization; (7) Perspective transformation stabilization: apply the obtained homography matrix to the current frame for projection transformation, eliminate the displacement and rotation relative to the reference frame, and output the image with the original size; (8) Write stable frame: write the processed frame (whether it has been transformed or not) into the output video file, and retain the original timestamp information; (9) Update reference frame: set the current frame as the new reference frame, save the grayscale image and cache the SURF feature points and descriptors, and realize incremental processing.

[0023] Release resources after video processing: close the video read-write stream and clear the cache data. Finally, output the stable video file, which has the same format and resolution as the original video, effectively eliminates picture jitter, and maintains time axis continuity.

[0024] II. Model training and improvement The present application optimizes the structure of YOLOv5_OBB model, replaces the SPPF (Spatial Pyramid Pooling-Fast) module in the original YOLOv5_OBB model with AIFI (Attention-based Information Fusion Module) module to improve the feature extraction capability of rotating vehicle targets. The AIFI module enhances the global context feature correlation through self-attention mechanism, and better adapts to the geometric deformation of rotating targets through variable convolution structure, thereby improving the regression accuracy and direction perception ability of rotating bounding boxes.

[0025] In the model training stage, a training set containing 3158 images and a test set containing 370 images are used. The training is 180 rounds. Experimental results show that the model with the introduction of AIFI module has a significantly higher improvement speed of multiple indicators such as precision (Precision), recall (Recall), mAP50 and mAP50:95 than the original YOLOv5_OBB benchmark model in the training process.

[0026] The results show that, under the precision and recall indicators, AIFI enhances the global context modeling capability through self-attention, making the model tend to detect more potential targets, thus improving the recall rate; at the same time, due to the increase in sensitivity, some false positives may be introduced, resulting in a slight decrease in precision; under the mAP50 indicator, the introduction of the AIFI module makes the original model increase from 0.945 to 0.961, indicating that the detection quality has improved; under the mAP50:95 indicator, the introduction of the AIFI module makes the original model increase from 0.597 to 0.615, reflecting the comprehensive positioning accuracy of the rotated bounding box under multiple intersection-over-union thresholds, especially the enhancement of high-precision positioning capability.

[0027] III. Extraction of the dataset First, an image dataset for training the tracking weights of the DeepSORT model is constructed. The target detection dataset label is converted from YOLO format to PascalVOC format. In the VOC format, each image corresponds to an XML file, which stores the image size, channel number, target class, and bounding box position information. Based on the vehicle position recorded in the XML file, 12649 small vehicle images and 4251 large vehicle images are extracted from the original images through an image segmentation algorithm. After completing the segmentation, the images are divided into training and test sets in an 8:2 ratio and stored in the designated model directory.

[0028] After completing the dataset construction, the DeepSORT model is trained to obtain the vehicle tracking weights. The training process outputs a video file and a TXT document recording the original vehicle motion trajectory. The trajectory data is arranged in ascending order of frame number and ascending order of vehicle ID within the same frame, with each row containing six fields: video frame number, vehicle ID, bounding box top-left vertex X coordinate, bounding box top-left vertex Y coordinate, vehicle pixel length, and vehicle pixel width.

[0029] To improve data quality, the following preprocessing operations are performed on the initial trajectory data: (1) Center point position calculation The bounding box vertex coordinates are converted to corresponding center point coordinates through geometric transformation, and the information arrangement order is reorganized as video frame sequence number, vehicle ID, center point X-axis coordinate, center point Y-axis coordinate, vehicle length, and vehicle width. The relationship of the above coordinate conversion is as follows: ; ; In the formula, is the center point X-axis coordinate; is the bounding box vertex X-axis coordinate; is the vehicle length; is the center point Y-axis coordinate; For the Y coordinate of the bounding box vertex; For the vehicle length; (2) Abnormal data rejection For non-target road vehicles and short-lived invalid trajectories, a double cleaning mechanism is adopted: first, the boundary of the target road is calibrated manually to remove out-of-bound trajectory points; second, based on trajectory continuity, a minimum trajectory point number threshold is set to filter out trajectory segments with too few frames, ensuring the spatial relevance and temporal integrity of the data.

[0030] (3) Edge data repair To address the issue of incomplete vehicle bounding boxes at the image edges, the average vehicle length is calculated as a reference to dynamically correct vehicles near the left and right boundaries of the image (left boundary 0, right boundary 5120 pixels). When the distance between the vehicle center and the boundary is less than half the original vehicle length, the vehicle length is adjusted to the average value, and the center point position is symmetrically adjusted to improve the positioning accuracy of edge vehicles.

[0031] (4) Coordinate conversion The original coordinate format of vehicle trajectory data is in pixels. Therefore, it is necessary to convert the pixel data to real coordinates to obtain the actual coordinates and dimensions of the vehicle. In this study, the conversion coefficient is determined by calculating the ratio of the real size of the reference object to its pixel size. If the actual road distance D corresponds to the number of image pixels d, the conversion ratio r is calculated as shown in equation (3): ; According to the People's Republic of China Industry Standard "Highway Route Design Specification", the width of an 8-lane two-way road is 30m, i.e. 3.75m per lane. The width of a 4-lane road corresponds to 177 pixels, i.e. 44.25 pixels per lane. Therefore, the conversion ratio r = D / d = 0.084745762712.

[0032] (5) Data smoothing Simple Moving Average (SMA) method is used to smooth the trajectory data. SMA is a classic data smoothing method that calculates the average value within a fixed time period to determine the overall trend. The core idea is to evenly distribute the weights: the data of the last N time points is averaged with the same proportion to reduce the impact of short-term data fluctuations, thereby highlighting the long-term trend. Compared with the exponential smoothing method which uses a decreasing weight distribution over time, the average weight method of SMA is slower in responding to sudden changes, but it is simpler to calculate and has stronger anti-interference ability.

[0033] The core calculation formula of SMA is shown in equation (4): ; Where S t t is the predicted value (or smoothed value) at time t; w is the window size, i.e. the time span for averaging; Ai This represents the actual observed value at time i. (6) Data downsampling The original vehicle trajectory data is downsampled by taking odd-numbered frames, reducing the 50Hz data to 25Hz.

[0034] The preprocessed vehicle trajectory data can be used to extract vehicle motion features, including speed, acceleration, lane number, and adjacent lane vehicle ID data. The vehicle speed calculation steps are as follows: Based on the time difference of the video frame sequence, the coordinate difference between the preceding and following frames is taken as the displacement, and the vehicle speed can be obtained by substituting it into the speed formula. The speed formulas are shown in (5) and (6): ; ; Among them, v x,t The center point of the target border in frame t. Directional velocity; x c,t f is the x-coordinate of the center point of the target border in frame t; t v is the sequence number of the t-th frame; FPS is the video frame rate; y,t Let y be the velocity of the center point of the target bounding box in the t-th frame in the y-th direction; c,t Let be the ordinate of the center point of the target border in frame t.

[0035] The acceleration calculation formulas are shown in (7) and (8), and the acceleration data is then smoothed using the simple moving average method described in S35: ; ; Among them, a x,t Let a be the acceleration in the x-direction at the center point of the target bounding box in frame t; y,t Let be the acceleration in the y-direction of the center point of the target border in frame t.

[0036] S37, add lane and adjacent vehicle information, divide the road into multiple lane areas and match lane numbers for each data, and assign a value of 0 if no match is found; record the IDs of the nearest vehicles in the same lane and adjacent lanes for each vehicle, and calculate the distance between vehicles; if the distance exceeds a set threshold, the corresponding ID is recorded as 0, otherwise the adjacent vehicle ID is retained.

[0037] The above embodiments have provided a detailed description of the technical solution of the present invention. Obviously, the present invention is not limited to the described embodiments. Based on the embodiments of the present invention, those skilled in the art can make various modifications, but any modifications that are equivalent to or similar to the present invention fall within the scope of protection of the present invention.

[0038] That which is not described in detail in the specification is considered to be of prior art to those skilled in the art.

Claims

1. A method for vehicle target detection, tracking, and trajectory data extraction based on deep learning, characterized in that, Includes the following steps: S1, UAV video frame stabilization processing: Acquire aerial video of vehicles traveling on a highway captured by a UAV. Using the first frame of the video as the initial reference frame, perform grayscale conversion on the initial reference frame, and then extract feature points and feature vectors using the SURF algorithm. Read subsequent frames to be processed frame by frame, and perform the same grayscale conversion and SURF feature extraction operation on the frames to be processed. Match the feature points of the reference frame and the frames to be processed using the FLANN algorithm to obtain reliable matching pairs. Determine whether the number of reliable matching pairs is not less than 4 pairs. If it is, calculate the homography matrix using the RANSAC algorithm, and perform perspective transformation on the frames to be processed based on the matrix to eliminate jitter. Update the transformed frame as the new reference frame. If it is not, directly retain the frames to be processed as stable frames. Repeat the above steps to output a stable video sequence. S2, Vehicle Target Detection: An AIFI module is introduced to construct an improved YOLOv5_OBB rotating target detection model; highway vehicle images with varying weather conditions and vehicle densities are collected to build a vehicle training set; the improved YOLOv5_OBB model is trained using this training set until convergence; each frame of the stable video sequence is input into the trained improved YOLOv5_OBB model, which outputs the vehicle's rotation bounding box parameters and vehicle category. The rotation bounding box parameters include coordinates of the center point. ,length ,width , and represent azimuth angles ; S3, Vehicle Tracking and Trajectory Extraction: Construct a DeepSORT multi-target tracking model, build a training set based on the rotated bounding box and train the model to obtain tracking weights; The trained model enables cross-frame vehicle tracking and outputs raw trajectory data. The original trajectory data is preprocessed to calculate vehicle speed and acceleration, and lane number and neighboring vehicle ID information are added to obtain the final vehicle trajectory data.

2. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, The steps for extracting feature points using the SURF algorithm in S1 include: S101, Calculate the integral image of the grayscale image: First, construct the integral image. Take the first pixel position in the original grayscale image as the origin O, establish a coordinate system with the positive x-axis to the right and the positive y-axis downward. The sum of the grayscale values ​​of all points in the rectangular area formed from the upper left corner of the image to any point is the I(x,y) value of that pixel. S102, Scale space is constructed based on Hessian matrix and box filter: The scale space is constructed by multiple groups and multiple layers. Different scale response images are generated by gradually increasing the template size of the box filter. Specifically, different groups keep the original resolution of the input image unchanged, but the basic template size of the box filter is increased group by group. Within the same group, each layer uses a gradually increasing box filter template size to achieve multi-scale response. For each point in the scale space, the determinant of its Hessian matrix is ​​calculated using a box filter of a specified size. This value represents the feature intensity of the point at the current scale. S103: Compare each pixel with 26 feature points in the two-dimensional space and scale space neighborhood. If it is greater than the other 26 points, locate it as a preliminary key point. Then, after filtering with different thresholds, the final strong feature points are obtained. S104. Draw a circle with the strong feature point as the center and 6s (s is the feature point scale) as the radius. Define a 60-degree sector area within this circular area. By rotating the sector area, calculate the sum of Haar wavelet features in each sector area. Finally, select the direction with the largest sum of Haar wavelet features as the main direction of the feature point. S105, Select a square frame with a side length of 20s×20s along the main direction, divide it into 4×4 sub-blocks, calculate the response value of each sub-block using a 2s-sized Haar template, and statistically calculate the response values ​​to form a 64-dimensional feature vector.

3. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, The steps of RANSAC feature point matching in S1 include: S111, random sampling: randomly select 4 pairs of non-collinear feature points from the reliable matching pairs to calculate the initial projection transformation matrix; S112, Model Assumptions: Calculate the candidate homography matrix based on the four extracted pairs of matching points; S113, Interior point detection: Distinguish between interior points and exterior points based on the reprojection error being less than a preset threshold. S114, Model Evaluation: Compare the number of interior points of the current model with the historical best model, and record the model parameters when the number of interior points is the maximum. S115, Iteration Termination: When the preset number of iterations is reached or the proportion of interior points meets the requirements, the iteration is terminated, and the matrix corresponding to the optimal model is taken as the final homography matrix.

4. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, In the S1 UAV video frame stabilization process, the FLANN algorithm uses the k-nearest neighbor algorithm (K=2) to find the best match when matching feature points, and filters reliable matching pairs through distance ratio test; and the reference frame update adopts an incremental strategy. After processing each stable frame, it is set as the new reference frame, and the grayscale image and SURF feature points and feature vectors are cached to achieve incremental processing.

5. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, The AIFI module includes a global attention mechanism, which can model global pixel dependencies through Transformer Encoder to better understand the overall shape and directional features of rotating targets, thereby improving the accuracy of vehicle heading angle extraction.

6. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, The training set of the DeepSORT model is constructed as follows: the rotated bounding box is converted into Pascal VOC format labels, and the vehicle image dataset is obtained by segmenting based on the labels. The training set and test set are divided in an 8:2 ratio. The DeepSORT model introduces a deep appearance feature extraction network and a cascaded matching strategy, and adopts motion information and appearance information fusion matching.

7. The method for vehicle target detection, tracking, and trajectory data extraction based on deep learning according to claim 1, characterized in that, The preprocessing of the raw trajectory data includes the following steps: S31, the coordinates of the bounding box vertices are converted into the corresponding center point coordinates through geometric transformation, and the information is rearranged into video frame sequence number, vehicle ID, center point X-axis coordinate, center point Y-axis coordinate, vehicle length, and vehicle width. The relationship of the above coordinate transformation is shown in equations (1) and (2): ; ; In the formula, The center point's X-axis coordinates; The x-coordinate of the bounding box vertex; For vehicle length; The center point's Y-axis coordinate; The Y-coordinate of the bounding box vertex; For vehicle width; S32, Abnormal data removal: For vehicles on non-target road sections and invalid trajectories that appear briefly, a dual cleaning mechanism is adopted: First, the target road boundary is manually marked to remove trajectory points that cross the boundary; Second, based on the continuity of the trajectory, a minimum threshold for the number of trajectory points is set to filter out trajectory segments with too few frames, ensuring the spatial correlation and temporal integrity of the data. S33, edge data repair, based on the average length of the vehicle, dynamically corrects the vehicles close to the left and right boundaries of the image. When the distance between the vehicle center and the boundary is less than half of the original vehicle length, the vehicle length is calibrated to the average value and the center point position is adjusted symmetrically. The left boundary is 0 pixels and the right boundary is 5120 pixels. S34, coordinate transformation, the transformation ratio is determined by the ratio of the actual road width D to the corresponding number of image pixels d, and the pixel coordinates are converted into real geographic coordinates. The transformation ratio r is calculated as shown in equation (3): ； Where D is taken as 3.75m and d as 44.25 pixels, r is calculated to be 0.084745762712; S35, Data smoothing: The trajectory data is smoothed using the Simple Moving Average (SMA) method, and its calculation formula is shown in (4): ； Among them, S t Let be the predicted value (or smoothed value) at time t; w is the window size, i.e., the average time span of the calculation; A i This represents the actual observed value at time i. S36 downsamples the original vehicle trajectory data by taking odd-numbered frames, reducing the 50Hz data to 25Hz, and extracting vehicle motion features, including speed, acceleration, lane number, and adjacent lane vehicle ID data. The vehicle speed is based on the time difference between video frames. The coordinate difference between consecutive frames is taken as the displacement and substituted into formulas (5) and (6) to calculate the speed. The speed data is then smoothed using the simple moving average method described in S35. ； ； Among them, v x,t The center point of the target border in frame t. Directional velocity; x c,t f is the x-coordinate of the center point of the target border in frame t; t v is the sequence number of the t-th frame; FPS is the video frame rate; y,t Let y be the velocity of the center point of the target bounding box in the t-th frame in the y-th direction; c,t Let y be the ordinate of the center point of the target border in frame t; The acceleration calculation formulas are shown in (7) and (8), and the acceleration data is then smoothed using the simple moving average method described in S35: ； ； Among them, a x,t Let a be the acceleration in the x-direction at the center point of the target bounding box in frame t; y,t The acceleration in the y-direction of the center point of the target border in frame t; S37, add lane and adjacent vehicle information, divide the road into multiple lane areas and match lane numbers for each data, and assign a value of 0 if no match is found; record the IDs of the nearest vehicles in the same lane and adjacent lanes for each vehicle, and calculate the distance between vehicles; if the distance exceeds a set threshold, the corresponding ID is recorded as 0, otherwise the adjacent vehicle ID is retained.

Images