Lane Change Trajectory Classification Method Based on UAV Aerial Videos

Abstract

The present invention discloses a lane-changing trajectory classification method based on UAV aerial video, comprising the following steps: obtaining the rectangular frame of the position of the vehicle in a single-frame image of an autonomous vehicle and the coordinates of 4 BoundingBox points; obtaining the trajectory coordinates of each BoundingBox point and the historical lane-changing trajectory map of the vehicle; establishing a data set for each first sub-segment; dividing the historical lane-changing trajectory segment of the autonomous vehicle to obtain the first sub-segment; establishing a standard data set for each first sub-segment, dividing each standard data set according to the lane-changing stage to obtain a sub-standard data set; obtaining a standard two-dimensional image of vehicle lane-changing based on the sub-standard data set; obtaining the running data of the vehicle to be tested, obtaining a two-dimensional image of the lane-changing of the vehicle to be tested, and judging the lane-changing trajectory type of the vehicle to be tested; The present invention can effectively distinguish the lane-changing behaviors of autonomous and human-driven vehicles, provide accurate data support for lane-changing analysis under mixed traffic flow, and provide technical guarantee for subsequent traffic behavior research.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and particularly to a method for classifying lane-changing trajectories based on UAV aerial video. Background Art

[0002] Autonomous vehicles have shown great potential in improving road safety and traffic efficiency with their precise control and efficient driving behaviors. However, in the actual road environment, autonomous vehicles and human-driven vehicles need to operate together, and this mixed traffic mode poses new requirements for the acquisition and processing of road traffic flow information.

[0003] Currently, when analyzing vehicle lane-changing behaviors based on UAV aerial video data, it is generally assumed that all the captured vehicles are human-driven. However, with the increase in autonomous vehicles, the scenarios where autonomous vehicles and human-driven vehicles run in parallel are more extensive, resulting in the gradual invalidation of this assumption. The lane-changing trajectories of human-driven vehicles are limited by the decision-making methods of drivers and are prone to randomness and uncertainty. While autonomous vehicles calculate through algorithms and sense the environment with sensors, and their lane-changing behaviors tend to be more regular and predictable. The different behavioral decision-making modes of the two generate differentiated data sets, which seriously affect the results of lane-changing trajectory analysis. Therefore, how to distinguish the lane-changing trajectories of these two different types of vehicles has become a technical problem to be solved urgently. Summary of the Invention

[0004] The present invention proposes a method for classifying lane-changing trajectories based on UAV aerial video, which can effectively distinguish the lane-changing behaviors of autonomous and human-driven vehicles, provide accurate data support for lane-changing analysis under mixed traffic flow, and provide technical guarantee for subsequent traffic behavior research.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention proposes a method for classifying lane-changing trajectories based on UAV aerial video, including the following steps:

[0007] S1. Obtain the rectangular box of the vehicle position and the coordinates of 4 BoundingBox points in a single-frame image of an autonomous vehicle based on the neural network YOLOv5;

[0008] S2. Obtain the trajectory coordinates of each BoundingBox point and the historical lane-changing trajectory map of the vehicle based on the DeepSort algorithm.

[0009] S3. Divide the historical lane-changing trajectory segments of the autonomous vehicle to obtain the first sub-segments, and establish a data set for each first sub-segment. The data set of the first sub-segment includes time tags, vehicle IDs, and the trajectory coordinates of each BoundingBox point of the vehicle.

[0010] S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub - segment. Calculate the expected lateral displacement, expected longitudinal displacement, expected lateral speed, and expected longitudinal speed of the autonomous vehicle for lane - changing based on the trajectory coordinates of each BoundingBox point of the vehicle. Then, extract the vehicle ID and time tag from the data set of each first sub - segment, and obtain the expected lane - changing duration of the vehicle through the time tag. Finally, establish the standard data set for each first sub - segment.

[0011] S5. Based on the vehicle lane - changing process hybrid search algorithm, divide the vehicle historical lane - changing trajectory map into 3 lane - changing stages, namely the starting turning stage, the straightening - back stage, and the adjustment stage. Divide each standard data set according to the lane - changing stage to obtain sub - standard data sets.

[0012] S6. Obtain the vehicle lane - changing standard two - dimensional image based on the sub - standard data sets.

[0013] S7. Obtain the running data of the vehicle to be tested, execute S1 to S6, obtain the two - dimensional image of the lane - changing of the vehicle to be tested, and judge the lane - changing trajectory type of the vehicle to be tested.

[0014] Preferably, in S1, the specific method for obtaining the coordinates of 4 BoundingBox points of the vehicle is: Split the lane - changing video of the autonomous vehicle at a fixed frame rate to obtain single - frame images.

[0015] Preferably, S2 specifically includes the following sub - steps:

[0016] S201. Match the rectangular boxes in the front and rear frames through the DeepSort algorithm, and assign a unique vehicle ID to each detected bounding box to obtain the trajectory coordinates of each BoundingBox point of each vehicle.

[0017] S202. Use Matplotlib to connect the trajectory coordinates of each BoundingBox point with line segments, display the trajectory paths of the four corner points of the vehicle in the two - dimensional plane, and generate the vehicle historical lane - changing trajectory map.

[0018] Preferably, in S3, the method for dividing the historical lane - changing trajectory segments of the autonomous vehicle is: Calculate the longitudinal speed at the starting moment of the historical lane - changing of each autonomous vehicle, divide the longitudinal speed at the starting moment of the vehicle historical lane - changing, the division interval is 20 km / h to 120 km / h, and the division step size is 10 km / h. Take the historical lane - changing trajectory segments of the vehicles corresponding to each category of longitudinal speed as a first sub - segment.

[0019] Preferably, S4 includes the following sub - steps:

[0020] S401. Extract the time tag, vehicle ID, and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub-fragment;

[0021] S402. Smooth the trajectory coordinates of each BoundingBox point using the Savitzky-Golay filtering method, and perform differencing on the smoothed trajectory coordinates to calculate the smoothed displacement Velocity Acceleration

[0022] Displacement Velocity Acceleration The calculation formulas are as follows:

[0023]

[0024]

[0025] In the formula, where is the lateral coordinate position and longitudinal coordinate position of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, is the lateral velocity and longitudinal velocity of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, is the lateral acceleration and longitudinal acceleration of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, and ΔT′ is the time interval of each frame when the data set is recorded;

[0026] Use statistical expectation to perform statistical analysis on the lane-changing lateral displacement, lane-changing longitudinal displacement, lane-changing lateral velocity, and lane-changing longitudinal velocity of each BoundingBox point in the first sub-fragment to obtain the expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral velocity, and expected lane-changing longitudinal velocity of each BoundingBox point:

[0027]

[0028] S403. Use the maximum likelihood estimation of the one-dimensional Gaussian function to perform statistical analysis on the lane-changing duration in each first sub-fragment to obtain the expected lane-changing duration corresponding to the first sub-fragment,

[0029] where the maximum likelihood estimation of the one-dimensional Gaussian function is specifically:

[0030]

[0031] where T is the lane-changing duration, μ T is the expected lane-changing time of this type of lane-changing trajectory segment, is the variance of the time elapsed in the lane-changing trajectory segment.

[0032] S404. Establish a standard data set for the first sub - segment. The standard data set is a one - dimensional time - series data set corresponding to the vehicle's expected lateral displacement during lane change, expected longitudinal displacement during lane change, expected lateral speed during lane change, expected longitudinal speed during lane change, and expected lane - change duration.

[0033] Preferably, S5 specifically includes:

[0034] S501. Determine the three stages of the vehicle's historical lane - change trajectory through threshold conditions and the trends of speed and acceleration changes;

[0035] S502. Mark the vehicle's historical lane - change trajectory diagram according to the determination results of the three lane - change stages, divide the vehicle's historical lane - change trajectory diagram into a starting - turning stage, a straightening - back stage, and an adjustment stage, calculate the duration of each lane - change stage, and take the average value for correction;

[0036] S503. Divide each standard data set according to the lane - change stage to obtain sub - standard data sets by comparing the duration of each lane - change stage.

[0037] Preferably, S6 specifically includes:

[0038] S601. Perform z - standardization on each sub - standard data set \(T=(t_1,t_2,\cdots,t_i),i\in[1,L]\) to obtain its standard normal distribution \(U=(u_1,u_2,\cdots,u_i)\). The calculation method of the \(i\) - th element \(u_i\) in \(U\) is as follows: L ), where \(t_1\) is the value of the time - series data at time \(i\), \(L\) is the length of the time - series data, L \(\mu\) is the mean of the time - series data \(T\), and \(\sigma\) is the variance of the time - series data; i The calculation method is as follows:

[0039]

[0040] \(t_1\) is the value of the time - series data at time \(i\), \(L\) is the length of the time - series data,

[0041] \(\mu\) is the mean of the time - series data \(T\), and \(\sigma\) is the variance of the time - series data;

[0042] S602. Use the piecewise aggregate approximation method to reduce the dimension of \(U\). Select a reduction factor \(r\) to reduce the dimension of \(U\) from \(L\) to \(m\) to obtain the time - series data \(X=(x_1,x_2,\cdots,x_i)\): m ):

[0043]

[0044] where \(m\) is the length of \(X\), and \(x_i\) i is the element corresponding to the \(i\) - th timestamp \(x_i\) i ;

[0045] S603. Sequentially take the element \(x_i\) corresponding to the \(i\) - th timestamp on the time - series data \(X\) iConvert it as a relative position reference point to obtain matrix M, where the element value of the i-th row is the element value of X minus x i It is calculated that the obtained matrix M is as follows:

[0046]

[0047] S604. Convert matrix M into a vehicle standard two-dimensional image F through min-max normalization. The calculation method of F is as follows:

[0048]

[0049] Preferably, in S7, an improved convolutional neural network classification model is used to judge the lane-changing trajectory type of the vehicle to be tested. The specific method is: use the standard two-dimensional image F and the two-dimensional image of the vehicle to be tested as the inputs of the improved convolutional neural network classification model respectively; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is greater than 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by autonomous driving; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is less than or equal to 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by manual driving.

[0050] Preferably, in S501, the specific method for determining the three stages of the vehicle's historical lane-changing trajectory is: start searching from the first frame. When is satisfied, it is recorded as the starting point of the starting turning stage, and continuously detect When drops to 0.5 m / s 2 , and is satisfied, it is recorded as the starting point of the returning-to-normal stage and the end point of the turning stage. When and the lateral displacement change is less than 0.5 m, it is recorded as the starting point of the adjustment stage and the end point of the returning-to-normal stage. Search until the last frame ends and is recorded as the end point of the adjustment stage.

[0051] Preferably, the improved convolutional neural network classification model includes a convolutional layer. The vehicle standard two-dimensional image F is the input of the convolutional layer. The output of the convolutional layer is the input of the batch normalization layer. The output of the batch normalization layer is the input of the activation layer. The output of the activation layer is the input of the first fully connected layer. The output of the first fully connected layer is the input of the second fully connected layer. The output of the second fully connected layer is the input of the output layer. The output of the output layer is the output of the convolutional neural network classification model.

[0052] Compared with the prior art, the beneficial effects of the present invention are:

[0053] The present invention combines an aerial video captured by a drone with a neural network algorithm to obtain the trajectory information of four BoundingBox points of a vehicle, making the lane-changing trajectory data of the autonomous vehicle extracted more comprehensive. A hybrid search algorithm is used to divide the lane-changing stage, optimizing the accuracy of the standard dataset for autonomous driving lane changes. The trajectory classification uses a relative position matrix representation method to fuse a convolutional neural network classification model, converting time-series data into a two-dimensional image for classification determination, improving the speed and accuracy of trajectory classification. Therefore, the present invention effectively improves the efficiency and accuracy of lane-changing trajectory classification for autonomous driving and manual driving, and fills the gap in the current lane-changing trajectory classification technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0055] Figure 1 It is a flowchart of the lane-changing trajectory classification method for the aerial video of the drone of the present invention;

[0056] Figure 2 It is a schematic diagram of the vehicle lane-changing trajectory coordinate extraction process of the present invention;

[0057] Figure 3 It is the setting of the tracking points of the DeepSort algorithm in the present invention;

[0058] Figure 4 It is a schematic diagram of the lane-changing trajectory classification process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following will describe the implementation of the present invention in detail with reference to the embodiments. However, those skilled in the art will understand that the following embodiments are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention.

[0060] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will give a detailed description of the specific implementation of the present invention with reference to the drawings.

[0061] Specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0062] Refer to Figure 1, the present invention proposes a lane-changing trajectory classification method based on UAV aerial video, which is characterized by the following steps:

[0063] S1. Use a UAV to take videos of the historical lane-changing process of an autonomous vehicle, and split the original vehicle lane-changing video data into single-frame images at a fixed frame rate. In this embodiment, the frame rate is selected as 60 FPS. By pre-training the YOLOv5 model and loading a suitable weight file, the YOLOv5 model can identify the vehicles in the images, output a rectangular box (bounding box) at the position of each vehicle, and obtain the four corner coordinates (upper left, upper right, lower left, lower right) of the rectangular box. The four corner coordinates serve as key data for subsequent processing;

[0064] The YOLOv5 model outputs the bounding box coordinates of the targets in each frame of the image. Specifically, each bounding box contains the position information of the vehicle, that is, the spatial coordinates of the vehicle in the image.

[0065] S2. Use the DeepSORT algorithm to track the vehicles identified by the YOLOv5 model. The DeepSORT algorithm initializes the target vehicle ID according to the output result of the YOLOv5 model and assigns a unique ID to each output bounding box.

[0066] The DeepSORT algorithm uses the Hungarian algorithm to perform target vehicle matching in each frame. By matching the rectangular boxes in the previous and subsequent frames, the trajectory of each target vehicle is determined. Finally, the rectangular boxes in the single-frame images of each vehicle are concatenated according to the lane-changing trajectory, and continuous vehicle trajectory data is output. These trajectory data contain the trajectory coordinates of 4 BoundingBox points of each vehicle and the vehicle ID. Among them, the DeepSORT algorithm is as follows:

[0067] Initialization: τ ← φ, detection set D ← φ;

[0068] Use YOLOv5 to perform detection in each frame K, and obtain the detected candidate boxes C det , use Kalman to collect the trajectory candidate boxes C in τ trk ;

[0069] C ← C det ∪C trk ;

[0070] C, S ← NMS(C, S, τ max );

[0071] Filter S < τ max ;

[0072] Extract the trajectory candidate box C trk Endpoint feature: F trk ; Extract the detected candidate box C det Endpoint feature: F det ;

[0073] Merge the trajectory candidate box C trk and the detected candidate box C det to form C det : C det ←C det ∪C trk ;

[0074] Input the τ value and merge it with the trajectory candidate box C trk : τ ← τ ∪ C trk ;

[0075] Output: the target trajectory τ.

[0076] After the target vehicle tracking step is completed, use Matplotlib to connect the coordinate data of the 4 BoundingBox points of each vehicle in consecutive multiple frames of images to form a continuous curve. Each trajectory represents the movement path of a vehicle from the start of lane change to the end of lane change. The horizontal axis represents the horizontal coordinate (X value) in the image, and the vertical axis represents the vertical coordinate (Y value) in the image. Each trajectory represents the complete path of a corner point from the start to the end of lane change. In this way, the movement trajectory of the vehicle in the video image during lane change can be clearly shown, providing a basis for subsequent trajectory classification.

[0077] S3. Calculate the longitudinal speed of each autonomous vehicle at the starting moment of historical lane change, divide the longitudinal speed of the vehicle at the starting moment of historical lane change, the division interval is 20 km / h to 120 km / h, the division step size is 10 km / h, and it is divided into 10 categories (for example: i1 = 20 - 30 km / h, i2 = 30 - 40 km / h..., i 10 = 110 - 120 km / h), and use the historical lane change trajectory segment of the vehicle corresponding to each category of longitudinal speed as a first sub-segment to analyze the lane change behavior at different speeds. Next, establish a data set for each first sub-segment. The data set of the first sub-segment includes time tags, vehicle IDs, and the trajectory coordinates of each BoundingBox point of the vehicle.

[0078] S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub - segment. Smooth the position coordinate data of each BoundingBox point of each vehicle. The smoothing method uses the Savitzky - Golay filtering method, and perform differencing on the smoothed coordinate data to calculate the smoothed speed. Acceleration Displacement Speed Acceleration The calculation formulas are as follows:

[0079]

[0080]

[0081] In the formula, where is the lateral coordinate position and longitudinal coordinate position of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, is the lateral speed and longitudinal speed of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, is the lateral acceleration and longitudinal acceleration of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, and ΔT′ is the time interval of each frame when the data set is recorded;

[0082] Use statistical expectation to perform statistical analysis on the lane - changing lateral displacement, lane - changing longitudinal displacement, lane - changing lateral speed, and lane - changing longitudinal speed of each BoundingBox point in the first sub - segment to obtain the expected lane - changing lateral displacement μ x and the expected lane - changing longitudinal displacement μ y and the expected lane - changing lateral speed the expected lane - changing longitudinal speed

[0083]

[0084] Use the maximum likelihood estimation of the one - dimensional Gaussian function to perform statistical analysis on the lane - changing duration in each first sub - segment to obtain the expected lane - changing duration corresponding to the first sub - segment. Among them, the maximum likelihood estimation of the one - dimensional Gaussian function is specifically:

[0085]

[0086] Among them, T is the lane - changing duration, μ T is the expected lane - changing time of this type of lane - changing trajectory segment, is the variance of the time elapsed in the lane - changing trajectory segment.

[0087] Establish a standard data set for each first sub - segment. The standard data set is a one - dimensional time - series data set corresponding to the expected lateral displacement, expected longitudinal displacement, expected lateral speed, expected longitudinal speed, and expected lane - change duration of the vehicle during lane - change.

[0088] S5. In each stage, parameters such as the lateral displacement and acceleration of the vehicle will be different. The start and end points of each stage can be accurately divided through an algorithm. Starting from the first frame of the lane - change video of the autonomous vehicle, when is satisfied, it is recorded as the starting point of the steering stage, and continuously detect When drops to 0.5 m / s 2 , and is satisfied, it is recorded as the starting point of the straightening stage and the end point of the steering stage. When and the change in lateral displacement is less than 0.5 m, it is recorded as the starting point of the adjustment stage and the end point of the straightening stage. Search until the last frame ends and is recorded as the end point of the adjustment stage.

[0089] Mark according to the determination results of the three lane - change stages on the vehicle's historical lane - change trajectory map. Divide the vehicle's historical lane - change trajectory map into a steering stage, a straightening stage, and an adjustment stage, calculate the duration of each lane - change stage, and take the average value of the duration for correction;

[0090] Compare with the duration of each lane - change stage, divide each standard data set according to the lane - change stage to obtain sub - standard data sets. According to the combination of the longitudinal speed at the start time of lane - change and the lane - change stage, 30 sub - standard data sets can be obtained. Each sub - standard data set has 16 one - dimensional time - series data sets, including the corresponding relationship between the displacements and speeds of the four endpoints of the vehicle and time. Based on this, an autonomous - driving lane - change standard parameter set is established As shown in Table 1:

[0091] Table 1 One - dimensional time - series data sets in the sub - standard data set

[0092]

[0093] S6. Obtain a standard two - dimensional image of vehicle lane - change based on each sub - standard data set. Each group of images expresses the patterns and features in 4 types of one - dimensional original time - series data, namely the change curves of lane - change lateral displacement over time, lane - change lateral and longitudinal displacements over time, lane - change lateral speed over time, and lane - change longitudinal speed over time. While being retained, it has a high intra - class similarity and a low inter - class similarity. The specific process of obtaining the standard two - dimensional image is as follows:

[0094] S601. Normalize each sub - standard data set \(T=(t_1,t_2,\cdots,t L ), i\in[1,L]\) by z - normalization to obtain its standard normal distribution \(U=(u_1,u_2,\cdots,uL ), the \(i\)-th element \(u\) in \(U\) i is calculated as follows:

[0095]

[0096] \(t_1\) is the value of the time-series data at time \(i\), \(L\) is the length of the time-series data,

[0097] \(\mu\) is the mean of the time-series data \(T\), and \(\sigma\) is the variance of the time-series data;

[0098] S602. Use the piecewise aggregate approximation method to reduce the dimension of \(U\). Select a reduction factor \(r\) to reduce the dimension of \(U\) from \(L\) to \(m\), obtaining the time-series data \(X=(x_1, x_2, \ldots, x\) m ):

[0099]

[0100] where \(m\) is the length of \(X\), and \(x\) i is the element corresponding to the \(i\)-th timestamp \(x\) i ;

[0101] S603. Successively use the element \(x\) corresponding to the \(i\)-th timestamp on the time-series data \(X\) i as the relative position reference point for transformation to obtain the matrix \(M\), where the element value of the \(i\)-th row is obtained by subtracting \(x\) from the element value of \(X\) i The obtained matrix \(M\) is as follows:

[0102]

[0103] Time-series data Any two timestamps on the corresponding elements are related by calculating their relative positions, retaining the internal correlation of the time-series data and providing redundant features of the time-series data to improve the generalization ability of the subsequent classification model.

[0104] S604. Transform the matrix \(M\) into the vehicle standard two-dimensional image \(F\) by min-max normalization. The calculation method of \(F\) is as follows:

[0105]

[0106] S7. Obtain the running data of the vehicle to be tested, execute S1 to S6 to obtain the two-dimensional image P of the lane change of the vehicle to be tested, and use the standard two-dimensional image F within the same longitudinal speed interval at the historical lane change start time and the two-dimensional image P of the vehicle to be tested as the input of the improved convolutional neural network classification model; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image P of the vehicle to be tested and the standard two-dimensional image F is greater than 0.75, it is determined that the lane change trajectory of the vehicle to be tested is generated by autonomous driving; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image P of the vehicle to be tested and the standard two-dimensional image F is less than or equal to 0.75, it is determined that the lane change trajectory of the vehicle to be tested is generated by manual driving.

[0107] The improved convolutional neural network includes six layers:

[0108] The first layer: convolutional layer. By convolving the output of the previous layer with a set of convolutional kernels, a set of features are extracted at different positions of the original image, thereby obtaining a feature extraction representation called a feature map.

[0109] The second layer: batch normalization layer. By performing batch normalization on the values of different feature maps in the previous layer, the previous non-fixed data distribution is converted into a fixed data distribution, so that the input distribution during the network training process remains consistent.

[0110] The third layer: activation layer. The scaled exponential linear unit activation function is used to perform a non-linear mapping on the output of the previous layer, introducing non-linearity into the entire network and improving the fitting ability of the model.

[0111] The fourth and fifth layers: two fully connected layers. Map the high-level representation of the original image features obtained in the previous feature extraction part to the categories of the image.

[0112] The sixth layer: output layer. Image classification is achieved through the Softmax function. Specifically: map the similarity of the two-dimensional image P to the probability space: assume the similarity value is s = sim(f1, f2), and use Softmax to convert the similarity and its complementary value (1 - s) into a binary classification probability distribution:

[0113]

[0114] Set the probability threshold to 75%, and the classification determination:

[0115] Determination result

[0116] Exemplary:

[0117] A video of a vehicle's lane-changing trajectory is collected, and the longitudinal speed at the initial stage of lane change is 25 km / h. The lane-changing parameters after collection and processing are represented by a relative position matrix and converted into a two-dimensional image. Then, a convolutional neural network classification model is used to match the image to be detected with the standard automatic driving lane-changing sample image at speed i1. The similarity probability of the parameters is 81%, and the output result is "auto". Therefore, it can be determined that the trajectory is a lane-changing behavior completed by automatic driving. If the similarity probability is less than 75% and does not meet the set probability threshold, the output result is "manual", and finally it is determined that the lane-changing trajectory is completed by manual driving.

[0118] Although the present invention has been described in detail with general descriptions and specific embodiments in this specification, based on the present invention, some modifications or improvements can be made, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the present invention fall within the scope of protection of the present invention.

Claims

1. A lane-changing trajectory classification method based on UAV aerial video, characterized in that, It includes the following steps: S1. Based on the neural network YOLOv5, obtain the rectangular box of the vehicle's location in a single-frame image of the autonomous vehicle and the coordinates of 4 BoundingBox points; S2. Based on the DeepSort algorithm, obtain the trajectory coordinates of each BoundingBox point and the vehicle's historical lane-changing trajectory map; S3. Divide the historical lane-changing trajectory segment of the autonomous vehicle to obtain the first sub-segment, and establish a data set for each first sub-segment. The data set of the first sub-segment includes time tags, vehicle IDs, and the trajectory coordinates of each BoundingBox point of the vehicle; S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub-segment. Calculate the expected lateral displacement, expected longitudinal displacement, expected lateral speed, and expected longitudinal speed of the autonomous vehicle's lane change through the trajectory coordinates of each BoundingBox point of the vehicle. Then extract the vehicle ID and time tag from the data set of each first sub-segment, and obtain the expected lane-changing duration of the vehicle through the time tag. Finally, establish a standard data set for each first sub-segment; S5. Based on the vehicle lane-changing process hybrid search algorithm, divide the vehicle's historical lane-changing trajectory map into 3 lane-changing stages, namely the starting steering stage, the straightening stage, and the adjustment stage. Divide each standard data set according to the lane-changing stage to obtain sub-standard data sets; S6. Obtain the vehicle lane-changing standard two-dimensional image based on the sub-standard data set; S7. Obtain the running data of the vehicle to be tested, execute S1 to S6, obtain the two-dimensional image of the lane change of the vehicle to be tested, and judge the lane-changing trajectory type of the vehicle to be tested.

2. The lane-changing trajectory classification method based on UAV aerial video according to claim 1, wherein, In S1, the specific method for obtaining the coordinates of 4 BoundingBox points of the vehicle is: divide the lane-changing video of the autonomous vehicle at a fixed frame rate to obtain single-frame images.

3. A lane-changing trajectory classification method based on UAV aerial video according to claim 1, characterized in that, S2 specifically includes the following sub-steps: S201. Match the rectangular boxes in the front and rear frames through the DeepSort algorithm, and assign a unique vehicle ID to each detected bounding box to obtain the trajectory coordinates of each BoundingBox point of each vehicle; S202. Use Matplotlib to connect the trajectory coordinates of each BoundingBox point with line segments, display the trajectory paths of the four corner points of the vehicle in the two-dimensional plane, and generate the vehicle's historical lane-changing trajectory map.

4. A lane-changing trajectory classification method based on UAV aerial video according to claim 1, characterized in that In S3, the method for dividing the historical lane-changing trajectory segment of the autonomous vehicle is: calculate the longitudinal speed at the starting moment of the historical lane change of each autonomous vehicle, divide the longitudinal speed at the starting moment of the historical lane change of the vehicle, the division interval is 20 km / h to 120 km / h, and the division step size is 10 km / h. Take the historical lane-changing trajectory segment of the vehicle corresponding to each category of longitudinal speed as a first sub-segment.

5. A lane-changing trajectory classification method based on UAV aerial video according to claim 1, characterized in that S4 includes the following sub-steps: S401. Extract the time tag, vehicle ID, and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub-segment; S402. Smooth the trajectory coordinates of each BoundingBox point using the Savitzky-Golay filtering method, perform differencing on the smoothed trajectory coordinates, and calculate the smoothed displacement Velocity Acceleration Displacement Velocity Acceleration The calculation formulas are as follows: In the formula, where is the lateral coordinate position and longitudinal coordinate position of the endpoint numbered n of a certain autonomous vehicle at the i-th frame moment, is the lateral speed and longitudinal speed of the endpoint numbered n of a certain autonomous vehicle at the i-th frame moment, is the lateral acceleration and longitudinal acceleration of the endpoint numbered n of a certain autonomous vehicle at the i-th frame moment, and ΔT′ is the time interval per frame when the data set is recorded; Statistical analysis is performed on the lane-changing lateral displacement, lane-changing longitudinal displacement, lane-changing lateral velocity, and lane-changing longitudinal velocity of each BoundingBox point in the first sub-segment using statistical expectation to obtain the expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral velocity, and expected lane-changing longitudinal velocity of each BoundingBox point: S403. Statistical analysis is performed on the lane-changing duration in each first sub-segment using the maximum likelihood estimation of the one-dimensional Gaussian function to obtain the expected lane-changing duration corresponding to the first sub-segment. Among them, the maximum likelihood estimation of the one-dimensional Gaussian function is specifically as follows: Among them, T is the lane-changing duration, and μ T is the expected lane-changing time of this type of lane-changing trajectory segment, and is the variance of the time elapsed for the lane-changing trajectory segment; S404. A standard data set for the first sub-segment is established. The standard data set is a one-dimensional time series data set corresponding to the expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral velocity, expected lane-changing longitudinal velocity, and expected lane-changing duration of the vehicle.

6. The lane-changing trajectory classification method based on UAV aerial video according to claim 1, wherein, S5 specifically includes: S501. Determine the three stages of the vehicle's historical lane-changing trajectory through threshold conditions and the changing trends of speed and acceleration. S502. Mark the vehicle's historical lane-changing trajectory diagram according to the determination results of the three lane-changing stages, divide the vehicle's historical lane-changing trajectory diagram into a starting turning stage, a returning-to-straight stage, and an adjustment stage, calculate the duration of each lane-changing stage, and take the average value for correction. S503. Compare the duration of each lane-changing stage and divide each standard data set according to the lane-changing stage to obtain a sub-standard data set.

7. A lane-changing trajectory classification method based on UAV aerial video according to claim 1, characterized in that S6 specifically includes: S601. Standardize each sub-standard data set \(T=(t_1,t_2,\cdots,t L ), i\in[1,L]\) by z-score to obtain its standard normal distribution \(U=(u_1,u_2,\cdots,u L ). The calculation method of the \(i\)-th element \(u i \) in \(U\) is as follows: t1 is the value of the time series data at time i, and L is the length of the time series data. μ is the mean of the time series data T, and σ is the variance of the time series data. S602. Use the piecewise aggregate approximation method to reduce the dimension of U. Select a reduction factor r to reduce the dimension of U from L to m, obtaining the time series data X = (x1, x2, …, x m ): where m is the length of X, and x i is the element corresponding to the i-th timestamp x i ; S603, sequentially add the element x corresponding to the i-th timestamp on the time series data X i As a relative position reference point, the transformation is performed to obtain the matrix M, where the element value of the i-th row is the element value of X minus x i The calculation shows that the matrix M is as follows: S604. Convert the matrix M into the vehicle standard two-dimensional image F through min-max normalization. The calculation method of F is as follows: 。 8. A lane-changing trajectory classification method based on UAV aerial video according to claim 1, characterized in that In S7, an improved convolutional neural network classification model is used to judge the lane-changing trajectory type of the vehicle to be tested. The specific method is as follows: The standard two-dimensional image F and the two-dimensional image of the vehicle to be tested are respectively used as the inputs of the improved convolutional neural network classification model. When the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is greater than 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by autonomous driving. When the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is less than or equal to 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by manual driving.

9. A lane-changing trajectory classification method based on UAV aerial video according to claim 6, characterized in that, In S501, the specific method for determining the three stages of the vehicle's historical lane-changing trajectory is as follows: Start searching from the first frame. When the following conditions are met , it is recorded as the starting point of the steering phase. Continuously detect When drops to 0.5 m / s 2 , and , it is recorded as the starting point of the straightening phase and the end point of the steering phase. When and the lateral displacement change is less than 0.5 m, it is recorded as the starting point of the adjustment phase and the end point of the straightening phase. Search until the last frame is reached and recorded as the end point of the adjustment phase.

10. A lane-changing trajectory classification method based on UAV aerial video according to claim 8, characterized in that, The improved convolutional neural network classification model includes a convolutional layer. The vehicle standard two-dimensional image F is the input of the convolutional layer. The output of the convolutional layer is the input of the batch normalization layer. The output of the batch normalization layer is the input of the activation layer. The output of the activation layer is the input of the first fully connected layer. The output of the first fully connected layer is the input of the second fully connected layer. The output of the second fully connected layer is the input of the output layer. The output of the output layer is the output of the convolutional neural network classification model.

Images