Lane changing trajectory classification method based on aerial video of unmanned aerial vehicle

Abstract

The invention discloses a lane changing trajectory classification method based on an unmanned aerial vehicle aerial video. The method comprises the following steps: obtaining a rectangular frame of the position of a vehicle in a single-frame image of an automatic driving vehicle and coordinates of four BoundingBox points; obtaining a track coordinate of each BoundingBox point and a historical lane changing track diagram of the vehicle; establishing a data set of each first sub-fragment; dividing the historical lane changing track fragment of the automatic driving vehicle to obtain a first sub-fragment; establishing a standard data set of each first sub-fragment, and dividing each standard data set according to a lane changing stage to obtain sub-standard data sets; obtaining a vehicle lane changing standard two-dimensional image based on the sub-standard data set; obtaining running data of a to-be-detected vehicle, obtaining a lane changing two-dimensional image of the to-be-detected vehicle, and judging a lane changing track type of the to-be-detected vehicle; according to the invention, lane changing behaviors of automatic driving and manual driving vehicles can be effectively distinguished, accurate data support is provided for lane changing analysis under mixed traffic flow, and technical support is provided for subsequent traffic behavior research.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and particularly relates to a method for classifying lane-changing trajectories based on UAV aerial video. Background Art

[0002] Autonomous vehicles have shown great potential in improving road safety and traffic efficiency with their precise control and efficient driving behavior. However, in the actual road environment, autonomous vehicles and human-driven vehicles need to run together, and this mixed traffic mode poses new requirements for the acquisition and processing of road traffic flow information.

[0003] Currently, when analyzing vehicle lane-changing behavior based on UAV aerial video data, it is generally assumed that all photographed vehicles are human-driven. However, with the increase in autonomous vehicles, the scenario of autonomous vehicles and human-driven vehicles running in parallel is more extensive, resulting in the gradual invalidation of this assumption. The lane-changing trajectory of human-driven vehicles is limited by the driver's decision-making method and is prone to randomness and uncertainty. While autonomous vehicles perceive the environment through algorithm calculation and sensors, and their lane-changing behavior is more regular and predictable. The different behavioral decision-making modes of the two produce differentiated data sets, which seriously affect the results of lane-changing trajectory analysis. Therefore, how to distinguish the lane-changing trajectories of these two different types of vehicles has become an urgent technical problem to be solved. Summary of the Invention

[0004] The present invention proposes a method for classifying lane-changing trajectories based on UAV aerial video, which can effectively distinguish the lane-changing behaviors of autonomous and human-driven vehicles, provide accurate data support for lane-changing analysis under mixed traffic flow, and provide technical guarantee for subsequent traffic behavior research.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention proposes a method for classifying lane-changing trajectories based on UAV aerial video, including the following steps:

[0007] S1. Obtain the rectangular frame of the vehicle position and the coordinates of 4 BoundingBox points in a single-frame image of an autonomous vehicle based on the neural network YOLOv5;

[0008] S2. Obtain the trajectory coordinates of each BoundingBox point and the vehicle historical lane-changing trajectory map based on the DeepSort algorithm.

[0009] S3. Divide the historical lane-changing trajectory segment of the autonomous vehicle to obtain a first sub-segment, and establish a data set for each first sub-segment. The data set of the first sub-segment includes time tags, vehicle IDs, and the trajectory coordinates of each BoundingBox point of the vehicle.

[0010] S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub - segment. Calculate the expected lateral displacement, expected longitudinal displacement, expected lateral speed, and expected longitudinal speed of the autonomous vehicle for lane - changing based on the trajectory coordinates of each BoundingBox point of the vehicle. Then, extract the vehicle ID and time tag from the data set of each first sub - segment, and obtain the expected lane - changing duration of the vehicle through the time tag. Finally, establish the standard data set for each first sub - segment.

[0011] S5. Based on the hybrid search algorithm for vehicle lane - changing process, divide the vehicle historical lane - changing trajectory map into 3 lane - changing stages, namely the starting - turning stage, the straightening - back stage, and the adjustment stage. Divide each standard data set according to the lane - changing stage to obtain sub - standard data sets.

[0012] S6. Obtain the standard two - dimensional image of vehicle lane - changing based on the sub - standard data sets.

[0013] S7. Obtain the running data of the vehicle to be tested, execute S1 to S6, obtain the two - dimensional image of the lane - changing of the vehicle to be tested, and judge the type of the lane - changing trajectory of the vehicle to be tested.

[0014] Preferably, in S1, the specific method for obtaining the coordinates of 4 BoundingBox points of the vehicle is: split the lane - changing video of the autonomous vehicle at a fixed frame rate to obtain single - frame images.

[0015] Preferably, S2 specifically includes the following sub - steps:

[0016] S201. Match the rectangular boxes in the front and rear frames through the DeepSort algorithm, and assign a unique vehicle ID to each detected bounding box to obtain the trajectory coordinates of each BoundingBox point of each vehicle.

[0017] S202. Use Matplotlib to connect the trajectory coordinates of each BoundingBox point with line segments, display the trajectory paths of the four corner points of the vehicle in the two - dimensional plane, and generate the vehicle historical lane - changing trajectory map.

[0018] Preferably, in S3, the method for dividing the historical lane - changing trajectory segments of the autonomous vehicle is: calculate the longitudinal speed at the starting moment of the historical lane - changing of each autonomous vehicle, divide the longitudinal speed at the starting moment of the vehicle historical lane - changing, the division interval is 20 km / h to 120 km / h, and the division step size is 10 km / h. Take the historical lane - changing trajectory segments of the vehicles corresponding to each class of longitudinal speed as a first sub - segment.

[0019] Preferably, S4 includes the following sub - steps:

[0020] S401. Extract the time tag, vehicle ID, and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub-fragment;

[0021] S402. Smooth the trajectory coordinates of each BoundingBox point using the Savitzky-Golay filtering method, and perform differencing on the smoothed trajectory coordinates to calculate the smoothed displacement Velocity Acceleration

[0022] Displacement Velocity Acceleration The calculation formulas are as follows:

[0023]

[0024]

[0025] In the formula, where is the lateral coordinate position and longitudinal coordinate position of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, is the lateral velocity and longitudinal velocity of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, is the lateral acceleration and longitudinal acceleration of the endpoint of an autonomous vehicle numbered n at the i-th frame moment, and ΔT′ is the time interval of each frame when the data set is recorded;

[0026] Use statistical expectation to perform statistical analysis on the lane-changing lateral displacement, lane-changing longitudinal displacement, lane-changing lateral velocity, and lane-changing longitudinal velocity of each BoundingBox point in the first sub-fragment to obtain the expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral velocity, and expected lane-changing longitudinal velocity of each BoundingBox point:

[0027]

[0028] S403. Use the maximum likelihood estimation of the one-dimensional Gaussian function to perform statistical analysis on the lane-changing duration in each first sub-fragment to obtain the expected lane-changing duration corresponding to the first sub-fragment,

[0029] where the maximum likelihood estimation of the one-dimensional Gaussian function is specifically:

[0030]

[0031] where T is the lane-changing duration, μ T is the expected lane-changing time of this type of lane-changing trajectory segment, is the variance of the time elapsed by the lane-changing trajectory segment.

[0032] S404. Establish a standard data set for the first sub - segment. The standard data set is a one - dimensional time - series data set corresponding to the expected lateral displacement, expected longitudinal displacement, expected lateral speed, expected longitudinal speed, and expected lane - change duration of the vehicle during lane - change.

[0033] Preferably, S5 specifically includes:

[0034] S501. Determine the three stages of the vehicle's historical lane - change trajectory through threshold conditions and the trends of speed and acceleration;

[0035] S502. Mark the vehicle's historical lane - change trajectory diagram according to the determination results of the three lane - change stages. Divide the vehicle's historical lane - change trajectory diagram into a starting - turning stage, a straightening - back stage, and an adjustment stage, calculate the duration of each lane - change stage, and take the average value for correction;

[0036] S503. Compare the duration of each lane - change stage and divide each standard data set according to the lane - change stage to obtain sub - standard data sets.

[0037] Preferably, S6 specifically includes:

[0038] S601. Perform z - normalization on each sub - standard data set \(T=(t_{1},t_{2},\cdots,t_{L})\), \(i\in[1,L]\) to obtain its standard normal distribution \(U=(u_{1},u_{2},\cdots,u_{L})\). The calculation method of the \(i\) - th element \(u_{i}\) in \(U\) is as follows: 1 ,t 2 ,…,t L ), \(t_{i}\) is the value of the time - series data at time \(i\), \(L\) is the length of the time - series data, 1 ,u 2 ,…,u L ), where \(\mu\) is the mean of the time - series data \(T\) and \(\sigma\) is the variance of the time - series data; i The calculation formula is as follows:

[0039]

[0040] t 1 is the value of the time - series data at time \(i\), \(L\) is the length of the time - series data,

[0041] \(\mu\) is the mean of the time - series data \(T\), \(\sigma\) is the variance of the time - series data;

[0042] S602. Use the piecewise aggregate approximation method to reduce the dimension of \(U\). Select a reduction factor \(r\) to reduce the dimension of \(U\) from \(L\) to \(m\) to obtain the time - series data \(X=(x_{1},x_{2},\cdots,x_{m})\): 1 ,x 2 ,…,x m ):

[0043]

[0044] where \(m\) is the length of \(X\), \(x_{j}\) iThe element x corresponding to the i-th timestamp i ;

[0045] S603. Sequentially convert the element x corresponding to the i-th timestamp on the time series data X i as a relative position reference point to obtain a matrix M, where the element value of the i-th row is the element value of X minus x i Calculated, the obtained matrix M is as follows:

[0046]

[0047] S604. Convert the matrix M into a vehicle standard two-dimensional image F through min-max normalization. The calculation method of F is as follows:

[0048]

[0049] Preferably, in S7, the type of the lane-changing trajectory of the vehicle to be tested is judged by an improved convolutional neural network classification model. The specific method is: using the standard two-dimensional image F and the two-dimensional image of the vehicle to be tested as the inputs of the improved convolutional neural network classification model respectively; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is greater than 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by autonomous driving; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is less than or equal to 0.75, it is judged that the lane-changing trajectory of the vehicle to be tested is generated by manual driving.

[0050] Preferably, in S501, the specific method for determining the three stages of the vehicle's historical lane-changing trajectory is: starting from the first frame for searching, when satisfying it is recorded as the starting point of the starting turning stage, and continuously detecting When drops to 0.5 m / s 2 , and it is recorded as the starting point of the returning-to-normal stage and the end point of the turning stage. When and the lateral displacement change is less than 0.5 m, it is recorded as the starting point of the adjustment stage and the end point of the returning-to-normal stage. Search until the last frame ends and is recorded as the end point of the adjustment stage.

[0051] Preferably, the improved convolutional neural network classification model includes a convolutional layer. The vehicle standard two-dimensional image F is the input of the convolutional layer. The output of the convolutional layer is the input of the batch normalization layer. The output of the batch normalization layer is the input of the activation layer. The output of the activation layer is the input of the first fully connected layer. The output of the first fully connected layer is the input of the second fully connected layer. The output of the second fully connected layer is the input of the output layer. The output of the output layer is the output of the convolutional neural network classification model.

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] Through the combination of UAV aerial video and neural network algorithm, the present invention obtains the trajectory information of four BoundingBox points of the vehicle, making the lane-changing trajectory data of the autonomous vehicle extracted more comprehensive; and uses a hybrid search algorithm to divide the lane-changing stage, optimizing the accuracy of the standard dataset for autonomous lane-changing; the trajectory classification uses a relative position matrix representation method to fuse the convolutional neural network classification model, converting the time-series data into a two-dimensional image for classification determination, improving the speed and accuracy of trajectory classification; therefore, the present invention effectively improves the efficiency and accuracy of lane-changing trajectory classification for autonomous driving and manual driving, and fills the gap in the current lane-changing trajectory classification technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the structures shown in these drawings.

[0055] Figure 1 It is a flowchart of the lane-changing trajectory classification method for the UAV aerial video of the present invention;

[0056] Figure 2 It is a schematic diagram of the vehicle lane-changing trajectory coordinate extraction process of the present invention;

[0057] Figure 3 It is the DeepSort algorithm tracking point setting of the present invention;

[0058] Figure 4 It is a schematic diagram of the lane-changing trajectory classification process of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] The following will describe the implementation scheme of the present invention in detail in combination with the embodiments. However, those skilled in the art will understand that the following embodiments are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention.

[0060] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific implementation manners of the present invention in detail with reference to the drawings.

[0061] Specific details are set forth in the following description in order to provide a thorough understanding of the invention. However, the invention can be practiced in many other ways different from those described herein and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited by the specific embodiments disclosed below.

[0062] Reference Figure 1 , the present invention proposes a lane - changing trajectory classification method based on UAV aerial video, which is characterized by including the following steps:

[0063] S1. Use a UAV to take videos of the historical lane - changing process of an autonomous vehicle, and divide the original video data of vehicle lane - changing into single - frame images at a fixed frame rate. In this embodiment, the frame rate is selected as 60FPS. By pre - training the YOLOv5 model and loading a suitable weight file, the YOLOv5 model can identify the vehicles in the images, output a rectangular box (bounding box) at the position of each vehicle, and obtain the four corner coordinates (upper - left corner, upper - right corner, lower - left corner, lower - right corner) of the rectangular box. These four corner coordinates are used as key data for subsequent processing;

[0064] The YOLOv5 model outputs the bounding box coordinates of the target in each frame of the image. Specifically, each bounding box contains the position information of the vehicle, that is, the spatial coordinates of the vehicle in the image.

[0065] S2. Use the DeepSORT algorithm to track the vehicles identified by the YOLOv5 model. The DeepSORT algorithm initializes the target vehicle ID according to the output result of the YOLOv5 model and assigns a unique ID to each output bounding box.

[0066] The DeepSORT algorithm uses the Hungarian algorithm to perform target vehicle matching in each frame. By matching the rectangular boxes in the front and back frames, the trajectory of each target vehicle is determined. Finally, the rectangular boxes in the single - frame images of each vehicle are concatenated according to the lane - changing trajectory, and continuous vehicle trajectory data is output. These trajectory data contain the trajectory coordinates of 4 BoundingBox points of each vehicle and the vehicle ID. Among them, the DeepSORT algorithm is as follows:

[0067] Initialization: τ←φ, detection set D←φ;

[0068] Use YOLOv5 to perform detection in each frame K, and obtain the detected candidate boxes C det , use Kalman to collect the trajectory candidate boxes C in τ trk ;

[0069] C←C det ∪C trk ;

[0070] C,S←NMS(C,S,τ max );

[0071] Filter S < τ max ;

[0072] Extract the endpoint features: F of the trajectory candidate box C trk ; Extract the endpoint features: F of the detected candidate box C trk ; Extract the endpoint features: F of the detected candidate box C det ; det ;

[0073] After merging the trajectory candidate box C trk and the detected candidate box C det it becomes C det : C det ← C det ∪ C trk ;

[0074] Input the τ value and merge it with the trajectory candidate box C trk : τ ← τ ∪ C trk ;

[0075] Output: the target trajectory τ.

[0076] After the target vehicle tracking step is completed, use Matplotlib to connect the coordinate data of the 4 BoundingBox points of each vehicle in consecutive multiple frames of images to form a continuous curve. Each trajectory represents the movement path of a vehicle from the start of lane change to the end of lane change. The horizontal axis represents the horizontal coordinate (X value) in the image, and the vertical axis represents the vertical coordinate (Y value) in the image. Each trajectory represents the complete path of a corner point from the start to the end of lane change. In this way, it can clearly show the movement trajectory of the vehicle in the video image during the lane change process, providing a basis for subsequent trajectory classification.

[0077] S3. Calculate the longitudinal speed of each autonomous vehicle at the starting moment of historical lane change, divide the longitudinal speed at the starting moment of vehicle historical lane change, the division interval is 20 km / h to 120 km / h, the division step size is 10 km / h, and it is divided into 10 categories in total (for example: i 1 = 20 - 30 km / h, i 2 = 30 - 40 km / h..., i 10 = 110 - 120 km / h), regard the historical lane change trajectory segment of the vehicle corresponding to each category of longitudinal speed as a first sub-segment for analyzing the lane change behavior at different speeds. Next, establish a data set for each first sub-segment. The data set of the first sub-segment includes time tags, vehicle IDs, and the trajectory coordinates of each BoundingBox point of the vehicle.

[0078] S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub - segment. Smooth the position coordinate data of each BoundingBox point of each vehicle. The smoothing method uses the Savitzky - Golay filtering method, and perform differencing on the smoothed coordinate data to calculate the smoothed speed. Acceleration Displacement Speed Acceleration The calculation formulas are as follows:

[0079]

[0080]

[0081] In the formula, where is the lateral coordinate position and the longitudinal coordinate position of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, is the lateral speed and the longitudinal speed of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, is the lateral acceleration and the longitudinal acceleration of the endpoint of an autonomous vehicle numbered n at the i - th frame moment, and ΔT′ is the time interval of each frame when the data set is recorded;

[0082] Use statistical expectation to perform statistical analysis on the lane - changing lateral displacement, lane - changing longitudinal displacement, lane - changing lateral speed, and lane - changing longitudinal speed of each BoundingBox point in the first sub - segment to obtain the expected lane - changing lateral displacement μ x and the expected lane - changing longitudinal displacement μ y and the expected lane - changing lateral speed the expected lane - changing longitudinal speed

[0083]

[0084] Use the maximum likelihood estimation of the one - dimensional Gaussian function to perform statistical analysis on the lane - changing duration in each first sub - segment to obtain the expected lane - changing duration corresponding to the first sub - segment. Among them, the maximum likelihood estimation of the one - dimensional Gaussian function is specifically:

[0085]

[0086] where T is the lane - changing duration, μ T is the expected lane - changing time of this type of lane - changing trajectory segment, is the variance of the time elapsed of the lane - changing trajectory segment.

[0087] Establish a standard data set for each first sub - segment. The standard data set is a one - dimensional time - series data set corresponding to the expected lateral displacement, expected longitudinal displacement, expected lateral speed, expected longitudinal speed, and expected lane - change duration of the vehicle during lane - change.

[0088] S5. In each stage, parameters such as the lateral displacement and acceleration of the vehicle will be different. The start and end points of each stage can be accurately divided through an algorithm. Starting from the first frame of the lane - change video of the autonomous vehicle, when is satisfied, it is recorded as the starting point of the steering stage, and continuously detect When drops to 0.5 m / s 2 , and is satisfied, it is recorded as the starting point of the straightening stage and the end point of the steering stage. When and the change in lateral displacement is less than 0.5 m, it is recorded as the starting point of the adjustment stage and the end point of the straightening stage. Search until the last frame ends and record it as the end point of the adjustment stage.

[0089] Mark according to the determination results of the three lane - change stages on the vehicle's historical lane - change trajectory map. Divide the vehicle's historical lane - change trajectory map into a steering stage, a straightening stage, and an adjustment stage, calculate the duration of each lane - change stage, and take the average value of the duration for correction;

[0090] Compare with the duration of each lane - change stage. Divide each standard data set according to the lane - change stage to obtain sub - standard data sets. According to the combination of the longitudinal speed at the start time of lane - change and the lane - change stage, 30 sub - standard data sets can be obtained. Each sub - standard data set has 16 one - dimensional time - series data sets, including the corresponding relationship between the displacements and speeds of the four endpoints of the vehicle and time. Based on this, an autonomous - driving lane - change standard parameter set is established As shown in Table 1:

[0091] Table 1 One - dimensional time - series data sets in the sub - standard data set

[0092]

[0093] S6. Obtain a standard two - dimensional image of vehicle lane - change based on each sub - standard data set. Each group of images expresses the patterns and features in 4 types of one - dimensional original time - series data, namely the change curves of lane - change lateral displacement over time, lane - change lateral and longitudinal displacements over time, lane - change lateral speed over time, and lane - change longitudinal speed over time. While retaining them, it has a high intra - class similarity and a low inter - class similarity. The specific process of obtaining the standard two - dimensional image is as follows:

[0094] S601. For each sub - standard data set \(T=(t 1 ,t 2 ,…,t L), for \(i\in[1,L]\), perform z - normalization to obtain its standard normal distribution \(U=(u 1 ,u 2 ,\cdots,u L ). The \(i\)-th element \(u i \) in \(U\) is calculated as follows:

[0095]

[0096] t 1 is the value of the time - series data at time \(i\), \(L\) is the length of the time - series data,

[0097] \(\mu\) is the mean of the time - series data \(T\), and \(\sigma\) is the variance of the time - series data;

[0098] S602. Use the Piecewise Aggregate Approximation method to reduce the dimension of \(U\). Select a reduction factor \(r\) to reduce the dimension of \(U\) from \(L\) to \(m\), obtaining the time - series data \(X=(x 1 ,x 2 ,\cdots,x m ):

[0099]

[0100] where \(m\) is the length of \(X\), and \(x i \) is the element \(x i \) corresponding to the \(i\)-th timestamp;

[0101] S603. Successively use the element \(x i \) corresponding to the \(i\)-th timestamp on the time - series data \(X\) as the relative - position reference point for conversion to obtain the matrix \(M\). The value of the \(i\)-th row element in \(M\) is calculated by subtracting \(x i \) from the element value of \(X\). The obtained matrix \(M\) is as follows:

[0102]

[0103] Time - series data Any two timestamps on the time - series data are related by calculating their relative positions, retaining the internal correlation relationship of the time - series data and providing redundant features of the time - series data to improve the generalization ability of the subsequent classification model.

[0104] S604. Convert the matrix \(M\) into a standard two - dimensional vehicle image \(F\) through min - max normalization. The calculation method of \(F\) is as follows:

[0105]

[0106] S7. Obtain the running data of the vehicle to be tested, execute S1 to S6, obtain the two-dimensional image P of the lane change of the vehicle to be tested, and use the standard two-dimensional image F in the same longitudinal speed interval at the starting moment of the historical lane change and the two-dimensional image P of the vehicle to be tested as the input of the improved convolutional neural network classification model; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image P of the vehicle to be tested and the standard two-dimensional image F is greater than 0.75, it is determined that the lane change trajectory of the vehicle to be tested is generated by autonomous driving; when the output result of the improved convolutional neural network classification model is that the similarity rate between the two-dimensional image P of the vehicle to be tested and the standard two-dimensional image F is less than or equal to 0.75, it is determined that the lane change trajectory of the vehicle to be tested is generated by manual driving.

[0107] The improved convolutional neural network includes six layers:

[0108] The first layer: the convolutional layer, by convolving the output of the previous layer with a set of convolutional kernels, extracts a set of features at different positions of the original image, thereby obtaining a feature extraction representation called a feature map.

[0109] The second layer: the batch normalization layer, by performing batch normalization on the values of different feature maps in the previous layer, converts the previous non-fixed data distribution into a fixed data distribution, so that the input distribution during the network training process remains consistent.

[0110] The third layer: the activation layer, uses the scaled exponential linear unit activation function to perform a non-linear mapping on the output of the previous layer, introduces non-linearity into the entire network, and improves the fitting ability of the model.

[0111] The fourth and fifth layers: two fully connected layers, which map the high-level representation of the original image features obtained in the previous feature extraction part to the categories of the images.

[0112] The sixth layer: the output layer, realizes image classification through the Softmax function. Specifically: maps the similarity of the two-dimensional image P to the probability space: assume the similarity value is s = sim(f 1 , f 2 ), uses Softmax to convert the similarity and its complementary value (1 - s) into a binary classification probability distribution:

[0113]

[0114] Set the probability threshold to 75%, and the classification determination:

[0115] Determination result

[0116] Exemplary:

[0117] A video of a vehicle's lane-changing trajectory is collected, and its longitudinal speed at the initial stage of lane-changing is 25 km / h. The lane-changing parameters after collection and processing are represented by a relative position matrix method and converted into a two-dimensional image. Then, the convolutional neural network classification model is used to match the image to be detected with the standard lane-changing sample image of autonomous driving at speed i 1 The similarity probability of the parameters is 81%, and the output result is "auto". Therefore, it can be determined that the trajectory is a lane-changing behavior completed by autonomous driving. If the similarity probability is less than 75% and does not meet the set probability threshold, the output result is "manual", and finally it is determined that the lane-changing trajectory is completed by manual driving.

[0118] Although the present invention has been described in detail with general descriptions and specific embodiments in this specification, some modifications or improvements can be made based on the present invention, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the present invention fall within the scope of the present invention claimed.

Claims

1. A lane-changing trajectory classification method based on drone aerial video, characterized in that: The following steps are involved: S1. Obtain the coordinates of the rectangular box of the vehicle location and the four BoundingBox points in a single-frame image of the autonomous driving vehicle based on the neural network YOLOv5. S2. Based on the DeepSort algorithm, the trajectory coordinates of each BoundingBox point and the vehicle's historical lane-changing trajectory map are obtained. S3. Divide the historical lane-changing trajectory segments of the autonomous driving vehicle to obtain first sub-segments, and establish a data set for each first sub-segment. The data set of the first sub-segment includes a time tag, a vehicle ID, and the trajectory coordinates of each BoundingBox point of the vehicle. S4. Extract the vehicle ID and the trajectory coordinates of each BoundingBox point of the vehicle from the data set of each first sub-segment, calculate the expected lane change lateral displacement, expected lane change longitudinal displacement, expected lane change lateral speed, and expected lane change longitudinal speed of the autonomous driving vehicle through the trajectory coordinates of each BoundingBox point of the vehicle, then extract the vehicle ID and time tag from the data set of each first sub-segment, obtain the vehicle's expected lane change duration through the time tag, and finally establish a standard data set for each first sub-segment. S5. Based on the hybrid search algorithm of the vehicle lane changing process, the historical lane changing trajectory map of the vehicle is divided into three lane changing stages, namely, the starting steering stage, the returning stage and the adjustment stage. Each standard data set is divided according to the lane changing stage to obtain a sub-standard data set. S6. Obtain a standard two-dimensional image of the vehicle lane changing based on the sub-standard data set. S7, obtaining the running data of the vehicle to be tested, executing S1 to S6, obtaining a two-dimensional image of the lane changing of the vehicle to be tested, and determining the lane changing trajectory type of the vehicle to be tested.

2. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: In S1, the specific method for obtaining the coordinates of the four BoundingBox points of the vehicle is: segmenting the lane changing video of the autonomous driving vehicle according to a fixed frame rate to obtain a single frame image.

3. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: S2 specifically includes the following sub-steps: S201, matching the rectangular boxes in the front and rear frames by using the DeepSort algorithm, and assigning a unique vehicle ID to each detected bounding box, to obtain the trajectory coordinates of each BoundingBox point of each vehicle; S202. Use Matplotlib to connect the trajectory coordinates of each BoundingBox point with a line segment, display the trajectory paths of the four corner points of the vehicle in a two-dimensional plane, and generate a historical lane-changing trajectory diagram of the vehicle.

4. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: In S3, the method for dividing the historical lane change trajectory segments of the autonomous driving vehicle is as follows: the longitudinal speed of each autonomous driving vehicle at the start time of the historical lane change is calculated, and the longitudinal speed of the vehicle at the start time of the historical lane change is divided into a division interval of 20km / h to 120km / h, and a division step of 10km / h. The historical lane change trajectory segment of the vehicle corresponding to each type of longitudinal speed is taken as a first sub-segment.

5. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: S4 includes the following sub-steps: S401, extracting a time tag, a vehicle ID, and the trajectory coordinates of each BoundingBox point of the vehicle from a data set of each first sub-segment; S402, using the Savitzky-Golay filter method to smooth the trajectory coordinates of each BoundingBox point, and performing a difference on the smoothed trajectory coordinates to calculate the smoothed displacement speed Acceleration Displacement speed Acceleration The calculation formula is as follows: In the formula, is the horizontal coordinate position and the vertical coordinate position of the endpoint of a certain autonomous driving vehicle numbered n at the i-th frame time, is the lateral speed and longitudinal speed of the endpoint numbered n of a certain autonomous driving vehicle at the i-th frame, is the lateral acceleration and longitudinal acceleration of the endpoint numbered n of a certain autonomous driving vehicle at the i-th frame, and ΔT′ is the time interval between each frame when the data set is recorded; The lane-changing lateral displacement, lane-changing longitudinal displacement, lane-changing lateral speed, and lane-changing longitudinal speed of each BoundingBox point in the first sub-segment are statistically analyzed by using statistical expectation to obtain the expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral speed, and expected lane-changing longitudinal speed of each BoundingBox point: S403, using one-dimensional Gaussian function maximum likelihood estimation to perform statistical analysis on the lane change duration in each first sub-segment to obtain an expected lane change duration corresponding to the first sub-segment, Among them, the maximum likelihood estimation of the one-dimensional Gaussian function is specifically: Where T is the lane change duration, μ T is the expected lane-changing time for this type of lane-changing trajectory segment, is the variance of the elapsed time of the lane-changing trajectory segments. S404: Establish a standard data set for the first sub-segment, where the standard data set is a one-dimensional time series data set corresponding to the vehicle's expected lane-changing lateral displacement, expected lane-changing longitudinal displacement, expected lane-changing lateral speed, expected lane-changing longitudinal speed, and expected lane-changing duration.

6. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: S5 specifically includes: S501, determining three stages of the vehicle's historical lane-changing trajectory through threshold conditions and speed and acceleration change trends; S502, marking the vehicle's historical lane-changing trajectory map according to the determination results of the three lane-changing stages, dividing the vehicle's historical lane-changing trajectory map into a starting turning stage, a returning stage, and an adjustment stage, calculating the duration of each lane-changing stage, and taking an average value for correction; S503 . Compare the duration of each lane-changing phase and divide each standard data set according to the lane-changing phase to obtain sub-standard data sets.

7. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: S6 specifically includes: S601, each sub-standard data set T=(t1, t2, ..., t L ), i∈[1,L], perform z standardization and obtain its standard normal distribution U=(u1,u2,…,u L ), the i-th element u in U i The calculation method is as follows: t1 is the value of the time series data at time i, L is the length of the time series data, μ is the mean of the time series data T, σ is the variance of the time series data; S602, use the segmented aggregation approximation method to reduce the dimension of U, select a reduction factor r, reduce the dimension of U from L to m, and obtain the time series data X = (x1, x2, ..., x m ): Where m is the length of X, x i is the element x corresponding to the i-th timestamp i ; S603, sequentially convert the element x corresponding to the i-th timestamp on the time series data X i As a relative position reference point, the matrix M is obtained, in which the element value of the i-th row is the element value of X minus x i The calculation shows that the obtained matrix M is as follows: S604, transform the matrix M into a vehicle standard two-dimensional image F through min-max standardization, and the calculation method of F is as follows:

8. The lane-changing trajectory classification method based on drone aerial video according to claim 1 is characterized in that: In S7, the lane-changing trajectory type of the vehicle to be tested is determined by an improved convolutional neural network classification model. The specific method is as follows: the standard two-dimensional image F and the two-dimensional image of the vehicle to be tested are respectively used as inputs of the improved convolutional neural network classification model; when the output result of the improved convolutional neural network classification model is: the similarity between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is greater than 0.75, it is determined that the lane-changing trajectory of the vehicle to be tested is generated by automatic driving; when the output result of the improved convolutional neural network classification model is: the similarity between the two-dimensional image of the vehicle to be tested and the standard two-dimensional image is less than or equal to 0.75, it is determined that the lane-changing trajectory of the vehicle to be tested is generated by manual driving.

9. The lane-changing trajectory classification method based on drone aerial video according to claim 6 is characterized in that: In S501, the specific method for determining the three stages of the vehicle's historical lane-changing trajectory is: start searching from the first frame, and when the When , it is recorded as the starting point of the turning phase, and the detection is continued. when Down to 0.5m / s 2 ,and When , it is recorded as the starting point of the return phase and the end point of the turning phase. If the lateral displacement change is less than 0.5 m, it is recorded as the starting point of the adjustment phase and the end point of the return phase. The search ends at the last frame and is recorded as the end point of the adjustment phase.

10. The lane-changing trajectory classification method based on drone aerial video according to claim 8, characterized in that: The improved convolutional neural network classification model includes a convolution layer, the vehicle standard two-dimensional image F is the input of the convolution layer, the output of the convolution layer is the input of the batch normalization layer, the output of the batch normalization layer is the input of the activation layer, the output of the activation layer is the input of the first fully connected layer, the output of the first fully connected layer is the input of the second fully connected layer, the output of the second fully connected layer is the input of the output layer, and the output of the output layer is the output of the convolutional neural network classification model.

Images