A vehicle motion trajectory prediction method for autonomous valet parking scenarios

Abstract

The present invention discloses a vehicle motion trajectory prediction method for autonomous valet parking scenarios, comprising the following steps: S1, calibrating the spatial perception range of the target vehicle, and vectorizing the map and objects in the bird's-eye view video of the autonomous valet parking scenario, and obtaining the motion information of the target vehicle and surrounding vehicles from the video; S2, searching for empty parking spaces and road endpoints around the target vehicle as parking intention points of the target vehicle within the vehicle perception range, extracting scene factors and vehicle self factors, and using neural networks and probability distribution calculations to obtain the parking intention information of the target vehicle; S3, generating multimodal vehicle future trajectories through an encoder-decoder algorithm structure of an attention mechanism and based on the motion information of the target vehicle and surrounding vehicles, the scene information of the target vehicle, the parking intention information of the target vehicle, and the vector map information. According to the present invention, the vehicle motion trajectory in the autonomous valet parking scenario can be predicted more accurately, thereby supporting autonomous driving decision-making planning.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous valet parking, and in particular to a vehicle motion trajectory prediction method for an autonomous valet parking scenario. Background Art

[0002] With the deployment of a large number of road test sensors in autonomous valet parking scenarios, a large amount of vehicle trajectory data in autonomous valet parking scenarios has been collected. These trajectory data provide a rich data foundation for intelligent management and safety detection in autonomous valet parking scenarios, and also provide trajectory information of surrounding vehicles for autonomous driving vehicles in autonomous valet parking scenarios. However, considering the occlusion between vehicles, there are missing vehicle trajectory data, and autonomous driving vehicles need to predict their future trajectories from the historical trajectory information of surrounding vehicles. Therefore, there is an urgent need for a vehicle trajectory prediction method for autonomous valet parking scenarios to complete the missing trajectories in autonomous valet parking scenarios, and provide trajectory prediction functions for autonomous driving vehicles.

[0003] At present, the prediction of vehicle motion trajectory in autonomous valet parking scenarios mainly faces the following difficulties: ① The parking intentions of vehicles in autonomous valet parking scenarios are diverse, and vehicles can choose different parking spaces to park or leave the area; ② The interaction of vehicles in autonomous valet parking scenarios is complex; ③ There is a lack of traffic rules in autonomous valet parking scenarios, and the motion trajectory of vehicles is more diverse and difficult to predict. In recent years, researchers in this field have predicted vehicle motion trajectories from various angles, such as using models based on Transformer and long short-term memory networks, but the following problems still exist:

[0004] (1) The research scenarios of existing methods are mostly urban roads, and there is a lack of vehicle motion trajectory prediction methods for autonomous valet parking scenarios.

[0005] (2) Existing methods lack research on the diverse parking intentions of vehicles in autonomous valet parking scenarios and the interactions between vehicles. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a vehicle motion trajectory prediction method for autonomous valet parking scenarios, which can more accurately predict the vehicle motion trajectory in autonomous valet parking scenarios, thereby supporting autonomous driving decision planning. In order to achieve the above-mentioned purpose and other advantages according to the present invention, a vehicle motion trajectory prediction method for autonomous valet parking scenarios is provided, comprising:

[0007] S1. Calibrate the spatial perception range of the target vehicle, vectorize the map and objects in the bird's-eye view video of the autonomous valet parking scene, and obtain the motion information of the target vehicle and surrounding vehicles from the video;

[0008] S2. Search for empty parking spaces and road endpoints around the target vehicle within the vehicle perception range as the parking intention points of the target vehicle, extract scene factors and vehicle factors, and use neural networks and probability distribution calculations to obtain the parking intention information of the target vehicle;

[0009] S3. Generate a multimodal vehicle future trajectory through the encoder-decoder algorithm structure of the attention mechanism and based on the motion information of the target vehicle and surrounding vehicles, the scene information where the target vehicle is located, the parking intention information of the target vehicle and the vector map information.

[0010] Preferably, step S1 specifically includes the following steps:

[0011] S11, setting the perception range of the target vehicle in the autonomous valet parking scenario so that the vehicle parking intention information and trajectory generation are calculated within the spatial perception range of the target vehicle;

[0012] S12, converting the original aerial map of the autonomous valet parking scene into a vectorized map, in which the road area, vehicles and parking spaces are marked with different colors, and the historical trajectory of the vehicle is represented as a color gradient behind the rear of the vehicle, and the lighter the color, the farther the historical time corresponding to the historical trajectory position;

[0013] S13. Collect the historical trajectory position coordinates of the target vehicle from the vector video.

[0014] Preferably, the parking intention information of the vehicle in step S2 means the probability that the vehicle intends to park in each empty parking space around or leave toward each road end position, which specifically includes the following steps:

[0015] S21, extracting scene factors and vehicle factors, wherein the scene factors are the target vehicle driving scene, the distance between the empty parking space and the target vehicle, and the steering angle, and the vehicle factors are the target vehicle historical trajectory, the distance between the target vehicle and the exit of the autonomous valet parking scene, and the time the vehicle stays in the autonomous valet parking scene;

[0016] S22. Based on the scene factors and vehicle factors, the convolutional neural network model is used to extract the scene factor features, the long short-term memory model is used to extract the historical trajectory features of the target vehicle, the linear layer is used to extract the distance and steering angle features between the target vehicle and the empty parking space in the vehicle factors, and the fully connected layer is used to merge all the factor features to calculate the probability that the target vehicle is expected to park in each empty parking space around it and the probability that it will leave without stopping.

[0017] S23, extracting the distance and steering angle between each road endpoint and the target vehicle, and sorting each road endpoint according to the distance and steering angle, and assigning the probability of the vehicle leaving without stopping to each road endpoint according to the sorting:

[0018]

[0019] in is the probability that the target vehicle i leaves the road end r, P′ p is the probability that the target vehicle leaves without stopping, Sort(i) represents the sorting position of the i road endpoints, k is the number of all road endpoints, and a and b are hyperparameters;

[0020] S24. After the probabilities of all parking intention points are calculated, the first n parking intention points sorted from largest to smallest are selected as the parking intention information of the vehicle.

[0021] Preferably, the encoder-decoder algorithm structure of the attention mechanism in step S3 is composed of an encoder part and a decoder part, wherein the encoder part is used to perform high-dimensional encoding on the vehicle trajectory, parking intention, environmental features and vehicle interaction features, and the encoder part extracts vehicle features with time features from the historical trajectory of the vehicle through a self-attention mechanism model, calculates the influence of the vehicle parking intention on the vehicle features using a cross-attention mechanism model, calculates the influence of the scene where the target vehicle is located on the vehicle features through a CNN model and a cross-attention mechanism model, and calculates the influence of the interaction between vehicles on the vehicle features through a vehicle interaction model of a vehicle interaction scene;

[0022] The decoder part is used to decode the encoding result into a future trajectory of the vehicle that meets the requirements, and output the future trajectory of the vehicle by superimposing multiple layers of Transformer decoders.

[0023] Preferably, the vehicle interaction model of the vehicle interaction scenario is composed of a vehicle interaction scenario classifier based on a convolutional neural network and a plurality of vehicle interaction calculation models based on an attention mechanism;

[0024] The interaction scenarios between the target vehicle and surrounding vehicles are classified through the vehicle interaction scenario classifier, and then the interaction impact between vehicles is calculated using the corresponding vehicle interaction calculation model according to different scenario categories.

[0025] Compared with the prior art, the present invention has the following advantages: it can fully consider the impact of driving scenarios, intentions, environmental influences and vehicle interaction factors on vehicle trajectory, and achieve high-precision, multi-modal vehicle motion trajectory prediction. While predicting the vehicle trajectory, it can also explain the parking intention of the vehicle in the autonomous valet parking scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of a vehicle motion trajectory prediction method for an autonomous valet parking scenario according to the present invention;

[0027] Figure 2 A vector map schematic diagram of a vehicle motion trajectory prediction method for an autonomous valet parking scenario according to the present invention;

[0028] Figure 3 A vehicle parking intention prediction flow chart of a vehicle motion trajectory prediction method for an autonomous valet parking scenario according to the present invention;

[0029] Figure 4 A CNN module diagram of a vehicle motion trajectory prediction method for an autonomous valet parking scenario according to the present invention;

[0030] Figure 5 The figure is a vehicle interaction influence calculation diagram of the vehicle motion trajectory prediction method for the autonomous valet parking scenario according to the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] Reference Figure 1-5 , a vehicle motion trajectory prediction method for autonomous valet parking scenarios, comprising the following steps:

[0033] S1. Calibrate the spatial perception range of the target vehicle, and vectorize the map and objects in the bird's-eye view video of the autonomous valet parking scene, and obtain the motion information of the target vehicle and surrounding vehicles from the video; this step is specifically as follows:

[0034] S11. Set the perception range of the target vehicle in the autonomous valet parking scenario to M×M, where M in this embodiment is 100, and the subsequent vehicle parking intention information and trajectory generation are calculated within the spatial perception range of the target vehicle;

[0035] S12. Convert the original aerial map of the autonomous valet parking scene into a vectorized map, such as Figure 2 As shown in the figure, the gray part of the map is the road area, the black part is the area outside the road boundary, the blue part is the position of the parked vehicle, the red part is the target vehicle, the yellow part is other vehicles still driving in the parking lot, and the green part is an empty parking space. At the same time, the historical trajectory of the vehicle is represented by the red gradient part behind the rear of the vehicle. The lighter the color, the farther the historical time corresponding to the historical trajectory position.

[0036] S13. Collect the historical trajectory position coordinates of the target vehicle from the vector video.

[0037] S2. Search for empty parking spaces and road endpoints around the target vehicle within the vehicle perception range as the parking intention points of the target vehicle, extract scene factors and vehicle factors, and use neural networks and probability distribution calculations to obtain the parking intention information of the target vehicle. This step is specifically as follows:

[0038] S21, extracting scene factors and vehicle factors, wherein the scene factors are the target vehicle driving scene, the distance between the empty parking space and the target vehicle, and the steering angle, and the vehicle factors are the target vehicle historical trajectory, the distance between the target vehicle and the exit of the autonomous valet parking scene, and the time the vehicle stays in the autonomous valet parking scene;

[0039] S22. Based on the scene factors and the vehicle's own factors, a convolutional neural network model is used to extract the scene factor features, a long short-term memory model is used to extract the historical trajectory features of the target vehicle, a linear layer is used to extract the distance and steering angle features between the target vehicle and the empty parking spaces in the vehicle's own factors, and a fully connected layer is used to merge all factor features to calculate the probability that the target vehicle expects to park in each empty parking space around it and the probability of leaving without stopping.

[0040] For the extraction of scene factor features, the method crops the map image to obtain an M×M image centered on the location of the target vehicle, and fixes the coordinate axes of the image to the target vehicle through offset rotation, always setting the direction angle of the target vehicle to the positive direction of the image x-axis. In order to represent the one-to-one correspondence between the target vehicle and the surrounding empty parking spaces, the model marks each empty parking space with purple, and obtains a set of images with the same number as the number of empty parking spots. In addition, an unmarked image is added to the image set to correspond to the situation where the vehicle is not parked. After obtaining the image set, the model uses CNN to extract the corresponding environmental information from the image. The CNN network structure used in this method is as follows: Figure 4 As shown in the figure, the CNN network contains a total of three convolutional modules, each of which contains a convolutional layer, a pooling layer and a Dropout layer.

[0041] For the historical trajectory features of the target vehicle, the method uses the LSTM long short-term memory model to extract the features of the vehicle's historical trajectory. The final model will output a length of L traj =1000 vehicle historical trajectory feature vector.

[0042] For the factor features of the vehicle itself, the method uses the Euclidean distance to represent the distance between the empty parking space and the vehicle, and calculates the steering angle between the vehicle and the empty parking space through a geometric formula. In addition, we set the distance and steering angle to 0 to indicate that the vehicle does not stop. At the same time, the method extracts the distance between the vehicle and the parking lot exit and the time the vehicle stays in the parking lot from the data through the code, and converts the above data into a L-sized data through the fully connected layer. dis_ang =5 eigenvector.

[0043] The final model integrates the features of all influencing factors through vector concatenation and fully connected layers, and outputs the selection score of each empty parking space. (value between 0 and 1), representing the attractiveness of the parking space to the target vehicle i. Then the model uses the Softmax function to convert the selected score of each empty parking space into Converted to the probability of selecting an empty parking space in P′ represents the probability that the target vehicle expects to park in the sth empty parking space. p Represents the probability that the target vehicle leaves the area without stopping.

[0044] S23, extracting the distance and steering angle between each road endpoint and the target vehicle, and sorting each road endpoint according to the distance and steering angle, and assigning the probability of the vehicle leaving without stopping to each road endpoint according to the sorting:

[0045]

[0046] in is the probability that the target vehicle i leaves the road end r, P′ p is the probability that the target vehicle leaves without stopping, Sort(i) represents the sorting position of the i road endpoints, k is the number of all road endpoints, a and b are hyperparameters, and a and b are set to 0.9 and 0.8 respectively in this example;

[0047] S24. After the probabilities of all parking intention points are calculated, the first n parking intention points sorted from largest to smallest are selected as the parking intention information of the vehicle.

[0048] S3. Generate a multimodal vehicle future trajectory through an encoder-decoder algorithm structure of an attention mechanism and based on the motion information of the target vehicle and surrounding vehicles, the scene information where the target vehicle is located, the parking intention information of the target vehicle, and the vector map information. The encoder-decoder algorithm structure of the attention mechanism in step S3 consists of an encoder part and a decoder part, wherein the encoder part is used to perform high-dimensional encoding of the vehicle trajectory, parking intention, environmental features, and vehicle interaction features. The encoder part extracts vehicle features with time features from the historical trajectory of the vehicle through a self-attention mechanism model, calculates the impact of the vehicle parking intention on the vehicle features using a cross-attention mechanism model, calculates the impact of the scene where the target vehicle is located on the vehicle features through a CNN model and a cross-attention mechanism model, and calculates the impact of the interaction between vehicles on the vehicle features through a vehicle interaction model of a vehicle interaction scene; the attention mechanism model inputs the historical trajectory sequence V of the target vehicle = (V 1 , V 2 , ..., V obs ), after position encoding, it is input into the encoder, and the output sequence χ=(χ 1 , χ 2 , ..., χ obs ). The self-attention mechanism is a variant of the attention mechanism. Given the trajectory of the vehicle L = (l1, l2, ..., l n ), the model first uses matrix operations to obtain the high-dimensional projection vector q = (q1, q2, ..., q n )、k=(k1,k2,...,k n )、v=(v1,v2,...,v n ), the calculation process is as follows:

[0049] q i =W q ·l i k i =W k ·l i v i =W v ·l i

[0050] Where W q , W k , W v They are the calculation matrices corresponding to vectors q, k, and v respectively.

[0051] Then, the model will calculate the sequence element l based on q, k, v i The correlation matrix Attn(q i , k j The specific calculation process is as follows:

[0052]

[0053] According to the correlation matrix, the sequence element l i The value vector v i After fusion, we finally get the vehicle feature L′ which takes into account the trajectory time feature. The specific calculation process is as follows, where φ is the activation function:

[0054]

[0055]

[0056] In the self-attention mechanism model, the position encoding used is as follows:

[0057]

[0058] Where i represents the channel number and d represents the vector dimension of the sequence.

[0059] Secondly, the encoder uses the cross-attention mechanism model to calculate the impact of the vehicle's parking intention on the vehicle's features. The cross-attention mechanism model is also a variant of the attention mechanism. In this cross-attention mechanism, q is set as the high-order projection vector of the vehicle's features, k and v are both set as the high-dimensional projection vectors of the vehicle's intentions, where q and k are used for correlation calculation, and v is used as a value vector for feature fusion.

[0060] Then, the encoder uses the CNN-based model and the cross-attention mechanism model to calculate the impact of the scene where the target vehicle is located on the vehicle characteristics. The method first collects a series of semantic driving scene pictures with the target vehicle as the center and the positive direction of the target vehicle as the positive direction of the x-axis from the original data set. There are obs pictures in the picture set, each picture records a vehicle driving scene at a historical moment, and the picture size is fixed to M×M. After collecting the driving scene picture set, the model uses the CNN convolutional network algorithm to process the pictures to obtain a set of one-dimensional vectors corresponding to the vehicle driving scene characteristics of the vehicle at each historical moment. Then, for the cross-attention mechanism model, the model sets q and v as the high-dimensional projection vectors of the vehicle driving scene characteristics on the basis of the original attention mechanism, and k is set as the high-dimensional projection vector of the vehicle characteristics, where q and k are used to calculate the interactive correlation between the vehicle and the scene characteristics, and v is used as the value vector for the feature fusion of the vehicle driving scene. The encoder based on the cross-attention mechanism can obtain the impact of the surrounding driving environment on the vehicle characteristics and integrate it into the vehicle feature sequence.

[0061] The encoder uses a vehicle interaction model based on vehicle interaction scenarios to calculate the impact of vehicle interactions on vehicle characteristics. The specific process is as follows Figure 5As shown in the figure, the vehicle interaction model based on the vehicle interaction scene is composed of a CNN-based vehicle interaction scene classifier and multiple vehicle interaction calculation models based on the attention mechanism. The model first collects the relative position picture of the vehicle. The target vehicle and the interaction vehicle in the picture are marked in red, and the future movement trend of the vehicle is additionally marked in the picture. The picture is centered on the target vehicle and the positive direction of the target vehicle is the positive direction of the x-axis. By inputting the relative position picture of the vehicle, the vehicle interaction scene classifier can output the category of the vehicle interaction scene. According to different interaction scene categories, different vehicle interaction calculation models are selected to calculate the interaction effect between vehicles. For the vehicle interaction calculation model based on the attention mechanism, we set q and k as the high-dimensional projection vectors of the target vehicle driving scene picture and the vehicle relative position picture on the basis of the original attention mechanism, and v is set as the high-dimensional projection vector of the historical trajectory of the surrounding vehicles, where q and k are used to calculate the interaction correlation between vehicles, and v is used as a value vector for the fusion of interaction features between vehicles. Finally, this part obtains the vehicle features that consider the interaction effect between vehicles.

[0062] The decoder part is used to decode the encoding result into a future trajectory of the vehicle that meets the requirements, and output the future trajectory of the vehicle by superimposing multiple layers of Transformer decoders.

[0063] In step S31, different vehicle parking intention positions calculated in step S2 may be selected to generate different future trajectories of the vehicle, thereby achieving multimodal trajectory prediction.

[0064] In summary, the present invention aims at the multimodal and difficult-to-predict problems of the vehicle motion trajectory prediction task in the autonomous valet parking scene, and proposes a high-precision vehicle motion trajectory prediction model that integrates the vehicle parking intention. The vehicle motion trajectory prediction method mainly includes: calibrating the spatial perception range of the target vehicle, and vectorizing the map and objects in the bird's-eye view video of the autonomous valet parking scene, and obtaining the motion information of the target vehicle and surrounding vehicles from the video; within the vehicle perception range, searching for empty parking spaces and road endpoints around the target vehicle as the parking intention points of the target vehicle, extracting scene factors and vehicle factors, and using neural networks and probability distribution calculations to obtain the parking intention information of the target vehicle; using a vehicle trajectory generation method based on an attention mechanism to generate a multimodal vehicle future trajectory based on the motion information of the target vehicle and surrounding vehicles, the scene information of the target vehicle, the parking intention information of the target vehicle, and the vector map information.

[0065] The number of devices and processing scales described here are used to simplify the description of the present invention, and the application, modification and variation of the present invention will be obvious to those skilled in the art.

[0066] Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and implementation modes. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

Claims

1. A vehicle motion trajectory prediction method for autonomous valet parking scenarios, characterized in that: The following steps are involved: S1. Calibrate the spatial perception range of the target vehicle, vectorize the map and objects in the bird's-eye view video of the autonomous valet parking scene, and obtain the motion information of the target vehicle and surrounding vehicles from the video; S2. Search for empty parking spaces and road endpoints around the target vehicle within the vehicle perception range as the parking intention points of the target vehicle, extract scene factors and vehicle factors, and use neural networks and probability distribution calculations to obtain the parking intention information of the target vehicle; The parking intention information of the vehicle in step S2 means the probability that the vehicle intends to park in each empty parking space around or leave towards each road end position, which specifically includes the following steps: S21, extracting scene factors and vehicle factors, wherein the scene factors are the target vehicle driving scene, the distance between the empty parking space and the target vehicle, and the steering angle, and the vehicle factors are the target vehicle historical trajectory, the distance between the target vehicle and the exit of the autonomous valet parking scene, and the time the vehicle stays in the autonomous valet parking scene; S22. Based on the scene factors and vehicle factors, the convolutional neural network model is used to extract the scene factor features, the long short-term memory model is used to extract the historical trajectory features of the target vehicle, the linear layer is used to extract the distance and steering angle features between the target vehicle and the empty parking space in the vehicle factors, and the fully connected layer is used to merge all the factor features to calculate the probability that the target vehicle is expected to park in each empty parking space around it and the probability that it will leave without stopping. S23, extracting the distance and steering angle between each road endpoint and the target vehicle, and sorting each road endpoint according to the distance and steering angle, and assigning the probability of the vehicle leaving without stopping to each road endpoint according to the sorting: ， in The target vehicle Towards the end of the road The probability of leaving is the probability that the target vehicle leaves without stopping, Indicated The sorted positions of the road endpoints, is the number of all road endpoints, , is a hyperparameter; S24, after calculating the probabilities of all parking intention points, select the first parking intention points as parking intention information of the vehicle; S3. Generate a multimodal vehicle future trajectory through the encoder-decoder algorithm structure of the attention mechanism and based on the motion information of the target vehicle and surrounding vehicles, the scene information where the target vehicle is located, the parking intention information of the target vehicle and the vector map information.

2. The vehicle motion trajectory prediction method for autonomous valet parking scenarios according to claim 1, characterized in that: Step S1 specifically includes the following steps: S11, setting the perception range of the target vehicle in the autonomous valet parking scenario so that the vehicle parking intention information and trajectory generation are calculated within the spatial perception range of the target vehicle; S12, converting the original aerial map of the autonomous valet parking scene into a vectorized map, in which the road area, vehicles and parking spaces are marked with different colors, and the historical trajectory of the vehicle is represented as a color gradient behind the rear of the vehicle, and the lighter the color, the farther the historical time corresponding to the historical trajectory position; S13. Collect the historical trajectory position coordinates of the target vehicle from the vector video.

3. The vehicle motion trajectory prediction method for autonomous valet parking scenarios according to claim 1, characterized in that: The encoder-decoder algorithm structure of the attention mechanism in step S3 is composed of an encoder part and a decoder part, wherein the encoder part is used to perform high-dimensional encoding of vehicle trajectory, parking intention, environmental features and vehicle interaction features, and the encoder part extracts vehicle features with time features from the historical trajectory of the vehicle through a self-attention mechanism model, uses a cross-attention mechanism model to calculate the impact of the vehicle parking intention on the vehicle features, calculates the impact of the scene where the target vehicle is located on the vehicle features through a CNN model and a cross-attention mechanism model, and calculates the impact of the interaction between vehicles on the vehicle features through a vehicle interaction model of a vehicle interaction scene; The decoder part is used to decode the encoding result into a future trajectory of the vehicle that meets the requirements, and output the future trajectory of the vehicle by superimposing multiple layers of Transformer decoders.

4. The vehicle motion trajectory prediction method for autonomous valet parking scenarios according to claim 3, characterized in that: The vehicle interaction model of the vehicle interaction scenario consists of a vehicle interaction scenario classifier based on a convolutional neural network and multiple vehicle interaction calculation models based on the attention mechanism; The interaction scenarios between the target vehicle and surrounding vehicles are classified through the vehicle interaction scenario classifier, and then the interaction impact between vehicles is calculated using the corresponding vehicle interaction calculation model according to different scenario categories.

Images