Automatic driving track planning method, device, equipment, medium and product

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an automatic driving trajectory planning method and device, equipment, a medium and a product, and the method comprises the steps: obtaining the historical driving trajectory of each entity, and calculating the spatial-temporal characteristics of a driving scene; based on the spatio-temporal features, utilizing a multi-head attention mechanism to model the interaction relationship among the entities to obtain splicing fusion features; predicting a possible destination of the target vehicle based on the splicing fusion features; predicting a future driving track of each surrounding vehicle, and performing feature fusion on the future driving track and each possible destination to obtain a safety score; and determining a final destination based on the safety score, and planning a driving track of the target vehicle based on the final destination. According to the method, the interaction relation of all entities is modeled, the information in the driving scene is fully extracted, the influence of surrounding vehicles on the own vehicle and the influence among the surrounding vehicles are considered, and collision with other vehicles can be avoided.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an autonomous driving trajectory planning method, apparatus, device, medium, and product. Background Technology

[0002] With the rapid development of computer-related technologies, autonomous driving systems have been increasingly widely applied and implemented. The goal of an autonomous driving system is to control the vehicle to autonomously perceive complex driving scenarios, plan its trajectory, and safely reach its destination with little or no driver intervention. Therefore, safety is the core evaluation metric for autonomous driving systems.

[0003] However, in related technologies, trajectory planning for autonomous driving systems requires filtering from a vast trajectory database. The final planning result cannot be fine-tuned according to the actual scenario, which has certain limitations. Moreover, the planning only considers the lane information of the vehicle in the driving scenario, which can easily lead to vehicle collisions and affect the safety of autonomous driving. Summary of the Invention

[0004] In view of the above problems, this disclosure is made to provide an autonomous driving trajectory planning method, apparatus, device, medium and product.

[0005] According to one aspect of this disclosure, an autonomous driving trajectory planning method is provided, comprising: The system acquires the historical driving trajectories of each entity in a preset driving scenario, and calculates the spatiotemporal characteristics of the driving scenario based on the historical driving trajectories. The entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points. Based on spatiotemporal features, a multi-head attention mechanism is used to model the interaction relationships between entities to obtain splicing and fusion features; Based on the splicing and fusion features, the possible destination of the target vehicle is predicted; Predict the future driving trajectories of surrounding vehicles, fuse the future driving trajectories of surrounding vehicles with features of each possible destination, and obtain a safety score for each possible destination. Based on the highest safety score, the final destination is identified from all possible destinations, and the driving trajectory of the target vehicle is planned based on the final destination.

[0006] The beneficial effects of this solution are as follows: By modeling the interaction relationships of all entities in the driving scene using attention, information from the driving scene is fully extracted, resulting in a better spatiotemporal feature representation of the driving scene, i.e., feature splicing and fusion. This takes into account the impact of surrounding vehicles on the vehicle itself, as well as the impact between surrounding vehicles, which is beneficial for real-time trajectory planning and avoiding collisions with other vehicles. By predicting the future trajectories of surrounding vehicles, the safety score of each possible destination is calculated, and the final destination is selected, ensuring safety while achieving trajectory planning. The future trajectories of surrounding vehicles can be predicted using a multi-head attention mechanism, which accelerates the inference speed of trajectory prediction compared to the single-step inference of traditional recurrent neural networks.

[0007] Furthermore, according to one aspect of the autonomous driving trajectory planning method of this disclosure, based on historical driving trajectories, the spatiotemporal characteristics of the driving scenario are obtained, including: Based on historical driving trajectories, obtain time features in the time series; Construct an occupied grid map of the driving scene with the current position of the target vehicle as the origin; Based on the relative positions of the surrounding vehicles and the target vehicle, obtain the index grid of each surrounding vehicle in the occupied grid map; The temporal characteristics of the surrounding vehicles are filled into the index raster to obtain the spatiotemporal characteristics of the driving scene.

[0008] In one or more embodiments, the beneficial effect of this solution is that by constructing an occupied grid map to model the driving scene, it is beneficial to obtain the spatiotemporal features of the complete scene that integrates spatial location and temporal features.

[0009] Furthermore, according to one aspect of the autonomous driving trajectory planning method of this disclosure, the multi-head attention mechanism includes multiple scaled dot product attention operations working in parallel; Based on spatiotemporal features, a multi-head attention mechanism is used to model the interaction relationships between entities, obtaining spliced ​​and fused features, including: Different linear transformations of the spatiotemporal features are applied using scaling dot product attention to obtain the query matrix, key matrix, and value matrix; wherein the feature dimensions of the query matrix, key matrix, and value matrix are the same. Obtain the transpose of the key matrix, and calculate the dot product of the query matrix and the transpose matrix; The attention weight matrix is ​​obtained by scaling the dot product by dividing it by the feature dimension and then normalizing it. Based on the attention weight matrix and value matrix, the output features of each scaled dot product attention are obtained; The output features of each scaled dot product attention are concatenated to obtain the concatenated and fused features.

[0010] In one or more embodiments, the beneficial effects of this solution are as follows: by quantifying the pairwise interactivity between all entities in the driving scene through the attention weight matrix, the output features of each scaled dot product attention (head) are spliced ​​together to obtain spliced ​​and fused features rich in interactive information, and a "network of interaction and influence relationship" between entities is constructed, which is conducive to forming a better spatiotemporal feature expression of the driving scene.

[0011] Furthermore, according to one aspect of the autonomous driving trajectory planning method of this disclosure, based on stitched and fused features, the possible destination of the target vehicle is predicted, including: Based on the target vehicle's historical driving trajectory, obtain the target vehicle's temporal characteristics; The splicing and fusion features and the temporal features of the target vehicle are input into a pre-defined multilayer perceptron to predict the possible destination of the target vehicle.

[0012] In one or more embodiments, the beneficial effects of this solution are: by using a multi-head attention mechanism, the dependency between the temporal features and the stitched-to-fuse features of the target vehicle is captured, and multiple possible destinations are predicted in parallel, which can reflect the diversity of autonomous driving intentions. In addition, for each possible destination, a complete and smooth trajectory from the current location to that destination can be completed.

[0013] Furthermore, according to one aspect of the autonomous driving trajectory planning method of this disclosure, the future driving trajectories of surrounding vehicles are fused with features of each possible destination to obtain a safety score for each possible destination, including: The driving scenario is modeled as a strongly connected graph, where nodes represent entities and edges represent the interaction relationships between entities. Obtain node i and its neighboring node j in the strongly connected graph. Calculate the attention weight between node i and its neighboring node j based on the future time features of node i and its neighboring node j. The future time features are obtained based on the corresponding future driving trajectory, and the attention weight represents the degree to which node i is influenced by its neighboring node j. Based on the attention weights between node i and each of its neighboring nodes, the node fusion features of each node are obtained. Using each possible destination as the query and the node fusion feature as the key and value, multi-head attention calculation is performed to obtain decision features; The decision features are input into a pre-defined multilayer perceptron to obtain a security score for each possible destination.

[0014] In one or more embodiments, the beneficial effects of this solution are as follows: by modeling the entire driving scene as a strongly connected graph through a graph attention network, the method of constructing an occupied grid map is abandoned, which can reduce overhead and avoid the secondary accumulation of prediction errors. Using the future driving trajectories of surrounding vehicles as the basis for assessing collision risk, a future scenario representing the coexistence of "the vehicle's possible destination" and "the future driving trajectories of surrounding vehicles" is constructed.

[0015] Furthermore, according to one aspect of the autonomous driving trajectory planning method disclosed herein, the possible destination is generated by a generator, the safety score is generated by a safety discriminator, and the adversarial training process between the safety discriminator and the generator includes: The training dataset obtained from real driving scenarios is input into the safety discriminator to generate the real purpose; the training dataset includes the vehicle's real driving purpose and historical trajectory; Input historical trajectories into the generator to generate predictions based on those trajectories. The security discriminator assigns a first security score to the true purpose and a second security score to the predicted purpose; wherein the first security score is greater than the second security score. Based on the second safety score, the generator's own parameters are updated, and the steps for generating predictions based on historical trajectories are repeated until the difference between the first and second safety scores meets the preset conditions, thus obtaining a converged safety discriminator and generator.

[0016] In one or more embodiments, the beneficial effect of this solution is that by conducting adversarial training between the safety discriminator and the generator, the driving scenario is measured from a safety perspective, thus solving the problem of lacking a differentiable safety loss function.

[0017] According to another aspect of this disclosure, an autonomous driving trajectory planning device is provided, comprising: The feature calculation module is used to obtain the historical driving trajectories of each entity in a preset driving scenario, and calculate the spatiotemporal features of the driving scenario based on the historical driving trajectories; where the entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points; The feature fusion module is used to model the interaction relationships between entities based on spatiotemporal features and utilize a multi-head attention mechanism to obtain spliced ​​and fused features; The destination prediction module is used to predict the possible destination of a target vehicle based on the splicing and fusion features. The safety scoring module is used to predict the future driving trajectories of surrounding vehicles, fuse the future driving trajectories of surrounding vehicles with features of each possible destination, and obtain the safety score of each possible destination. The trajectory planning module is used to identify the final destination from all possible destinations based on the highest safety score, and then plan the driving trajectory of the target vehicle based on the final destination.

[0018] According to another aspect of this disclosure, a computer device is provided, including a memory, a processor, and a computer program stored in the memory, the processor executing the computer program to implement the method of one aspect above.

[0019] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method of one aspect above.

[0020] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of the above-described aspect.

[0021] It should be understood that both the foregoing general description and the following detailed description are exemplary and intended to provide further illustration of the claimed technology. Attached Figure Description

[0022] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 This is a system architecture diagram illustrating an application of an autonomous driving trajectory planning method according to an embodiment of the present disclosure.

[0024] Figure 2 This is a flowchart illustrating the calculation process of a single scaled dot product attention according to an embodiment of this disclosure.

[0025] Figure 3 This is a flowchart illustrating an application of an autonomous driving trajectory planning method according to an embodiment of the present disclosure.

[0026] Figure 4 This is a schematic diagram of the structure of an autonomous driving trajectory planning device according to an embodiment of the present disclosure.

[0027] Figure 5 This is a schematic diagram illustrating the structure of a computer device according to an embodiment of the present disclosure.

[0028] Figure 6 This is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this disclosure more apparent, exemplary embodiments according to this disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this disclosure, and not all embodiments of this disclosure. It should be understood that this disclosure is not limited to the exemplary embodiments described herein.

[0030] With the rapid development of computer-related technologies, autonomous driving systems have been increasingly widely applied and implemented. The goal of an autonomous driving system is to control a vehicle to autonomously perceive complex driving scenarios, plan its trajectory, and safely reach its destination with little or no driver intervention. Safety is the core requirement of autonomous driving systems and a prerequisite for widespread adoption based on user trust. When system safety is ensured, autonomous driving systems can avoid improper driving behaviors by human drivers (such as malicious lane changes, cutting in line), and even speeding and drunk driving, eliminating potential traffic accident hazards and ensuring public traffic safety. Without sufficient safety, autonomous driving systems may make abnormal behavioral decisions, which can directly lead to traffic accidents in severe cases. Therefore, safety is the core evaluation indicator for autonomous driving systems.

[0031] However, the relevant technologies still have the following problems: (1) Trajectory planning based on predefined rules is time-consuming and requires different rules to be designed for different driving scenarios. It lacks universality for complex and ever-changing real-world driving scenarios.

[0032] (2) Only the lane information of the vehicle in the driving scenario is considered, without considering the dynamic information of other vehicles. Ignoring the interaction between dynamic entities will cause the planned trajectory to collide.

[0033] (3) It requires filtering from a large trajectory library, which is computationally expensive. Furthermore, the planning results are constrained by the trajectory library and cannot be fine-tuned according to the actual scenario, which has certain limitations.

[0034] (4) Existing neural network models treat this problem as a regression task and use an error metric as the loss function to train the model, such as The loss mechanism neglects the safety issues of trajectory planning. Furthermore, there is currently a lack of a differentiable loss function to measure the safety of trajectory planning, making it impossible to train the model from a safety perspective.

[0035] The above description, with reference to the accompanying drawings, illustrates an autonomous driving trajectory planning method, apparatus, device, medium, and product according to embodiments of the present disclosure. By modeling the interaction relationships of all entities in a driving scene using attention, it fully extracts information from the driving scene, obtaining a better spatiotemporal feature representation of the driving scene, i.e., splicing and fusing features. It considers the impact of surrounding vehicles on the vehicle itself, as well as the impact between surrounding vehicles, which facilitates real-time trajectory planning and avoids collisions with other vehicles. By predicting the future trajectories of surrounding vehicles, it calculates the safety scores of each possible destination and selects the final destination, ensuring safety while achieving trajectory planning. The future trajectories of surrounding vehicles can be predicted using a multi-head attention mechanism, which accelerates the inference speed of trajectory prediction compared to the single-step inference of traditional recurrent neural networks.

[0036] To facilitate understanding of this embodiment, a detailed description of the autonomous driving trajectory planning method disclosed in this disclosure is provided first. The executing entity of the autonomous driving trajectory planning method provided in this disclosure is generally a computer device with a certain computing capability. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this autonomous driving trajectory planning method can be implemented by a processor calling computer-readable instructions stored in memory.

[0037] like Figure 1 The diagram shown is a system architecture diagram of the autonomous driving trajectory planning method provided in this embodiment of the disclosure, including a perception module 1, a generator 2, and a safety discriminator 3. The following is a detailed description of each module: Perception Module 1: Used to collect the historical driving trajectories of various entities in the driving scenario.

[0038] The entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points.

[0039] Generator 2: Primarily used to generate the final destination and perform trajectory planning based on the final destination, specifically including: Encoder 21: Used to extract time features from historical driving trajectories to form a spatiotemporal feature representation of the driving scene.

[0040] The historical driving trajectory of entity a is a time series composed of a series of coordinate points. Its dimensions are The Long Short-Term Memory (LSTM) network model is used to extract time series data. Time features in The formula is as follows:

[0041] To characterize the spatiotemporal features of the driving scenario, this embodiment constructs an occupied grid map with the current position of the autonomous vehicle equipped with TP-GAN (i.e., the target vehicle) at the planning time as the origin, and its dimension is [dimension missing]. D. Among them, To occupy the horizontal grid cells of the raster map, D represents the number of vertical grid cells occupied in the raster map, where D is the time feature. Dimensions.

[0042] Based on the relative positions of surrounding vehicles and the autonomous vehicle in the driving scenario Through indexing functions Calculate the index grid of each surrounding vehicle in the occupied grid map, and fill the index grid with the temporal features h of each surrounding vehicle to form the spatiotemporal features of the driving scene. The formula is as follows:

[0043] Self-attention social pooling layer 22: Models the interaction relationship between entities using a multi-head attention mechanism, forms an attention weight matrix, and completes the spatiotemporal feature fusion between entities based on the attention weight matrix.

[0044] To learn the interactions between entities, this embodiment designs a self-attention social pooling layer to fuse features between dynamic entities, retaining features of entities with strong interactivity while diluting features of entities with weak interactivity. Previous methods only focus on the interactions between autonomous vehicles and surrounding vehicles, ignoring the interactions that also exist between surrounding vehicles, resulting in insufficient modeling and learning of the driving scene and affecting trajectory planning performance.

[0045] Self-attentional social pooling can model the interactions between all entities in a scene using attention, and autonomously learn attention weights. This is particularly relevant for the spatiotemporal characteristics of driving scenarios. The self-attention social pooling layer uses a multi-head attention mechanism to obtain an attention weight matrix between grids, and performs feature fusion between entities based on this matrix. The weights represent the strength of interactivity. For unoccupied grids, the multi-head attention mechanism tends to output smaller attention weights; for grids occupied by entities, the multi-head attention mechanism autonomously generates attention weights based on its learning of entity interactivity.

[0046] The dimension of the spatiotemporal feature T is D, to reduce computational overhead, the dimensions of the spatiotemporal features are flattened into... D performs multi-head attention computation.

[0047] Multi-head attention mechanisms consist of several scaled dot-product attention mechanisms that operate in parallel, each called a "head" of the multi-head attention mechanism. The parameters of each head are independent of each other, enabling them to extract features from different feature subspaces, which is the greatest advantage of multi-head attention mechanisms.

[0048] Specifically, such as Figure 2 The diagram shown is a flowchart of the computation process for a single scaled dot product attention, including: S201: Through three different linear transformations, spatiotemporal features are transformed. Mapped to the query matrix respectively Key matrix Value matrix .

[0049] The dimensions of the three matrices are all 1. , This represents the feature dimension of the transformed D. Each scaled dot product attention matrix, after a linear transformation, yields a different matrix.

[0050] S202: Calculate the attention weight matrix using query matrix Q and key matrix K. The formula is as follows:

[0051] Among them, the attention weight matrix The dimension is , Let K be the transpose of the key matrix K, and Softmax function represents normalization.

[0052] S203: Combine the attention weight matrix A with the value matrix Multiplying these together yields the output features of the scaled dot product attention, calculated as follows:

[0053] Among them, output features The dimension is .

[0054] S204: Summarize the output features of each scaled dot product attention, and concatenate them to obtain the concatenated and fused feature M, as shown in the following formula:

[0055]

[0056] Where h represents the total number of heads. This represents the attention of the i-th scaled dot product.

[0057] Goal prediction layer 23: Predicts multiple possible destinations and selects the final destination based on the score of the security discriminator.

[0058] In real-world driving, for a given driving scenario, there are typically multiple different driving intentions (such as changing lanes, accelerating, and decelerating). Different driving intentions lead to different driving trajectories. The driver autonomously selects a driving intention and forms a driving trajectory corresponding to that intention. Therefore, this embodiment designs a multimodal destination prediction model, enabling the destination prediction model to correspond to different modal driving trajectories and generate different destinations. The destination prediction model uses a multi-head attention mechanism to capture the temporal features of the autonomous vehicle. The dependency between the splicing and fusion features M is determined; then, a multi-head multilayer perceptron (MLP) is used to generate multiple possible destinations. The formula is as follows:

[0059] in, Indicates the i-th possible destination. For the first MLP functions of various sizes.

[0060] Trajectory completion layer 24: A Long Short-Term Memory (LSTM) network model is used for trajectory completion, targeting each possible destination. Plan the corresponding driving trajectory The formula is as follows:

[0061] Security Discriminator 3: Used to evaluate each possible destination The security of the system specifically includes: Trajectory prediction network 31: Responsible for predicting the future driving trajectories of surrounding vehicles.

[0062] The trajectory prediction network has a similar structure to the generator, using LSTM to encode trajectories and a self-attention social pooling layer to extract the interaction relationships between entities in the driving scene. The difference lies in that, unlike the single-step inference of traditional recurrent neural networks, the output layer of the trajectory prediction network uses a multi-head attention mechanism to directly predict the future driving trajectories of all surrounding vehicles in the driving scene. This helps accelerate the inference speed of the trajectory prediction network and improve the training efficiency of the generator and safety discriminator. During the training of the generator and safety discriminator, the parameters of the trajectory prediction network are locked, preventing it from participating in the training process.

[0063] Graph Attention Layer 32: The safety discriminator needs to combine the future driving trajectories of surrounding vehicles with the possible destinations predicted by the generator to determine whether a possible destination is safe. However, the method of extracting spatiotemporal features of the driving scene by constructing an occupied grid map is too costly, and constructing a grid map based on the predicted future driving trajectory will lead to secondary error accumulation. Therefore, the safety discriminator uses a graph attention layer to extract spatiotemporal features. In this layer, the driving scene is modeled as a strongly connected graph, entities are regarded as nodes in the strongly connected graph, and the interaction relationships between entities are regarded as edges in the strongly connected graph.

[0064] Specifically, it includes: Obtain node i and its neighboring nodes j in the strongly connected graph. Use LSTM to extract future time features of the future driving trajectory. Based on the future time features of node i and its neighboring nodes j, calculate the attention weights between node i and its neighboring nodes j. The formula is as follows:

[0065] Where 'a' represents the shared attention mechanism, Represents the future time characteristics of node i. Representing the future time features of node j, attention weights This indicates the degree to which node i is affected by node j when assessing security.

[0066] In a strongly connected graph, each node is summed based on the attention weights of all its neighbors, and this summation is passed through a linear transformation W and an activation function μ to obtain a node fusion feature that incorporates neighborhood information. The calculation formula is as follows:

[0067] in, This represents the node fusion feature of the i-th node, and the global node fusion feature is... .

[0068] Security score output layer 33: Uses MLP to output a decimal number from 0 to 1 to represent the security score of the scene.

[0069] Specifically, a multi-head attention mechanism is used again. The possible destination Y of the vehicle is used as the query, and features are fused using global nodes. Calculate decision features as keys and values. The formula is as follows:

[0070] The potential destination Y actively queries the entire future driving scenario, focusing on the information most relevant to its own safety, forming the final decision feature C. Finally, a multilayer perceptron is used. Then, the safety score d for each possible destination is obtained by applying the Sigmoid activation function, as shown in the following formula:

[0071] The value of d ranges from [0,1], where 0 is the preset minimum value and 1 is the preset maximum value.

[0072] Finally, the destination with the highest safety score is selected as the final destination, and the corresponding optimal planned trajectory is generated by the trajectory completion layer.

[0073] Adversarial training between the generator and the security discriminator (TP-GAN adversarial network model) includes: S1: Generate the training dataset.

[0074] The training dataset includes the vehicle's real driving purpose and historical trajectory. This embodiment uses the NGSIM and INTERACTION datasets to process driving data. The NGSIM dataset is highway scene driving data collected and processed by a camera at a frequency of 10Hz. The driving data is segmented into 8-second segments, resulting in 739,954 driving data points. The first 3 seconds of data are used as input to the TP-GAN, requiring the generator to plan the driving trajectory for the next 5 seconds. The INTERACTION dataset is urban driving scene driving data collected by a camera at a frequency of 10Hz. It is segmented into 4-second segments, resulting in 518,290 driving data points. The first second of data is used as input to the TP-GAN, requiring the model to plan the driving trajectory for the next 3 seconds.

[0075] S2: Input the historical trajectory into the generator, and the generator outputs the prediction purpose based on the historical trajectory.

[0076] S3: The security discriminator assigns security scores to the actual purpose and the predicted purpose, respectively.

[0077] Specifically, the security discriminator should output a higher security score for the true purpose and a lower security score for the predicted purpose planned by the generator, thereby distinguishing between destinations originating from the training set and those from the generator. For example, the security score for the true purpose (i.e., the first security score) should be close to a preset maximum value of 1, and the security score for the predicted purpose (i.e., the second security score) should be close to a preset minimum value of 0. For the predicted purpose planned by the generator... And the true purpose of the training set Adversarial loss function of security discriminator for:

[0078] S4: Obtain the generator's output that matches the true objective. The attention head with the smallest score difference is used to update its parameters to prevent mode collapse. Then, S1-S3 are repeated for multiple iterations.

[0079] Adversarial training affects the generator's destination prediction. The generator should plan destinations with high safety scores to deceive the safety discriminator. After several iterations, when the safety discriminator can no longer distinguish whether the destination comes from the generator or the training set, i.e., the predicted destination has been determined... When the score is around 0.5, the adversarial network model converges.

[0080] This error is combined with the adversarial loss of the discriminator. The loss function for target prediction in the generator. for:

[0081] The training of trajectory completion uses predicted trajectories. With the actual trajectory Between The loss is as follows:

[0082] In actual testing, compared with traditional methods and existing technologies, the TP-GAN in this embodiment improves the accuracy of trajectory planning and reduces the collision rate. The planning accuracy and collision rate are statistically analyzed using root mean square error and percentage, respectively. The planning accuracy performance results are shown in Table 1: Table 1. Planning accuracy performance results

[0083] The collision rate performance results are shown in Table 2: Table 2 Collision Rate Performance Results

[0084] As can be seen from Tables 1 and 2, compared with other methods, TP-GAN has the lowest root mean square error in trajectory planning and the lowest collision rate, which demonstrates the effectiveness of the TP-GAN trajectory planning system.

[0085] Based on the above embodiments, this embodiment also provides an autonomous driving trajectory planning method, such as... Figure 3 The diagram shows a flowchart of an autonomous driving trajectory planning method, including S301-S305: S301: Obtain the historical driving trajectory of each entity in the preset driving scenario, and calculate the spatiotemporal characteristics of the driving scenario based on the historical driving trajectory.

[0086] The entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points.

[0087] S302: Based on spatiotemporal features, a multi-head attention mechanism is used to model the interaction relationships between entities to obtain splicing and fusion features.

[0088] S303: Based on stitching and fusion features, predict the possible destination of the target vehicle.

[0089] S304: Predict the future driving trajectories of surrounding vehicles, fuse the future driving trajectories of surrounding vehicles with features of each possible destination, and obtain a safety score for each possible destination.

[0090] S305: Based on the highest safety score, identify the final destination from all possible destinations, and plan the driving trajectory of the target vehicle based on the final destination.

[0091] In one or more embodiments, S301 includes: Based on historical driving trajectories, obtain time features in the time series; Construct an occupied grid map of the driving scene with the current position of the target vehicle as the origin; Based on the relative positions of the surrounding vehicles and the target vehicle, obtain the index grid of each surrounding vehicle in the occupied grid map; The temporal characteristics of the surrounding vehicles are filled into the index raster to obtain the spatiotemporal characteristics of the driving scene.

[0092] In one or more embodiments, the multi-head attention mechanism includes multiple scaled dot product attention operations working in parallel, and S302 includes: Different linear transformations of the spatiotemporal features are applied using scaling dot product attention to obtain the query matrix, key matrix, and value matrix; wherein the feature dimensions of the query matrix, key matrix, and value matrix are the same. Obtain the transpose of the key matrix, and calculate the dot product of the query matrix and the transpose matrix; The attention weight matrix is ​​obtained by scaling the dot product by dividing it by the feature dimension and then normalizing it. Based on the attention weight matrix and value matrix, the output features of each scaled dot product attention are obtained; The output features of each scaled dot product attention are concatenated to obtain the concatenated and fused features.

[0093] In one or more embodiments, S303 includes: Based on the target vehicle's historical driving trajectory, obtain the target vehicle's temporal characteristics; The splicing and fusion features and the temporal features of the target vehicle are input into a pre-defined multilayer perceptron to predict the possible destination of the target vehicle.

[0094] In one or more embodiments, S304 includes: The driving scenario is modeled as a strongly connected graph, where nodes represent entities and edges represent the interaction relationships between entities. Obtain node i and its neighboring node j in the strongly connected graph. Calculate the attention weight between node i and its neighboring node j based on the future time features of node i and its neighboring node j. The future time features are obtained based on the corresponding future driving trajectory, and the attention weight represents the degree to which node i is influenced by its neighboring node j. Based on the attention weights between node i and each of its neighboring nodes, the node fusion features of each node are obtained. Using each possible destination as the query and the node fusion feature as the key and value, multi-head attention calculation is performed to obtain decision features; The decision features are input into a pre-defined multilayer perceptron to obtain a security score for each possible destination.

[0095] In one or more embodiments, the adversarial training process between the security discriminator and the generator includes: The training dataset obtained from real driving scenarios is input into the safety discriminator to generate the real purpose; the training dataset includes the vehicle's real driving purpose and historical trajectory; Input historical trajectories into the generator to generate predictions based on those trajectories. The security discriminator assigns a first security score to the true purpose and a second security score to the predicted purpose; wherein the first security score is greater than the second security score. Based on the second safety score, the generator's own parameters are updated, and the steps for generating predictions based on historical trajectories are repeated until the difference between the first and second safety scores meets the preset conditions, thus obtaining a converged safety discriminator and generator.

[0096] According to another aspect of the embodiments of this disclosure, an autonomous driving trajectory planning device is provided, such as... Figure 4 As shown, the device includes: The feature calculation module 401 is used to obtain the historical driving trajectory of each entity in the preset driving scenario, and calculate the spatiotemporal features of the driving scenario based on the historical driving trajectory; wherein, the entity includes the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points. The feature fusion module 402 is used to model the interaction relationship between entities based on spatiotemporal features and using a multi-head attention mechanism to obtain spliced ​​and fused features; The destination prediction module 403 is used to predict the possible destination of the target vehicle based on the splicing and fusion features. The safety scoring module 404 is used to predict the future driving trajectories of surrounding vehicles, fuse the future driving trajectories of surrounding vehicles with features of each possible destination, and obtain the safety score of each possible destination. The trajectory planning module 405 is used to identify the final destination from all possible destinations based on the maximum safety score, and to plan the driving trajectory of the target vehicle based on the final destination.

[0097] In one or more embodiments, the feature calculation module 401 is used for: Based on historical driving trajectories, obtain time features in the time series; Construct an occupied grid map of the driving scene with the current position of the target vehicle as the origin; Based on the relative positions of the surrounding vehicles and the target vehicle, obtain the index grid of each surrounding vehicle in the occupied grid map; The temporal characteristics of the surrounding vehicles are filled into the index raster to obtain the spatiotemporal characteristics of the driving scene.

[0098] In one or more embodiments, the beneficial effect of this solution is that by constructing an occupied grid map to model the driving scene, it is beneficial to obtain the spatiotemporal features of the complete scene that integrates spatial location and temporal features.

[0099] In one or more embodiments, the feature fusion module 402 is used to: Different linear transformations of the spatiotemporal features are applied using scaling dot product attention to obtain the query matrix, key matrix, and value matrix; wherein the feature dimensions of the query matrix, key matrix, and value matrix are the same. Obtain the transpose of the key matrix, and calculate the dot product of the query matrix and the transpose matrix; The attention weight matrix is ​​obtained by scaling the dot product by dividing it by the feature dimension and then normalizing it. Based on the attention weight matrix and value matrix, the output features of each scaled dot product attention are obtained; The output features of each scaled dot product attention are concatenated to obtain the concatenated and fused features.

[0100] In one or more embodiments, the target prediction module 403 is used to: Based on the target vehicle's historical driving trajectory, obtain the target vehicle's temporal characteristics; The splicing and fusion features and the temporal features of the target vehicle are input into a pre-defined multilayer perceptron to predict the possible destination of the target vehicle.

[0101] In one or more embodiments, the security scoring module 404 is used to: The driving scenario is modeled as a strongly connected graph, where nodes represent entities and edges represent the interaction relationships between entities. Obtain node i and its neighboring node j in the strongly connected graph. Calculate the attention weight between node i and its neighboring node j based on the future time features of node i and its neighboring node j. The future time features are obtained based on the corresponding future driving trajectory, and the attention weight represents the degree to which node i is influenced by its neighboring node j. Based on the attention weights between node i and each of its neighboring nodes, the node fusion features of each node are obtained. Using each possible destination as the query and the node fusion feature as the key and value, multi-head attention calculation is performed to obtain decision features; The decision features are input into a pre-defined multilayer perceptron to obtain a security score for each possible destination.

[0102] The autonomous driving trajectory planning device and the autonomous driving trajectory planning method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0103] This disclosure also provides a computer device for executing the above-described autonomous driving trajectory planning method. Please refer to... Figure 5 It illustrates a schematic diagram of a computer device provided by some embodiments of this disclosure. For example... Figure 5 As shown, the computer device 5 includes: a processor 500, a memory 501, a bus 502, and a communication interface 503. The processor 500, the communication interface 503, and the memory 501 are connected via the bus 502. The memory 501 stores a computer program that can run on the processor 500. When the processor 500 runs the computer program, it executes the autonomous driving trajectory planning method provided in any of the foregoing embodiments of this disclosure.

[0104] The memory 501 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 503 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0105] Bus 502 can be an ISA bus, PCI bus, or EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Memory 501 is used to store programs. After receiving an execution instruction, the processor 500 executes the program. The autonomous driving trajectory planning method disclosed in any of the foregoing embodiments of this disclosure can be applied to the processor 500, or implemented by the processor 500.

[0106] The processor 500 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 500 or by instructions in software form. The processor 500 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPTA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this disclosure. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this disclosure can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 501. The processor 500 reads the information in memory 501 and, in conjunction with its hardware, completes the steps of the above method.

[0107] The computer device provided in this disclosure and the autonomous driving trajectory planning method provided in this disclosure are based on the same inventive concept and have the same beneficial effects as the methods they adopt, operate or implement.

[0108] This disclosure also provides a computer-readable storage medium corresponding to the autonomous driving trajectory planning method provided in the foregoing embodiments. The computer-readable storage medium is an optical disc, on which a computer program (i.e., a computer program product) is stored. When the computer program is run by a processor, it executes the autonomous driving trajectory planning method provided in any of the foregoing embodiments.

[0109] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here.

[0110] The computer-readable storage medium provided in the above embodiments of this disclosure and the autonomous driving trajectory planning method provided in the embodiments of this disclosure are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the applications stored therein.

[0111] This disclosure also provides a computer program product; please refer to [reference needed]. Figure 6 The computer program product 600 carries program code, namely computer program 601. The instructions included in the computer program 601 can be used to execute the steps of the autonomous driving trajectory planning method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0112] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0113] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0114] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0115] Additionally, as used herein, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not imply that the described example is preferred or better than other examples.

[0116] It should also be noted that in the systems and methods of this disclosure, the components or steps can be decomposed and / or recombined. These decompositions and / or recombinations should be considered as equivalent solutions to this disclosure.

[0117] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of this disclosure is not limited to the specific aspects of the processes, machines, manufactures, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufactures, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufactures, events, means, methods, or actions within their scope.

[0118] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0119] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. An autonomous driving trajectory planning method, characterized in that, include: The historical driving trajectories of each entity in a preset driving scenario are obtained, and the spatiotemporal characteristics of the driving scenario are calculated based on the historical driving trajectories; wherein, the entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points; Based on the aforementioned spatiotemporal features, a multi-head attention mechanism is used to model the interaction relationships between the entities to obtain splicing and fusion features. Based on the stitching and fusion features, the possible destination of the target vehicle is predicted; Predict the future driving trajectory of each of the surrounding vehicles, fuse the future driving trajectory of each of the surrounding vehicles with the features of each of the possible destinations, and obtain the safety score of each of the possible destinations; Based on the highest safety score, the final destination is determined from each of the possible destinations, and the driving trajectory of the target vehicle is planned based on the final destination.

2. The autonomous driving trajectory planning method as described in claim 1, characterized in that, Based on the historical driving trajectory, the spatiotemporal features of the driving scenario are obtained, including: Based on the historical driving trajectory, the time features in the time series are obtained; Using the current position of the target vehicle as the origin, construct an occupied grid map of the driving scene; Based on the relative positions of each of the surrounding vehicles and the target vehicle, obtain the index grid of each of the surrounding vehicles in the occupied grid map; The temporal characteristics of each of the surrounding vehicles are filled into the index grid to obtain the spatiotemporal characteristics of the driving scene.

3. The autonomous driving trajectory planning method as described in claim 1, characterized in that, The multi-head attention mechanism comprises multiple scaled dot product attention operations working in parallel; Based on the aforementioned spatiotemporal features, a multi-head attention mechanism is used to model the interaction relationships between the entities, obtaining splicing and fusion features, including: Different linear transformations are applied to the spatiotemporal features using the scaling dot product attention methods described above to obtain a query matrix, a key matrix, and a value matrix; wherein the query matrix, key matrix, and value matrix have the same feature dimension. Obtain the transpose of the key matrix, and calculate the dot product of the query matrix and the transpose matrix; The dot product is scaled by the feature dimension and then normalized to obtain the attention weight matrix. Based on the attention weight matrix and the value matrix, the output features of each scaled dot product attention are obtained; The output features of each scaling dot product attention are concatenated to obtain concatenated and fused features.

4. The autonomous driving trajectory planning method as described in claim 1, characterized in that, Based on the stitching and fusion features, the possible destination of the target vehicle is predicted, including: Based on the historical driving trajectory of the target vehicle, the temporal characteristics of the target vehicle are obtained; The stitching and fusion features and the temporal features of the target vehicle are input into a preset multilayer perceptron to predict the possible destination of the target vehicle.

5. The autonomous driving trajectory planning method as described in claim 1, characterized in that, The future driving trajectories of the surrounding vehicles are fused with the features of the possible destinations to obtain a safety score for each possible destination, including: The driving scenario is modeled as a strongly connected graph, where the nodes of the strongly connected graph represent entities, and the edges of the strongly connected graph represent the interaction relationships between entities. Obtain node i and its neighboring node j in the strongly connected graph. Calculate the attention weight between node i and its neighboring node j based on the future time features of node i and its neighboring node j. The future time features are obtained based on the corresponding future driving trajectory, and the attention weight represents the degree to which node i is influenced by its neighboring node j. Based on the attention weights between node i and each of its neighboring nodes, the node fusion features of each node are obtained; Using each of the possible destinations as queries and the node fusion features as keys and values, multi-head attention calculation is performed to obtain decision features; The decision features are input into a preset multilayer perceptron to obtain a security score for each of the possible destinations.

6. The autonomous driving trajectory planning method as described in claim 1, characterized in that, The possible destinations are generated by a generator, the security score is generated by a security discriminator, and the adversarial training process between the security discriminator and the generator includes: The training dataset obtained from real driving scenarios is input into the safety discriminator to generate the real purpose; the training dataset includes the vehicle's real driving purpose and historical trajectory; The historical trajectory is input into the generator, and a prediction purpose is generated based on the historical trajectory; The security discriminator assigns a first security score to the true purpose and a second security score to the predicted purpose; wherein the first security score is greater than the second security score. Based on the second safety score, the generator's own parameters are updated, and the steps for generating predictions based on the historical trajectory are repeated until the difference between the first safety score and the second safety score meets a preset condition, thereby obtaining a converged safety discriminator and the generator.

7. An autonomous driving trajectory planning device, characterized in that, include: The feature calculation module is used to obtain the historical driving trajectories of each entity in a preset driving scenario, and calculate the spatiotemporal features of the driving scenario based on the historical driving trajectories; wherein, the entities include the target vehicle and surrounding vehicles, and the historical driving trajectory is a time series composed of multiple coordinate points; The feature fusion module is used to model the interaction relationship between the entities based on the spatiotemporal features using a multi-head attention mechanism to obtain spliced ​​and fused features; The destination prediction module is used to predict the possible destination of the target vehicle based on the splicing and fusion features. The safety scoring module is used to predict the future driving trajectory of each of the surrounding vehicles, and to fuse the future driving trajectory of each of the surrounding vehicles with the features of each of the possible destinations to obtain the safety score of each of the possible destinations. The trajectory planning module is used to identify the final destination from each of the possible destinations based on the maximum safety score, and to plan the driving trajectory of the target vehicle based on the final destination.

8. A computer embedded device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 6.

Images