Track generation method and system for autonomous vehicle and storage medium

Abstract

The invention relates to the technical field of automatic driving, and discloses a track generation method and system of an automatic driving vehicle and a storage medium. The track generation method of the autonomous vehicle provided by the invention comprises the following steps: determining an own vehicle driving intention of the vehicle based on a historical frame image sequence acquired from the vehicle; predicting a vehicle direction angle sequence and a vehicle speed based on a pre-configured initial direction angle sequence and a vehicle driving intention; and determining a driving track of the vehicle at the future time based on the predicted vehicle direction angle sequence and the predicted vehicle speed. According to the scheme, the application capability of the automatic driving vehicle in different scenes is improved, and the stability and precision of vehicle driving track generation are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a method, system, and storage medium for generating the trajectory of an autonomous vehicle. Background Technology

[0002] With the rapid development of autonomous driving technology, the trajectory generation of autonomous vehicles has become an important factor affecting driving safety.

[0003] Current mainstream trajectory generation methods are typically based on diffusion models. When generating autonomous driving trajectories using diffusion models, fixed-scale anchors (predefined boxes or bounding boxes used to define the position and size of a target) are often used to guide trajectory generation. However, in different driving scenarios such as highways and city streets, the significant differences in vehicle speed lead to inconsistent spatial distribution densities of anchors, resulting in lower accuracy of the generated vehicle trajectories. Summary of the Invention

[0004] In view of this, the present invention proposes a trajectory generation method, system and storage medium for autonomous vehicles, which solves the technical problem of low trajectory prediction accuracy in traditional trajectory generation methods based on diffusion models.

[0005] On one hand, embodiments of the present invention provide a trajectory generation method for an autonomous vehicle, the trajectory generation method for an autonomous vehicle comprising: Based on the historical frame image sequence obtained from the vehicle, the autonomous driving intention of the vehicle is determined; Based on the pre-configured initial direction angle sequence and the vehicle's driving intention, predict the vehicle's direction angle sequence and vehicle speed; Based on the predicted vehicle heading angle sequence and the predicted vehicle speed, the vehicle's trajectory in the future is determined.

[0006] In some implementations, predicting the vehicle's heading angle sequence and vehicle speed based on a pre-configured initial heading angle sequence and the vehicle's driving intention includes: The initial direction angle sequence and the vehicle's driving intention are input into a pre-trained vehicle direction angle trajectory prediction model to obtain the predicted vehicle direction angle sequence. The driver's intention is input into a pre-trained vehicle speed prediction model to obtain the predicted vehicle speed.

[0007] In some implementations, the method further includes: Based on the initial direction angle sequence and the vehicle's driving intention, the diffusion model is trained to obtain a trained vehicle direction angle trajectory prediction model.

[0008] In some implementations, training the diffusion model based on the initial direction angle sequence and the vehicle's driving intention includes: Based on the vehicle's actual trajectory, determine the vehicle's actual direction angle sequence; The diffusion model is trained based on the vehicle's actual direction angle sequence, the initialized direction angle sequence, and the vehicle's driving intention.

[0009] In some implementations, the method further includes: The fully connected layer is trained based on the vehicle's driving intention to obtain a trained vehicle speed prediction model.

[0010] In some implementations, training the fully connected layer based on the vehicle's driving intention includes: Determine the vehicle's actual speed based on its actual trajectory; The fully connected layer is trained based on the vehicle's actual speed and the driver's intention.

[0011] In some implementations, determining the vehicle's autonomous driving intention based on a sequence of historical frame images acquired from the vehicle includes: Based on the historical frame image sequence obtained from the vehicle and the large language model, the autonomous driving intention of the vehicle is determined.

[0012] In some implementations, determining the vehicle's autonomous driving intention based on historical frame image sequences acquired from the vehicle and a large language model includes: Based on the historical frame image sequence collected from the vehicle, the autonomous vehicle interaction features are determined; The autonomous vehicle interaction features are mapped to the semantic space of the large language model to obtain autonomous vehicle text features; The vehicle text features and the navigation text features determined according to the navigation instructions are input into the large language model to obtain multimodal features containing the vehicle's driving intentions.

[0013] On the other hand, embodiments of the present invention also provide a trajectory generation system for an autonomous vehicle, the trajectory generation system for an autonomous vehicle including at least one processor and a memory, the memory storing a computer program that can run on the processor, the processor executing the steps of the method as described in any of the above embodiments when executing the program.

[0014] On the other hand, embodiments of the present invention also provide a computer storage medium, which stores a computer program that, when executed by a processor, performs the steps of the method described in any of the above embodiments.

[0015] The present invention has at least the following beneficial effects: This invention provides a trajectory generation method, system, and storage medium for autonomous vehicles. The trajectory generation method for autonomous vehicles includes: determining the vehicle's driving intention based on a sequence of historical frame images acquired from the vehicle; predicting a vehicle direction angle sequence and vehicle speed based on a pre-configured initial direction angle sequence and the driving intention; and determining the vehicle's future trajectory based on the predicted vehicle direction angle sequence and predicted vehicle speed. This invention determines the vehicle's driving intention and the matching vehicle direction angle sequence and speed through the above scheme, and further uses the predicted vehicle speed to reconstruct the predicted vehicle direction angle sequence into a future trajectory with an adaptive scale based on vehicle speed, thereby improving the vehicle's applicability in different scenarios and enhancing the stability and accuracy of the generated vehicle trajectory. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating a trajectory generation method for an autonomous vehicle provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the initialization of the direction angle sequence in the trajectory generation method for an autonomous vehicle provided in an embodiment of the present invention; Figure 3 A flowchart of a method for determining the autonomous driving intention of a vehicle, which is part of the trajectory generation method for autonomous vehicles provided in an embodiment of the present invention; Figure 4 A flowchart of a method for determining autonomous vehicle interaction features in the trajectory generation method for autonomous vehicles provided in an embodiment of the present invention; Figure 5 A flowchart of a method for predicting vehicle direction angle sequence and vehicle speed in the trajectory generation method for autonomous vehicles provided in an embodiment of the present invention; Figure 6 A flowchart of another method for predicting vehicle direction angle sequence and vehicle speed in the trajectory generation method for autonomous vehicles provided in this embodiment of the invention; Figure 7 A flowchart of another method for predicting vehicle direction angle sequence and vehicle speed in the trajectory generation method for autonomous vehicles provided in this embodiment of the invention; Figure 8A schematic diagram illustrating a trajectory generation method for an autonomous vehicle provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a trajectory generation system for an autonomous vehicle provided in an embodiment of the present invention; Figure 10 A schematic diagram of the structure of a computer storage medium provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to specific examples and the accompanying drawings.

[0019] It should be noted that all uses of "first" and "second" in the embodiments of the present invention are for the purpose of distinguishing two entities or parameters with the same name but different names. It is clear that "first" and "second" are only for the convenience of expression and should not be construed as limiting the embodiments of the present invention. Subsequent embodiments will not explain this in detail.

[0020] The present invention will now be described in detail with reference to the embodiments and accompanying drawings.

[0021] The first aspect of this invention provides a trajectory generation method for autonomous vehicles, such as... Figure 1 As shown, the trajectory generation method for the autonomous vehicle specifically includes steps S100 to S120.

[0022] S100. Based on the historical frame image sequence obtained from the vehicle, determine the vehicle's autonomous driving intention.

[0023] The historical frame image sequence can be images acquired by the vehicle before the current moment, including information about the vehicle's driving environment. In some examples, to improve the accuracy of subsequent trajectory generation, the number of frames in the historical frame image can be multiple, such as 3, 5, 10, 15, 20, or 30 frames, etc., without specific limitations. In some examples, to further improve the accuracy of subsequent trajectory generation, multi-view images from the vehicle can be acquired from vision sensors installed at different locations on the vehicle.

[0024] The vehicle's autonomous driving intention refers to the macroscopic driving target that the vehicle selects from a limited number of maneuver categories such as going straight, turning left, turning right, and changing lanes before the next road decision point (such as an intersection or lane change), in accordance with regulations. This target is related to the subsequent predicted future continuous trajectory in a category-instance relationship of "first determining the direction, then calculating the path".

[0025] In this embodiment of the invention, the autonomous driving intention of the vehicle can be determined based on the historical frame image sequence obtained from the vehicle.

[0026] S110: Based on the pre-configured initial direction angle sequence and the vehicle's driving intention, predict the vehicle's direction angle sequence and vehicle speed.

[0027] The initialization direction angle sequence is a unit-length direction vector, used to represent the vehicle trajectory sequence based on the direction angles. In the initialization direction angle sequence, the distance between any two adjacent vehicle trajectory points can be a unit length, which can be 1, 2, or 3, etc., without specific limitations.

[0028] By representing future trajectories as a sequence of orientation angles, coordinate scale errors caused by speed variations can be eliminated, thereby improving the vehicle's applicability in different scenarios and increasing the accuracy of vehicle trajectory generation.

[0029] In some examples, to improve the matching degree between the generated future trajectory and the vehicle's actual trajectory, there can be multiple initialization direction angle sequences. Figure 2 In the schematic diagram shown, 11 initial direction angle sequences are predefined. Each initial direction angle sequence varies with the vehicle's forward angle from -120° to 120°, thereby further improving the accuracy of vehicle trajectory generation.

[0030] In this embodiment of the invention, the vehicle's direction angle sequence and vehicle speed can be predicted by combining the vehicle's driving intention and the initial direction angle sequence, so that the predicted vehicle direction angle sequence and vehicle speed match the vehicle's driving intention.

[0031] S120. Based on the predicted vehicle heading angle sequence and the predicted vehicle speed, determine the vehicle's trajectory in the future.

[0032] In this embodiment of the invention, the predicted vehicle direction angle sequence can be restored to a future driving trajectory based on the predicted vehicle speed and an adaptive scale according to the vehicle speed. This improves the applicability of vehicle driving in different scenarios and enhances the stability and accuracy of the generated future vehicle driving trajectory.

[0033] In this embodiment of the invention, the above-described scheme can determine the driving intention of the vehicle and the vehicle direction angle sequence and vehicle speed that match the driving intention. Furthermore, the predicted vehicle direction angle sequence can be restored to a future driving trajectory based on the vehicle speed adaptive scale using the predicted vehicle speed, thereby improving the applicability of vehicle driving in different scenarios and improving the stability and accuracy of the generated vehicle driving trajectory.

[0034] In some embodiments of the present invention, determining the vehicle's autonomous driving intention based on the historical frame image sequence obtained from the vehicle in step S100 may include: determining the vehicle's autonomous driving intention based on the historical frame image sequence obtained from the vehicle and a large language model.

[0035] In embodiments of the present invention, such as Figure 3 As shown, the vehicle's autonomous driving intention can be determined based on steps S101 to S103.

[0036] S101. Based on the historical frame image sequence collected from the vehicle, determine the vehicle interaction features.

[0037] S102. Map the autonomous vehicle interaction features to the semantic space of the large language model to obtain the autonomous vehicle text features.

[0038] S103. Input the vehicle text features and the navigation text features determined according to the navigation instructions into the large language model to obtain multimodal features containing the vehicle's driving intentions.

[0039] In step S101, the vehicle interaction features are used to represent its own state. Each frame in the historical frame image sequence can be converted into a BEV (bird's-eye view) feature map, and the vehicle interaction features can be extracted from the BEV feature map.

[0040] In this embodiment of the invention, the autonomous vehicle interaction features can also integrate environmental elements such as the autonomous vehicle and surrounding traffic participants, lane lines, lane markings, and traffic lights, so that in the subsequent vehicle trajectory generation process, violations of traffic rules and collision risks can be avoided, thereby improving the safety of autonomous vehicle driving.

[0041] In step S102, the vehicle interaction features can be mapped to the semantic space of the Large Language Model (LLM) through linear mapping, thereby obtaining the target text features.

[0042] In step S103, navigation instructions can be sent to a text tokenizer to obtain navigation text features. Then, the vehicle text features and navigation text features are concatenated, and the concatenated features are input into a large language model to obtain multimodal features containing the vehicle's driving intentions.

[0043] In this embodiment of the invention, the vehicle interaction features are determined based on the historical frame image sequence collected from the vehicle; the vehicle interaction features are mapped to the semantic space of the large language model to obtain the vehicle text features; the vehicle text features and the navigation text features determined according to the navigation instructions are input into the large language model to obtain multimodal features containing the vehicle's driving intentions, thereby realizing the potential association between the vehicle's current semantic target and the driving environment. This can improve the accuracy of the generated trajectory when generating the vehicle's future driving trajectory based on the vehicle's driving intentions.

[0044] In some embodiments of the present invention, such as Figure 4As shown, in the trajectory generation method for autonomous vehicles provided in this embodiment of the invention, the autonomous vehicle interaction features can be determined based on steps S1011 to S1013.

[0045] S1011. Based on the pre-configured target query parameters, extract the target element features from each frame of the historical frame image sequence.

[0046] Specifically, the target query parameters correspond one-to-one with the target element features. The target element features can be understood as features related to the vehicle's driving, which may include one or more of the following: vehicle features (i.e., the vehicle itself), features of traffic participants around the vehicle (such as other vehicles, obstacles, etc.), traffic light features, traffic sign features (such as warning signs, prohibitory signs, instruction signs, directional signs, etc.), and lane line features.

[0047] First, each frame in the historical frame image sequence can be converted into a BEV (bird's-eye view) feature map. Then, using a cross-attention mechanism, based on multiple pre-configured target query parameters, the target element features can be extracted from each frame in the historical frame image sequence.

[0048] In this embodiment of the invention, there are various ways to convert images acquired by the vehicle into BEV feature maps. For example, BEV Former (a vision-based 3D object detection algorithm) can be used as the base network to convert each frame in the historical frame image sequence into a BEV feature map Ft. For example, if each frame in the historical frame image sequence contains multiple viewpoint images, the multi-viewpoint images can also be converted into corresponding BEV feature maps based on the Lift-Splat operation. Here, Lift is used to convert image features from multiple viewpoints at the same time into 3D image features, and the 3D image features from multiple viewpoints are stitched together to obtain a unified representation of the entire scene. Splat is used to project the unified representation onto the BEV mesh to obtain the BEV feature map.

[0049] In this embodiment of the invention, the target query parameter may include a learnable query, which is any one or more of the following: learnable query for traffic participants (vehicles), learnable query for traffic signs, learnable query for traffic lights, and learnable query for lane lines.

[0050] In some examples, the target query parameters may include four sets of learnable queries: learnable queries for traffic participants-vehicles, learnable queries for traffic signs, learnable queries for traffic lights, and learnable queries for lane lines. Then, for the BEV feature map corresponding to each frame in the historical frame image sequence, a cross-attention mechanism can be used, and the corresponding target element features can be extracted from the above four sets of learnable queries.

[0051] S1012. The target element features extracted from the same frame image are spliced ​​and fused to obtain the fused features.

[0052] Specifically, the target element features corresponding to the same target query parameters extracted from the same frame image are spliced ​​in time sequence, and the spliced ​​features are fused based on the self-attention mechanism to obtain the fused features. This establishes the relationship between vehicle motion changes and spatiotemporal dynamics, thereby improving the accuracy of subsequent driving trajectory prediction.

[0053] S1013. The fused features corresponding to different target element features are spliced ​​and fused to obtain the vehicle interaction features.

[0054] Specifically, the fused features corresponding to different types of target element features can be concatenated to obtain concatenated fused features. Based on the self-attention mechanism, the concatenated fused features are then fused to obtain autonomous vehicle interaction features. This establishes the relationship between the autonomous vehicle and the driving environment (such as traffic participants, lane markings, lane lines, etc.), so that the autonomous vehicle interaction features incorporate information from various target query parameters. This enables the subsequent vehicle trajectory generation process to avoid violating traffic rules and the risk of collision, thereby improving the safety of autonomous vehicle driving.

[0055] The embodiments of the present invention improve the accuracy of generating the driving trajectory of autonomous vehicles through the above-described solution, avoid the risk of autonomous vehicles violating traffic rules and causing collisions, and improve driving safety.

[0056] In some embodiments of the present invention, such as Figure 5 As shown, step S110, which predicts the vehicle's direction angle sequence and speed based on the pre-configured initial direction angle sequence and the vehicle's driving intention, may include steps S111 to S112.

[0057] S111. Input the initial direction angle sequence and the vehicle's driving intention into the pre-trained vehicle direction angle trajectory prediction model to obtain the predicted vehicle direction angle sequence.

[0058] In this embodiment of the invention, the pre-trained vehicle heading angle trajectory prediction model is determined based on a diffusion model. By inputting the autonomous vehicle's driving intention and the initialized heading angle sequence into the vehicle heading angle trajectory prediction model, the heading angle sequence of the vehicle over a future period can be predicted, thereby representing the vehicle's trajectory over that period. The vehicle trajectory obtained here is not the final trajectory used to guide the autonomous vehicle's driving; the subsequent trajectory used to guide the autonomous vehicle's driving is generated based on the vehicle trajectory obtained here combined with the predicted vehicle speed.

[0059] S112. Input the driver's intention into the pre-trained vehicle speed prediction model to obtain the predicted vehicle speed.

[0060] In this embodiment of the invention, the pre-trained vehicle speed prediction model is determined based on a fully connected layer. By inputting the driver's intention into the vehicle speed prediction model, the vehicle's speed over a future period can be predicted.

[0061] The embodiments of the present invention improve the accuracy and stability of subsequent vehicle trajectory generation through the above-described scheme.

[0062] In some embodiments of the present invention, such as Figure 6 As shown, in the trajectory generation method for autonomous vehicles provided in this embodiment of the invention, the specific prediction process of vehicle speed and vehicle direction angle sequence includes steps S1101 to S1104.

[0063] S1101. Based on the initial direction angle sequence and the vehicle's driving intention, the diffusion model is trained to obtain a trained vehicle direction angle trajectory prediction model.

[0064] S1102. Train the fully connected layer based on the driver's intention to obtain a trained vehicle speed prediction model.

[0065] S1103. Input the initial direction angle sequence and the vehicle's driving intention into the pre-trained vehicle direction angle trajectory prediction model to obtain the predicted vehicle direction angle sequence.

[0066] S1104. Input the driver's intention into the pre-trained vehicle speed prediction model to obtain the predicted vehicle speed.

[0067] In step S1101, noise can be added to the initial direction angle sequence to obtain a noisy direction angle sequence. Then, based on the noisy direction angle sequence and the vehicle's driving intention, the diffusion model is trained to obtain a trained vehicle direction angle trajectory prediction model.

[0068] The noise added to the initial direction angle sequence can be Gaussian noise, but is not limited to it; it can also be other types of noise.

[0069] In step S1102, the fully connected layer (MLP layer for short) may include a linear transformation matrix. At the start of training, the linear transformation matrix is ​​first initialized (in the MLP layer, the linear transformation matrix is ​​used to map the input vector to the output space through matrix multiplication; each element in the matrix represents the degree of influence of the input feature on the corresponding output feature; the matrix dimension is the output feature dimension × the input feature dimension). Then, the vehicle's driving intention is input into the initialized linear transformation matrix to train it until the training result meets preset conditions. Training then stops, and the linear transformation matrix obtained at this point is determined as the vehicle speed prediction model.

[0070] In this embodiment of the invention, the specific implementation of step S1103 can be the same as that of step S111, and the specific implementation of step S1104 can be the same as that of step S112. Therefore, the specific implementation of steps S1103 and S1104 will not be described in detail here.

[0071] The embodiments of the present invention determine the vehicle direction angle trajectory prediction model and the vehicle speed prediction model through the above scheme. By combining the vehicle direction angle trajectory prediction model and the vehicle speed prediction model, and based on the initial direction angle sequence and the driving intention of the vehicle, the vehicle direction angle sequence and vehicle speed in the future time can be accurately predicted, thereby improving the accuracy of vehicle trajectory generation.

[0072] In some embodiments of the present invention, such as Figure 7 As shown, in the trajectory generation method for autonomous vehicles provided in this embodiment of the invention, the specific prediction process of vehicle speed and vehicle direction angle sequence includes steps S700 to S704.

[0073] S700: Based on the vehicle's actual trajectory, determine the vehicle's actual direction angle sequence and actual speed.

[0074] S701. Based on the vehicle's real direction angle sequence, the initial direction angle sequence, and the vehicle's driving intention, the diffusion model is trained to obtain a trained vehicle direction angle trajectory prediction model.

[0075] S702. The fully connected layer is trained based on the vehicle's actual speed and the driver's intention to drive, resulting in a trained vehicle speed prediction model.

[0076] S703. Input the initial direction angle sequence and the vehicle's driving intention into the pre-trained vehicle direction angle trajectory prediction model to obtain the predicted vehicle direction angle sequence.

[0077] S704. Input the driver's intention into the pre-trained vehicle speed prediction model to obtain the predicted vehicle speed.

[0078] In step S700, there are several ways to obtain the vehicle's true trajectory. For example, the vehicle's true trajectory can be obtained based on the vehicle's historical driving trajectory. Alternatively, the vehicle can be driven along a pre-set route on a real road to obtain the vehicle's true trajectory. After obtaining the vehicle's true trajectory, it is normalized to obtain the vehicle's true direction angle sequence, and the true speed of each trajectory point in the vehicle's true direction angle sequence is calculated.

[0079] In step S701, assuming there are multiple initialization direction angle sequences, Gaussian noise can be added to each initialization direction angle sequence to obtain a noisy direction angle sequence. Then, the noisy direction angle sequence is compared with the vehicle's driving intention... Figure 1 The data is then input into the diffusion model to predict the score of the noisy orientation angle sequence and the difference between the noisy orientation angle sequence and the true orientation angle sequence. The loss is then calculated based on the difference between the highest-scoring orientation angle sequence and the true orientation angle sequence, and the parameters of the diffusion model are modified based on the loss calculation results. Finally, the above process is repeated until the set number of training iterations is reached. The parameters of the diffusion model are then determined based on the current model parameters, resulting in a trained diffusion model (i.e., a vehicle orientation angle trajectory prediction model).

[0080] In step S702, the driving intention of the vehicle is first input into the MLP layer to predict the vehicle speed. Then, based on the loss function, the deviation between the predicted vehicle speed and the actual vehicle speed is calculated, and the parameters of the MLP layer are modified based on the deviation. Finally, the above process is repeated until the number of training times reaches the set value, and the training ends. Based on the parameters of the MLP layer during the last round of training, the trained MLP layer (i.e., the vehicle speed prediction model) is obtained.

[0081] In this embodiment of the invention, the specific implementation of step S703 can be the same as that of step S111, and the specific implementation of step S704 can be the same as that of step S112. Therefore, the specific implementation of steps S703 and S704 will not be described in detail here.

[0082] In this embodiment of the invention, a vehicle direction angle trajectory prediction model is determined by the vehicle's actual trajectory, the initialized direction angle sequence, and the driver's intention, and a vehicle speed prediction model is determined by the vehicle's actual speed and the driver's intention, thereby improving the model accuracy and further improving the accuracy of vehicle trajectory generation.

[0083] The following describes the concept of the present invention in conjunction with a specific process of track generation for autonomous vehicles. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0084] Reference Figure 8The trajectory generation method for autonomous vehicles provided in this embodiment of the invention is based on a visual encoder, a large language model, and a trajectory prediction module. The visual encoder uses BEVFormer as the basic visual perception network, and the trajectory prediction module includes a vehicle heading angle trajectory prediction model and a vehicle speed prediction model. The vehicle heading angle trajectory prediction model is used to predict the vehicle heading angle sequence, and the vehicle speed prediction model is used to predict the vehicle speed.

[0085] like Figure 8 As shown, the following process is implemented based on a visual encoder, a large language model, and a trajectory prediction module.

[0086] 1) Receive multi-view, multi-frame historical image sequences from different cameras in the vehicle, and input these sequences into BEV Former to project the multi-view images into the BEV space. Extract spatial features through the Transformer (a deep learning model architecture based on attention mechanism) structure to obtain the BEV feature map Frame0~FrameN-1 corresponding to each frame in the historical image sequence.

[0087] 2) Initialize three types of learnable query vectors for each frame of BEV features: learnable query for traffic participants-vehicles, learnable query for traffic signs, and learnable query for lane lines, to represent traffic participants, lane lines, and traffic signs, respectively. These queries are shared across frames, and information from corresponding scene elements in each frame is aggregated through a self-attention mechanism to achieve cross-time information integration and interaction between different queries, thereby establishing the relationship between the vehicle and the driving environment, and thus obtaining the vehicle interaction features.

[0088] 3) The vehicle interaction features are projected onto the semantic space of the large language model through a linear layer to obtain vehicle text features. Navigation commands are then fed into a text encoder to obtain navigation text features. The vehicle text features and navigation text features are concatenated and input into the large language model based on the TinyLLama architecture to obtain multimodal features containing the vehicle's driving intention. The token of the vehicle's driving intention is extracted from the multimodal features and used as a guiding condition for the subsequent trajectory prediction module.

[0089] 4) Input the vehicle's driving intention and initial direction sequence into the trajectory prediction module. Based on the vehicle's direction angle trajectory prediction model, predict the vehicle's direction angle sequence according to the vehicle's driving intention and initial direction sequence. Based on the vehicle speed prediction model, predict the vehicle speed according to the vehicle's driving intention. Based on the predicted vehicle direction angle sequence and vehicle speed, obtain the vehicle's driving trajectory in the future.

[0090] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 9 As shown, an embodiment of the present invention also provides a trajectory generation system for an autonomous vehicle. The trajectory generation system for an autonomous vehicle includes an electronic device 90, which includes a processor 91 and a memory 92. The memory 92 stores a computer program 93 that can run on the processor. When the processor 91 executes the program, it performs the steps of the method as described in any of the preceding embodiments.

[0091] The memory, as a non-volatile storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the methods described in the embodiments of this application. The processor executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above embodiments.

[0092] The memory may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the device, etc. Furthermore, the memory may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the local module via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0093] Based on the same inventive concept, according to another aspect of the present invention, such as Figure 10 As shown, an embodiment of the present invention also provides a computer storage medium 100, which stores a computer program 110 that executes the above method when executed by a processor.

[0094] Finally, it should be noted that those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium for the program can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc. The above computer program embodiments can achieve the same or similar effects as any of the corresponding foregoing method embodiments.

[0095] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, the functionality of various illustrative components, blocks, modules, circuits, and steps has been generally described. Whether this functionality is implemented as software or as hardware depends on the specific application and the design constraints imposed on the system as a whole. Those skilled in the art can implement the functionality in various ways for each specific application, but such implementation decisions should not be construed as departing from the scope of the embodiments disclosed herein.

[0096] The above are exemplary embodiments disclosed in this invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments of this invention as defined by the claims. The functions, steps, and / or actions of the methods according to the disclosed embodiments described herein do not need to be performed in any particular order. The sequence numbers of the disclosed embodiments of this invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. Furthermore, although the elements disclosed in the embodiments of this invention may be described or claimed individually, they may be understood as multiple unless explicitly limited to a singular number.

[0097] It should be understood that, as used herein, the singular form “a” is intended to include the plural form as well, unless the context clearly supports an exception. It should also be understood that, as used herein, “and / or” refers to any and all possible combinations of one or more of the associated listed items.

[0098] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples. Within the framework of the invention, technical features of the above embodiments or different embodiments can be combined, and many other variations of different aspects of the invention exist, which are not provided in the details for the sake of brevity. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the protection scope of the invention.

Claims

1. A method for generating the trajectory of an autonomous vehicle, characterized in that, include: Based on the historical frame image sequence obtained from the vehicle, the autonomous driving intention of the vehicle is determined; Based on the pre-configured initial direction angle sequence and the vehicle's driving intention, predict the vehicle's direction angle sequence and vehicle speed; Based on the predicted vehicle heading angle sequence and the predicted vehicle speed, the vehicle's trajectory in the future is determined.

2. The method according to claim 1, characterized in that, Based on a pre-configured initialization direction angle sequence and the vehicle's driving intention, the predicted vehicle direction angle sequence and vehicle speed include: The initial direction angle sequence and the vehicle's driving intention are input into a pre-trained vehicle direction angle trajectory prediction model to obtain the predicted vehicle direction angle sequence. The driver's intention is input into a pre-trained vehicle speed prediction model to obtain the predicted vehicle speed.

3. The method according to claim 2, characterized in that, Also includes: Based on the initial direction angle sequence and the vehicle's driving intention, the diffusion model is trained to obtain a trained vehicle direction angle trajectory prediction model.

4. The method according to claim 3, characterized in that Training the diffusion model based on the initialization direction angle sequence and the vehicle's driving intention includes: Based on the vehicle's actual trajectory, determine the vehicle's actual direction angle sequence; The diffusion model is trained based on the vehicle's actual direction angle sequence, the initialized direction angle sequence, and the vehicle's driving intention.

5. The method according to claim 2, characterized in that, Also includes: The fully connected layer is trained based on the driver's intention to drive, resulting in a well-trained vehicle speed prediction model.

6. The method according to claim 5, characterized in that, Training the fully connected layer based on the driver's intention includes: Determine the vehicle's actual speed based on its actual trajectory; The fully connected layer is trained based on the vehicle's actual speed and the driver's intention.

7. The method according to claim 1, characterized in that, The determination of the vehicle's autonomous driving intention based on the historical frame image sequence obtained from the vehicle includes: Based on the historical frame image sequence obtained from the vehicle and the large language model, the autonomous driving intention of the vehicle is determined.

8. The method according to claim 7, characterized in that, Based on the historical frame image sequence obtained from the vehicle and the large language model, the autonomous driving intention of the vehicle is determined as follows: Based on the historical frame image sequence collected from the vehicle, the autonomous vehicle interaction features are determined; The autonomous vehicle interaction features are mapped to the semantic space of the large language model to obtain autonomous vehicle text features; The vehicle text features and the navigation text features determined according to the navigation instructions are input into the large language model to obtain multimodal features containing the vehicle's driving intentions.

9. A trajectory generation system for an autonomous vehicle, characterized in that, include: At least one processor, and a memory storing a computer program executable on the processor, wherein the processor, when executing the program, performs the steps of the method as described in any one of claims 1 to 8.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1 to 8.

Images