Trajectory generation model training, trajectory generation method and device

Abstract

The application discloses a kind of trajectory generation model training, trajectory generation method and device, belong to intelligent driving field.Method includes: obtaining the historical data of automatic driving vehicle;Based on initial trajectory generation model, the history data is processed, at least one of uniform prompt word vector, category prompt word vector, modal prompt word vector and traffic environment vector is obtained;Based on the setting template of semantic prompt word vector, at least one in uniform prompt word vector, category prompt word vector, modal prompt word vector is selected and processed, and semantic prompt word vector is obtained;Traffic environment vector and semantic prompt word vector are used to train initial trajectory generation model, and target trajectory generation model is obtained.Through setting template, the semantic prompt word vector of obstacle is constructed, meet the demand of each obstacle, realize the reuse of part unit in initial generation model, enhance the generalization and scalability of model, reduce the difficulty and cost of model design and model maintenance.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and in particular to a method and apparatus for training a trajectory generation model and generating a trajectory. Background Technology

[0002] With the continuous development of intelligent transportation, autonomous vehicles can obtain information about surrounding obstacles and predict their movement trajectories based on this information. This provides autonomous vehicles with predicted obstacle movement trajectories, allowing them to plan their routes in advance and ensuring safe driving.

[0003] Currently, repetitively expanded trajectory generation models are generally used to predict the motion trajectories of different types of obstacles, including vehicle obstacles and pedestrian obstacles. The repetitive expansion of the trajectory generation model leads to high maintenance costs, and different types of obstacles have different requirements for trajectory prediction. Furthermore, using repetitively expanded trajectory generation models to predict the motion trajectories of different types of obstacles results in significant discrepancies between the predicted and actual motion trajectories. Summary of the Invention

[0004] This application provides a method and apparatus for training a trajectory generation model and generating trajectories, which can be used to solve problems in related technologies. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide a training method for a trajectory generation model. The method may include: acquiring historical data of an autonomous vehicle, the historical data including obstacle information around the autonomous vehicle; processing the historical data based on an initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector, and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of the obstacle, and the modal prompt word vector is used to represent the modal information of the obstacle; selecting at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector based on a semantic prompt word vector setting template to process and obtain a semantic prompt word vector; and training the initial trajectory generation model using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model, the target trajectory generation model being used to generate the motion prediction trajectory of the obstacle.

[0006] In one possible implementation, processing the historical data based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector, and a traffic environment vector, may include: extracting features from the historical data using the initial trajectory generation model to obtain initial features; determining semantic features from the initial features; encoding the semantic features based on a first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector; determining traffic environment features from the initial features; and encoding the traffic environment features based on a second encoding unit of the initial trajectory generation model to obtain the traffic environment vector.

[0007] In one possible implementation, after extracting features from the historical data using the initial trajectory generation model to obtain initial features, the method may further include: determining the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned based on the historical data; generating a learning vector set based on the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned; and encoding the semantic features based on the first encoding unit of the initial trajectory generation model, which includes: encoding the semantic features based on the first encoding unit of the initial trajectory generation model and the learning vector set.

[0008] In one possible implementation, the semantic cue word vector includes the unified cue word vector, the category cue word vector, and the modal cue word vector. When there are multiple semantic cue word vectors for the obstacle, the unified cue word vector among the multiple semantic cue word vectors is the same.

[0009] In one possible implementation, training the initial trajectory generation model using the traffic environment vector and the semantic cue word vector to obtain the target trajectory generation model may include: decoding the traffic environment vector and the semantic cue word vector using the decoding unit of the initial trajectory generation model to obtain an obstacle prediction trajectory; calculating the loss value between the obstacle prediction trajectory and the actual trajectory; adjusting the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model, wherein the loss value of the target trajectory generation model is less than or equal to a reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0010] In one possible implementation, the decoding unit of the initial trajectory generation model decodes the traffic environment vector and the semantic cue word vector to obtain the obstacle prediction trajectory, which may include: using the decoding unit to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and a confidence level, wherein the confidence level is the probability that the obstacle will move to the corresponding mode in the future; filtering the initial obstacle prediction trajectory based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to a reference confidence level.

[0011] In one possible implementation, before processing the historical data based on the initial trajectory generation model, the method may further include: preprocessing the historical data to obtain preprocessed data, wherein the preprocessing includes at least one of standardizing the historical data, removing outlier data, and determining a preset range for the autonomous vehicle.

[0012] On the other hand, embodiments of this application provide a trajectory generation method, which may include: acquiring driving information of an autonomous vehicle, the driving information including obstacle information around the autonomous vehicle; predicting the motion trajectory of the obstacle based on a target trajectory generation model and the driving information to obtain a predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the above method.

[0013] In one possible implementation, after predicting the motion trajectory of the obstacle based on the target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle, the method may further include: generating assisted driving information based on the predicted motion trajectory of the obstacle, wherein the assisted driving information is used to assist in planning the driving path of the autonomous vehicle.

[0014] On the other hand, embodiments of this application provide a training device for a trajectory generation model. The device may include: an acquisition module for acquiring historical data of an autonomous vehicle, the historical data including obstacle information around the autonomous vehicle; an encoding module for processing the historical data based on an initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector, and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of the obstacle, and the modal prompt word vector is used to represent the modal information of the obstacle; a setting module for selecting at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector based on a setting template of the semantic prompt word vector to obtain a semantic prompt word vector; and a training module for training the initial trajectory generation model using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model, the target trajectory generation model being used to generate the motion prediction trajectory of the obstacle.

[0015] In one possible implementation, the encoding module is used to extract features from the historical data using the initial trajectory generation model to obtain initial features; determine semantic features from the initial features, and encode the semantic features based on the first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector; determine traffic environment features from the initial features, and encode the traffic environment features based on the second encoding unit of the initial trajectory generation model to obtain the traffic environment vector.

[0016] In one possible implementation, the encoding module is configured to determine the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned based on the historical data, and to generate a learning vector set based on the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned; the encoding module is further configured to encode the semantic features based on the first encoding unit of the initial trajectory generation model and the learning vector set.

[0017] In one possible implementation, the semantic cue word vector includes the unified cue word vector, the category cue word vector, and the modal cue word vector. When there are multiple semantic cue word vectors for the obstacle, the unified cue word vector among the multiple semantic cue word vectors is the same.

[0018] In one possible implementation, the training module is used to decode the traffic environment vector and the semantic cue word vector using the decoding unit of the initial trajectory generation model to obtain the obstacle prediction trajectory; calculate the loss value between the obstacle prediction trajectory and the actual trajectory; adjust the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model, wherein the loss value of the target trajectory generation model is less than or equal to the reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0019] In one possible implementation, the training module is used to decode the traffic environment vector and the semantic cue word vector using the decoding unit to obtain an initial obstacle prediction trajectory and a confidence level, wherein the confidence level is the probability that the obstacle will move to the corresponding mode in the future; the initial obstacle prediction trajectory is filtered based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

[0020] In one possible implementation, the acquisition module is further configured to preprocess the historical data to obtain preprocessed data, wherein the preprocessing includes at least one of standardizing the historical data, removing abnormal data, and determining a preset range for the autonomous vehicle.

[0021] On the other hand, embodiments of this application provide a trajectory generation apparatus, which may include: a collection module for acquiring driving information of an autonomous vehicle, the driving information including obstacle information around the autonomous vehicle; and a generation module for predicting the motion trajectory of the obstacle based on a target trajectory generation model and the driving information to obtain a predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the above-described method.

[0022] In one possible implementation, the generation module is further configured to generate assisted driving information based on the motion prediction trajectory of the obstacle, the assisted driving information being used to assist in planning the driving path of the autonomous vehicle.

[0023] On the other hand, embodiments of this application provide a computer device, which includes a processor and a memory. The memory stores at least one piece of program code, which is loaded and executed by the processor to enable the computer device to implement the training method or trajectory generation method of any of the trajectory generation models described above.

[0024] On the other hand, a computer-readable storage medium is also provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the at least one piece of program code being loaded and executed by a processor to enable a computer to implement the training method or trajectory generation method of any of the trajectory generation models described above.

[0025] On the other hand, a computer program or computer program product is also provided, wherein the computer program or computer program product stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to implement the training method or trajectory generation method of any of the above-mentioned trajectory generation models.

[0026] The technical solution provided in this application has at least the following beneficial effects:

[0027] The technical solution provided in this application utilizes semantic cue word vectors of obstacles and traffic environment vectors to train an initial trajectory generation model, thereby obtaining a target trajectory generation model. By setting templates to construct semantic cue word vectors for obstacles, the needs of various obstacles can be met, enabling the reuse of some units in the initial generation model, enhancing the model's scalability, and reducing the difficulty and cost of model design and maintenance to a certain extent. Training the initial trajectory generation model based on semantic cue word vectors results in a target trajectory generation model with strong learning ability and better generalization. Furthermore, the target trajectory generation model uses semantic cue word vectors and traffic environment vectors to generate predicted motion trajectories of obstacles, which can improve the accuracy of predicted motion trajectories to a certain extent. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the implementation environment for a trajectory generation model training or trajectory generation method provided in an embodiment of this application;

[0030] Figure 2 A flowchart illustrating a training method for a trajectory generation model provided in an embodiment of this application;

[0031] Figure 3 A flowchart for obtaining at least one of unified prompt word vectors, category prompt word vectors, modal prompt word vectors, and traffic environment vectors is provided in this application embodiment;

[0032] Figure 4A flowchart illustrating an embodiment of this application for obtaining a target trajectory generation model;

[0033] Figure 5 A flowchart for obtaining an obstacle prediction trajectory is provided as an embodiment of this application;

[0034] Figure 6 A flowchart for trajectory generation provided in an embodiment of this application;

[0035] Figure 7 A block diagram of a training device for a trajectory generation model provided in an embodiment of this application;

[0036] Figure 8 A block diagram of a trajectory generation device provided in an embodiment of this application;

[0037] Figure 9 This is a structural block diagram of the terminal device provided in the embodiments of this application;

[0038] Figure 10 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0040] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] Figure 1 This is a schematic diagram illustrating the implementation environment of a trajectory generation model training or trajectory generation method provided in an embodiment of this application. For example... Figure 1 As shown, the implementation environment includes: terminal device 101 and server 102.

[0042] The training method and trajectory generation method of the trajectory generation model provided in this application embodiment can be executed by the terminal device 101, the server 102, or jointly by the terminal device 101 and the server 102. This application embodiment does not limit this. In the case where the training method and trajectory generation method of the trajectory generation model provided in this application embodiment are jointly executed by the terminal device 101 and the server 102, the server 102 undertakes the main computational work, and the terminal device 101 undertakes the secondary computational work; or, the server 102 undertakes the secondary computational work, and the terminal device 101 undertakes the main computational work; or, the server 102 and the terminal device 101 use a distributed computing architecture for collaborative computation.

[0043] It should be noted that the execution device of the trajectory generation model training method and the execution device of the trajectory generation method can be the same or different, and this application embodiment does not limit this. For example, the execution device of the trajectory generation model training method is terminal device 101, and the execution device of the trajectory generation method is server 102; or, both the execution device of the trajectory generation model training method and the execution device of the trajectory generation method are terminal device 101.

[0044] Optionally, the terminal device 101 can be any electronic product capable of human-computer interaction with a user through one or more methods such as a keyboard, touchpad, touchscreen, remote control, voice interaction, or handwriting device. The terminal device 101 includes, but is not limited to, mobile phones, computers, intelligent voice interaction devices, vehicle terminals, and aircraft. The server 102 can be a single server, a server cluster consisting of multiple servers, or any of the following: a cloud computing platform or a virtualization center. This application embodiment does not limit this. The server 102 communicates with the terminal device 101 via a wired or wireless network. The server 102 has data receiving, data processing, and data sending functions. Of course, the server 102 may also have other functions, which this application embodiment does not limit.

[0045] Those skilled in the art should understand that the terminal device 101 and server 102 described above are merely illustrative examples. Other existing or future terminal devices or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0046] This application provides a training method for a trajectory generation model, which can be applied to the above-described implementation environment. Figure 2 The flowchart shown in this embodiment of the present application illustrates a training method for a trajectory generation model. This method can be executed by an electronic device, which may be... Figure 1 Terminal device 101 in the middle can also be Figure 1The server 102 in this embodiment is not limited thereto. Figure 2 As shown, the method includes the following steps 110 to 140.

[0047] In step 110, historical data of the autonomous vehicle is acquired, including information on obstacles around the autonomous vehicle.

[0048] In an exemplary embodiment of this application, the autonomous vehicle is equipped with a data acquisition device, which may include GPS (Global Positioning System), INS (Inertial Navigation System), odometer, camera, radar, etc., and the radar may include lidar, millimeter-wave radar, ultrasonic radar, etc. The autonomous vehicle can obtain information about itself and its surroundings through the data acquisition device.

[0049] For example, within a set historical time period, historical data of autonomous vehicles is acquired. This historical data includes data collected by the autonomous vehicles through data acquisition devices. For instance, the historical data may include information about the autonomous vehicles themselves, such as their speed, acceleration, position, time, and direction of travel.

[0050] For example, historical data can also include information about the surroundings of autonomous vehicles. This information can include obstacle information and traffic environment information. Obstacle information can include at least one of the obstacle's location, speed, and trajectory. Obstacles can include dynamic and static obstacles around the autonomous vehicle. Dynamic obstacles can include at least one of pedestrians, motor vehicles, and non-motorized vehicles, while static obstacles can include at least one of buildings, curbs, and guardrails. Traffic environment information can include traffic information and environmental information. Traffic information can include at least one of map information, lane markings, and traffic light information; environmental information can include at least one of weather information, brightness information, and the vehicle's driving environment. The information about the surroundings of autonomous vehicles can include various types of obstacle information, traffic information, and environmental information. For example, historical data can be obtained by collecting driving trajectory data for different scenarios and obstacle types.

[0051] It should be noted that the content of historical data of autonomous vehicles described in this application is illustrative only, and historical data may also include other information, and this application is not limited thereto.

[0052] In step 120, historical data is processed based on the initial trajectory generation model to obtain at least one of unified prompt word vector, category prompt word vector, and modal prompt word vector, as well as a traffic environment vector. The unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of obstacles, and the modal prompt word vector is used to represent the modal information of obstacles.

[0053] In an exemplary embodiment of this application, before processing historical data using the initial trajectory generation model, the historical data can be preprocessed to obtain preprocessed data. This preprocessing may include standardizing the historical data, removing outliers, and determining a preset range for the autonomous vehicle. By standardizing the historical data and removing outliers, more accurate historical data can be obtained, which improves the efficiency and accuracy of subsequent data processing.

[0054] In an exemplary embodiment of this application, preprocessing may further include determining a preset range based on the environmental information of the autonomous vehicle. The preset range may vary depending on the driving environment of the autonomous vehicle. For example, the driving environment of the autonomous vehicle may include urban roads, highways, rainy days, sunny days, foggy days, and accident-prone road sections. When the autonomous vehicle is driving on a highway, the speed of the autonomous vehicle and surrounding motor vehicles is generally high, allowing for a larger preset range. When the autonomous vehicle is driving in foggy weather, visibility is low and the speed is slow, allowing for a smaller preset range. After determining the preset range based on the environmental information, obstacles around the autonomous vehicle can be filtered based on the preset range, removing obstacles outside the preset range. Since obstacles within the preset range have a significant impact on the driving safety of the autonomous vehicle, historical data on these obstacles needs to be processed. Obstacles outside the preset range have little impact on the driving safety of the autonomous vehicle, so historical data on obstacles outside the preset range is removed. By preprocessing the historical data of obstacles within the preset range, the number of obstacles that need to be processed can be reduced while ensuring the driving safety of the autonomous vehicle, saving computational resources and improving the efficiency and accuracy of subsequent model training.

[0055] In exemplary embodiments of this application, preprocessed data or historical data can also be processed based on the initial trajectory generation model to obtain at least one of unified prompt word vectors, category prompt word vectors, and modal prompt word vectors, as well as a traffic environment vector.

[0056] Figure 3 This document provides a flowchart for obtaining at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector, as provided in an embodiment of this application. Figure 3As shown, obtaining at least one of the unified prompt word vector, category prompt word vector, modal prompt word vector, and traffic environment vector may include steps 121 to 123.

[0057] In step 121, the initial trajectory generation model is used to extract features from historical data to obtain initial features.

[0058] For example, the initial trajectory generation model can extract features from historical data or preprocessed data based on a convolutional neural network to obtain initial features, which include features of obstacles around the autonomous vehicle and traffic environment features.

[0059] In step 122, semantic features are determined from the initial features, and the semantic features are encoded based on the first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, category prompt word vector, and modality prompt word vector.

[0060] In the exemplary embodiments of this application, the Unified Prompt vector is used to represent scene information, the Class-specific Prompt vector is used to represent the category information of the obstacle, and the Modality-specific Prompt vector is used to represent the modal information of the obstacle. The modal information can be the direction of the obstacle's trajectory. It should be noted that the scene information in the Unified Prompt can be the scene information of the obstacle. Since the scene in which the autonomous vehicle is driving is the same as the scene of the obstacles around the autonomous vehicle, the scene information can be obtained based on historical data corresponding to the autonomous vehicle or the obstacle. This application uses the scene information of the obstacle as an example for illustration.

[0061] In an exemplary embodiment of this application, unified prompt word vectors, category prompt word vectors, and modal prompt word vectors to be learned can be determined based on historical data. A learning vector set is then established using these vectors, which includes the unified prompt word vectors, category prompt word vectors, and modal prompt word vectors that the initial trajectory generation model can learn. Furthermore, this application can also set the lengths of the unified prompt word vectors, category prompt word vectors, and modal prompt word vectors to be learned. The lengths of these vectors can be the same or different.

[0062] For example, the category cue word vectors in the learning vector set are used to represent the category information of obstacles. For instance, the category cue word vectors in the learning vector set can be set to three categories: motor vehicles, non-motor vehicles, and pedestrians, that is, three learnable category cue word vectors can be set; the category cue word vectors can also be set to seven categories: large vehicles, medium-sized vehicles, small vehicles, motorcycles, electric vehicles, bicycles, and pedestrians, that is, seven learnable category cue word vectors can be set.

[0063] For example, the modal cue word vectors in the learning vector set are used to represent the modal information of obstacles. For instance, the modal cue word vectors can be set to six modalities: left turn, left forward, straight ahead, right forward, right turn, and U-turn, i.e., six learnable modal cue word vectors are set. The modal cue word vectors can also divide the autonomous vehicle's trajectory into N sector regions, where N is a positive integer greater than or equal to a preset value. The angles corresponding to the N sector regions can be the same, and the preset value is the minimum number of sector regions. For example, the angle corresponding to each sector region is 360 / N, and each sector region can represent one modality, i.e., N learnable modal cue word vectors are set.

[0064] For example, the uniform cue word vector in the learning vector set is used to represent the scene information of the obstacle. The same obstacle or different obstacles with different category cue word vectors and / or modal cue word vectors can share the uniform cue word vector; that is, when an obstacle has multiple semantic cue word vectors, the uniform cue word vectors in these multiple semantic cue word vectors are the same. The uniform cue word vector can describe the obstacle's general scene information. The uniform cue word vector can contain one sub-vector or multiple sub-vectors. The sub-vectors of the learnable uniform cue words can include: highway sections, urban sections, rural sections, accident-prone sections, driving in rain, driving in fog, driving in sunny weather, daytime driving, and nighttime driving, etc.

[0065] In an exemplary embodiment of this application, after setting the learnable unified prompt word vectors, category prompt word vectors, and modal prompt word vectors in the learning vector set, the semantic features in the initial features are encoded based on the first encoding unit of the initial trajectory generation model and the learning vector set to obtain the corresponding unified prompt word vectors, category prompt word vectors, and modal prompt word vectors. The semantic features include scene features, category features, and modal features. For example, object recognition algorithms or clustering algorithms can be used to determine the semantic features in the initial features. It should be noted that the method for determining semantic features in the initial features in this application is illustrative and this application is not limited thereto.

[0066] For example, a set of initialized modal cue word vectors is input into the initial trajectory generation model. The initial trajectory generation model optimizes the initialized modal cue word vectors based on modal features to obtain the corresponding modal cue word vectors. It should be noted that the process of obtaining uniform cue word vectors and category cue word vectors is similar and will not be described in detail here.

[0067] In step 123, traffic environment features are determined from the initial features, and the traffic environment features are encoded by the second encoding unit based on the initial trajectory generation model to obtain the traffic environment vector.

[0068] In exemplary embodiments of this application, traffic environment features may include time information at each historical moment, static obstacle information, traffic light status, the position and attitude of the autonomous vehicle, the position and attitude of obstacles, lane lines, traffic flow, road signs, etc. The traffic environment features are encoded by the second encoding unit of the initial trajectory generation model to obtain a traffic environment vector corresponding to the traffic environment features. The second encoding unit can convert the traffic environment features into a fixed-length traffic environment vector. For example, target recognition algorithms or clustering algorithms can be used to determine the traffic environment features from the initial features. It should be noted that the method for determining traffic environment features from the initial features in this application is illustrative and not limited thereto.

[0069] In step 130, based on the semantic prompt word vector setting template, at least one of the unified prompt word vector, category prompt word vector, and modal prompt word vector is selected for processing to obtain the semantic prompt word vector.

[0070] In an exemplary embodiment of this application, the semantic cue word vector setting template can be set to <uniform cue word vector, category cue word vector, modal cue word vector>. The semantic cue word vector corresponding to each obstacle is determined using this template, wherein each obstacle can correspond to at least one semantic cue word vector. For example, the semantic cue word vectors of obstacles around an autonomous vehicle may include <daytime urban road driving, motor vehicle, left turn>, <daytime urban road driving, non-motorized vehicle, straight ahead>, etc.

[0071] According to an exemplary embodiment of this application, by using a unified semantic cue word vector setting template, corresponding semantic cue word vectors can be constructed for different types of obstacles, which can meet the needs of different types of obstacles. This allows for the reuse of the first encoding unit in the initial generation model across different types of obstacles, enhancing the model's scalability. Furthermore, the semantic cue word vectors are concise, which can reduce the difficulty and cost of model design and maintenance to a certain extent.

[0072] In step 140, the initial trajectory generation model is trained using traffic environment vectors and semantic cue word vectors to obtain the target trajectory generation model, which is used to generate the motion prediction trajectory of the obstacle.

[0073] In an exemplary embodiment of this application, after obtaining the traffic environment vector and the semantic cue word vector, the initial trajectory generation model is trained using the traffic environment vector and the semantic cue word vector to obtain the target trajectory generation model. Figure 4 This is a flowchart illustrating a target trajectory generation model provided in an embodiment of this application. Figure 4 As shown, obtaining the target trajectory generation model may include steps 141 and 142.

[0074] In step 141, the decoding unit of the initial trajectory generation model is used to decode the traffic environment vector and semantic cue word vector to obtain the obstacle prediction trajectory.

[0075] In an exemplary embodiment of this application, taking an obstacle around an autonomous vehicle as an example, the traffic environment vector and semantic cue word vector corresponding to the obstacle are selected, and the decoding unit of the initial trajectory generation model is used to decode the traffic environment vector and semantic cue word vector of the obstacle to obtain the movement trajectory of the obstacle.

[0076] Figure 5 This is a flowchart illustrating how to obtain a predicted trajectory for an obstacle, as provided in an embodiment of this application. Figure 5 As shown, the obstacle prediction trajectory may include steps 1411 and 1412.

[0077] In step 1411, the traffic environment vector and semantic cue word vector are decoded by the decoding unit to obtain the initial obstacle prediction trajectory and confidence level. The confidence level is the probability that the obstacle will move to the corresponding mode in the future.

[0078] For example, the traffic environment vector and semantic cue word vector of the obstacle are divided into a prediction vector and a baseline vector based on time information. Both the prediction vector and the baseline vector contain the traffic environment vector and the semantic cue word vector. The prediction vector is used to predict the obstacle trajectory, and the baseline vector is used to verify the obstacle trajectory. The second time period corresponding to the baseline vector is located after the first time period corresponding to the prediction vector, and the length of the second time period corresponding to the baseline vector is shorter than the length of the first time period corresponding to the prediction vector.

[0079] For example, the decoding unit of the initial trajectory generation model decodes the traffic environment vector and semantic cue word vector for the first time period, and uses the decoding results of the first time period to predict the movement trajectory of the obstacle in the second time period, obtaining the initial obstacle prediction trajectory and corresponding confidence score for each modality in the second time period. The confidence score represents the probability that the obstacle will move to the corresponding modality in the future. The number of initial obstacle prediction trajectories is the same as the number of modalities; that is, each modality corresponds to one initial obstacle prediction trajectory.

[0080] For example, the decoding unit of the initial trajectory generation model is used to decode the traffic environment vector and semantic cue word vector in the second time period to obtain the actual trajectory of the obstacle in the second time period.

[0081] In step 1412, the initial obstacle prediction trajectory is filtered based on the confidence level to obtain the obstacle prediction trajectory. The confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

[0082] For example, the confidence level corresponding to the initial obstacle prediction trajectory is compared with a reference confidence level, and the initial obstacle prediction trajectory with a confidence level greater than or equal to the reference confidence level is taken as the obstacle prediction trajectory. It should be noted that, in this application, the initial obstacle prediction trajectory corresponding to the highest confidence level can also be selected as the obstacle prediction trajectory.

[0083] In step 142, the loss value between the predicted trajectory and the actual trajectory of the obstacle is calculated. Based on the loss value, the parameters of the initial trajectory generation model are adjusted to obtain the target trajectory generation model. The loss value of the target trajectory generation model is less than or equal to the reference loss value. The loss value is used to characterize the degree of difference between the predicted trajectory and the actual trajectory of the obstacle.

[0084] For example, the loss value between the predicted trajectory and the actual trajectory of the obstacle is calculated based on the loss function. The loss value can characterize the degree of difference between the predicted trajectory and the actual trajectory. Then, the parameters of the initial trajectory generation model are adjusted based on the magnitude of the loss value until the loss value is less than or equal to the reference loss value, thus obtaining the target trajectory generation model. The target trajectory generation model can be used to generate the motion prediction trajectory of obstacles around the autonomous vehicle.

[0085] In an exemplary embodiment of this application, the initial trajectory generation model is trained using semantic cue word vectors of obstacles and traffic environment vectors to obtain the target trajectory generation model. The relationship between different types of semantic cue word vectors and their corresponding historical data is clear, allowing for stronger semantic cues; therefore, semantic cue word vectors have stronger data generalization ability. Training the initial trajectory generation model based on semantic cue word vectors results in a target trajectory generation model with strong learning ability and better generalization. Furthermore, the target trajectory generation model utilizes semantic cue word vectors and traffic environment vectors to generate the predicted motion trajectory of obstacles, which can improve the accuracy of the predicted motion trajectory to a certain extent.

[0086] This application also provides a method for trajectory generation. Figure 6 This is a flowchart illustrating trajectory generation as provided in an embodiment of this application. The method can be executed by an electronic device, which may be... Figure 1 Terminal device 101 in the middle can also be Figure 1 The server 102 in this embodiment is not limited thereto. Figure 6 As shown, trajectory generation may include steps 210 and 220.

[0087] In step 210, driving information of the autonomous vehicle is obtained, including information on obstacles around the autonomous vehicle.

[0088] In an exemplary embodiment of this application, a data acquisition device is used to collect driving information during the operation of an autonomous vehicle. This driving information may include information about the autonomous vehicle itself and information about its surroundings, including information about obstacles around the vehicle. The details regarding the information about the autonomous vehicle itself and its surroundings have already been described above and will not be repeated here.

[0089] In step 220, the motion trajectory of the obstacle is predicted based on the target trajectory generation model and driving information to obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the above method.

[0090] In an exemplary embodiment of this application, a trained target trajectory generation model is used to process driving information to obtain semantic cue word vectors and traffic environment vectors of obstacles surrounding the autonomous vehicle. A decoder is then used to decode these vectors to obtain the predicted motion trajectory of each obstacle around the autonomous vehicle. After obtaining the predicted motion trajectories of the obstacles around the autonomous vehicle, these trajectories can be further processed to obtain assisted driving information. For example, from multiple predicted motion trajectories, the predicted motion trajectory of an obstacle that may affect the driving safety of the autonomous vehicle can be selected to generate assisted driving information. This assisted driving information can suggest a path for the autonomous vehicle, and can generate a driving path during the autonomous vehicle's operation, thus ensuring the driving safety of the autonomous vehicle.

[0091] According to an exemplary embodiment of this application, by collecting driving information of autonomous vehicles and processing the driving information through a target trajectory generation model, the motion prediction trajectory of obstacles around the autonomous vehicle is obtained, and a driving path is generated during the driving of the autonomous vehicle, thereby ensuring the driving safety of the autonomous vehicle.

[0092] This application also provides a training device for a trajectory generation model. Figure 7 This is a block diagram of a training device for a trajectory generation model provided in an embodiment of this application. Figure 7 As shown, the training device for the trajectory generation model may include:

[0093] The acquisition module 310 is used to acquire historical data of the autonomous vehicle, including information on obstacles around the autonomous vehicle.

[0094] The encoding module 320 is used to process historical data based on the initial trajectory generation model to obtain at least one of unified prompt word vector, category prompt word vector, and modal prompt word vector, as well as a traffic environment vector. The unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of obstacles, and the modal prompt word vector is used to represent the modal information of obstacles.

[0095] The setting module 330 is used to set a template based on semantic prompt word vectors. It selects at least one of the unified prompt word vectors, category prompt word vectors, and modal prompt word vectors for processing to obtain semantic prompt word vectors.

[0096] Training module 340 is used to train the initial trajectory generation model using traffic environment vectors and semantic cue word vectors to obtain the target trajectory generation model, which is used to generate the motion prediction trajectory of obstacles.

[0097] In one possible implementation, the acquisition module 310 is further configured to preprocess historical data to obtain preprocessed data. The preprocessing includes at least one of standardizing the historical data, removing outlier data, and determining a preset range for the autonomous vehicle.

[0098] In one possible implementation, the encoding module 320 is used to determine the unified prompt word vector, category prompt word vector, and modal prompt word vector to be learned based on historical data, and to generate a learning vector set based on the unified prompt word vector, category prompt word vector, and modal prompt word vector to be learned; the encoding module 320 is also used to encode semantic features based on the first encoding unit of the initial trajectory generation model and the learning vector set.

[0099] In one possible implementation, the encoding module 320 is used to extract features from historical data using an initial trajectory generation model to obtain initial features; determine semantic features from the initial features; encode the semantic features based on the first encoding unit of the initial trajectory generation model to obtain at least one of unified prompt word vectors, category prompt word vectors, and modal prompt word vectors; determine traffic environment features from the initial features; and encode the traffic environment features based on the second encoding unit of the initial trajectory generation model to obtain a traffic environment vector.

[0100] In one possible implementation, the semantic cue word vector includes a uniform cue word vector, a category cue word vector, and a modal cue word vector. When there are multiple semantic cue word vectors for obstacles, the uniform cue word vectors in the multiple semantic cue word vectors are the same.

[0101] In one possible implementation, the training module 340 is used to decode the traffic environment vector and semantic cue word vector using the decoding unit of the initial trajectory generation model to obtain the obstacle prediction trajectory; calculate the loss value between the obstacle prediction trajectory and the actual trajectory; adjust the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model; the loss value of the target trajectory generation model is less than or equal to the reference loss value; and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0102] In one possible implementation, the training module 340 is used to decode the traffic environment vector and semantic prompt word vector using the decoding unit to obtain the initial obstacle prediction trajectory and confidence level, where the confidence level is the probability that the obstacle will move to the corresponding mode in the future; the initial obstacle prediction trajectory is filtered based on the confidence level to obtain the obstacle prediction trajectory, where the confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

[0103] In an exemplary embodiment of this application, the initial trajectory generation model is trained using semantic cue word vectors of obstacles and traffic environment vectors to obtain the target trajectory generation model. The correspondence between different types of semantic cue word vectors and historical data is clear, resulting in stronger semantic cueing capabilities and greater data generalization ability. Training the initial trajectory generation model based on semantic cue word vectors leads to a target trajectory generation model with strong learning ability and better generalization. Furthermore, the target trajectory generation model utilizes semantic cue word vectors and traffic environment vectors to generate the predicted motion trajectory of obstacles, which can improve the accuracy of the predicted motion trajectory to a certain extent.

[0104] This application also provides a trajectory generation apparatus. Figure 8 This is a block diagram of a trajectory generation device provided in an embodiment of this application. Figure 8 As shown, the trajectory generation device may include:

[0105] The acquisition module 350 is used to acquire driving information of autonomous vehicles, including information on obstacles around the autonomous vehicles.

[0106] The generation module 360 ​​is used to predict the motion trajectory of the obstacle based on the target trajectory generation model and driving information, and obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the above method.

[0107] In one possible implementation, the generation module 360 ​​is also used to generate assisted driving information based on the motion prediction trajectory of obstacles, and the assisted driving information is used to assist in planning the driving path of the autonomous vehicle.

[0108] According to an exemplary embodiment of this application, by collecting driving information of autonomous vehicles and processing the driving information through a target trajectory generation model, the motion prediction trajectory of obstacles around the autonomous vehicle is obtained, and a driving path is generated during the driving of the autonomous vehicle, thereby ensuring the driving safety of the autonomous vehicle.

[0109] It should be noted that the specific functions of the training device for the trajectory generation model and each module of the trajectory generation device have been explained in detail above, and will not be repeated here.

[0110] It should be understood that the above-described apparatus is only illustrated by the division of the functional modules described above when implementing its functions. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0111] Figure 9 This is a structural block diagram of a terminal device provided in an embodiment of this application. The terminal device 1100 can be a portable mobile terminal, such as a smartphone, tablet computer, laptop computer, or desktop computer. The terminal device 1100 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, intelligent voice interaction device, vehicle terminal, or other names.

[0112] Typically, terminal device 1100 includes a processor 1101 and a memory 1102.

[0113] Processor 1101 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1101 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1101 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1101 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0114] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1102 is used to store at least one instruction, which is executed by the processor 1101 to implement the present application. Figure 2 The method embodiments shown provide a training method for the trajectory generation model, or, to implement the method described in this application. Figure 6 The method embodiment shown provides a trajectory generation method.

[0115] In some embodiments, the terminal device 1100 may also optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, memory 1102, and peripheral device interface 1103 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1103 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107, and a power supply 1109.

[0116] Peripheral device interface 1103 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1101 and memory 1102. In some embodiments, processor 1101, memory 1102 and peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1101, memory 1102 and peripheral device interface 1103 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0117] The radio frequency (RF) circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1104 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1104 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1104 can communicate with other terminal devices through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1104 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0118] Display screen 1105 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1105 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1101 for processing. In this case, display screen 1105 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 1105, disposed on the front panel of terminal device 1100; in other embodiments, there may be at least two display screens, disposed on different surfaces of terminal device 1100 or in a folded design; in still other embodiments, display screen 1105 may be a flexible display screen, disposed on a curved or folded surface of terminal device 1100. Furthermore, display screen 1105 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0119] The camera assembly 1106 is used to acquire images or videos. Optionally, the camera assembly 1106 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal device 1100, and the rear-facing camera is located on the back of the terminal device 1100. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0120] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1101 for processing, or input to the radio frequency circuit 1104 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located at a different part of the terminal device 1100. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0121] Power supply 1109 is used to supply power to the various components in terminal device 1100. Power supply 1109 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1109 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, and a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0122] In some embodiments, the terminal device 1100 further includes one or more sensors 1110. The one or more sensors 1110 include, but are not limited to: an acceleration sensor 1111, a gyroscope sensor 1112, a pressure sensor 1113, an optical sensor 1115, and a proximity sensor 1116.

[0123] Accelerometer 1111 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal device 1100. For example, accelerometer 1111 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1101 can control display screen 1105 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1111. Accelerometer 1111 can also be used for games or for acquiring user motion data.

[0124] The gyroscope sensor 1112 can detect the orientation and rotation angle of the terminal device 1100. The gyroscope sensor 1112 can work in conjunction with the accelerometer sensor 1111 to collect the user's 3D movements on the terminal device 1100. Based on the data collected by the gyroscope sensor 1112, the processor 1101 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0125] The pressure sensor 1113 can be disposed on the side bezel of the terminal device 1100 and / or on the lower layer of the display screen 1105. When the pressure sensor 1113 is disposed on the side bezel of the terminal device 1100, it can detect the user's grip signal on the terminal device 1100, and the processor 1101 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1113. When the pressure sensor 1113 is disposed on the lower layer of the display screen 1105, the processor 1101 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1105. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0126] An optical sensor 1115 is used to collect ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the display screen 1105 based on the ambient light intensity collected by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is decreased. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 based on the ambient light intensity collected by the optical sensor 1115.

[0127] The proximity sensor 1116, also known as a distance sensor, is typically located on the front panel of the terminal device 1100. The proximity sensor 1116 is used to detect the distance between the user and the front of the terminal device 1100. In one embodiment, when the proximity sensor 1116 detects that the distance between the user and the front of the terminal device 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from a screen-on state to a screen-off state; when the proximity sensor 1116 detects that the distance between the user and the front of the terminal device 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from a screen-off state to a screen-on state.

[0128] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the terminal device 1100, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0129] Figure 10This is a schematic diagram of the server structure provided in the embodiments of this application. The server 1200 can vary considerably due to different configurations or performance. It may include one or more processors (Central Processing Units, CPUs) 1201 and one or more memories 1202. The one or more memories 1202 store at least one line of program code, which is loaded and executed by the one or more processors 1201 to implement [the desired function]. Figure 2 The method embodiments shown provide a training method for the trajectory generation model, or, to implement the method described in this application. Figure 6 The method embodiment shown provides a trajectory generation method. Of course, the server 1200 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 1200 may also include other components for implementing device functions, which will not be described in detail here.

[0130] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one line of program code, which is loaded and executed by a processor to enable the computer to perform the above-described functions. Figure 2 The method embodiments shown provide a training method for the trajectory generation model, or, to implement the method described in this application. Figure 6 The method embodiment shown provides a trajectory generation method.

[0131] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0132] In an exemplary embodiment, a computer program or computer program product is also provided, which stores at least one computer instruction, which is loaded and executed by a processor to enable the computer to perform the above-described functions. Figure 2 The method embodiments shown provide a training method for the trajectory generation model, or, to implement the method described in this application. Figure 6 The method embodiment shown provides a trajectory generation method.

[0133] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the historical data and driving information of autonomous vehicles involved in this application were obtained with full authorization.

[0134] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0135] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A training method for a trajectory generation model, characterized in that, The method includes: Acquire historical data of the autonomous vehicle, including obstacle information around the autonomous vehicle; The historical data is processed based on the initial trajectory generation model to obtain at least one of unified prompt word vector, category prompt word vector, and modal prompt word vector, as well as a traffic environment vector. The unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of the obstacle, and the modal prompt word vector is used to represent the modal information of the obstacle. Based on the template for setting semantic prompt word vectors, at least one of the unified prompt word vectors, the category prompt word vectors, and the modal prompt word vectors is selected for processing to obtain semantic prompt word vectors; The initial trajectory generation model is trained using the traffic environment vector and the semantic cue word vector to obtain the target trajectory generation model, which is used to generate the motion prediction trajectory of the obstacle. The semantic cue word vector includes the unified cue word vector, the category cue word vector, and the modality cue word vector. When there are multiple semantic cue word vectors for the obstacle, the unified cue word vector among the multiple semantic cue word vectors is the same.

2. The method according to claim 1, characterized in that, The historical data is processed based on the initial trajectory generation model to obtain at least one of unified prompt word vectors, category prompt word vectors, and modal prompt word vectors, as well as a traffic environment vector, including: The initial trajectory generation model is used to extract features from the historical data to obtain initial features; Semantic features are determined from the initial features, and the semantic features are encoded based on the first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modality prompt word vector; Traffic environment features are determined from the initial features, and the traffic environment features are encoded by the second encoding unit of the initial trajectory generation model to obtain the traffic environment vector.

3. The method according to claim 2, characterized in that, After extracting features from the historical data using the initial trajectory generation model to obtain initial features, the method further includes: Based on the historical data, determine the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned, and generate a learning vector set based on the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned; The first encoding unit based on the initial trajectory generation model encodes the semantic features, including: encoding the semantic features based on the first encoding unit of the initial trajectory generation model and the learning vector set.

4. The method according to any one of claims 1-3, characterized in that, The step of training the initial trajectory generation model using the traffic environment vector and the semantic cue word vector to obtain the target trajectory generation model includes: The traffic environment vector and the semantic cue word vector are decoded by the decoding unit of the initial trajectory generation model to obtain the obstacle prediction trajectory; The loss value between the predicted trajectory of the obstacle and the actual trajectory is calculated. Based on the loss value, the parameters of the initial trajectory generation model are adjusted to obtain the target trajectory generation model. The loss value of the target trajectory generation model is less than or equal to the reference loss value. The loss value is used to characterize the degree of difference between the predicted trajectory of the obstacle and the actual trajectory.

5. The method according to claim 4, characterized in that, The decoding unit utilizing the initial trajectory generation model decodes the traffic environment vector and the semantic cue word vector to obtain the obstacle prediction trajectory, including: The decoding unit is used to decode the traffic environment vector and the semantic cue word vector to obtain the initial obstacle prediction trajectory and confidence level, where the confidence level is the probability that the obstacle will move to the corresponding mode in the future. The initial obstacle prediction trajectory is filtered based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

6. The method according to any one of claims 1-3, characterized in that, Before processing the historical data based on the initial trajectory generation model, the method further includes: The historical data is preprocessed to obtain preprocessed data. The preprocessing includes at least one of the following: standardizing the historical data, removing outlier data, and determining a preset range for autonomous vehicles.

7. A method for trajectory generation, characterized in that, The method includes: Acquire driving information of an autonomous vehicle, including obstacle information around the autonomous vehicle; The motion trajectory of the obstacle is predicted based on the target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the method described in any one of claims 1 to 6.

8. A training device for a trajectory generation model, characterized in that, The device includes: An acquisition module is used to acquire historical data of the autonomous vehicle, including obstacle information around the autonomous vehicle. The encoding module is used to process the historical data based on the initial trajectory generation model to obtain at least one of unified prompt word vector, category prompt word vector, and modal prompt word vector, and a traffic environment vector. The unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent the category information of the obstacle, and the modal prompt word vector is used to represent the modal information of the obstacle. The setting module is used to set a template based on semantic prompt word vectors, and to process at least one of the unified prompt word vectors, the category prompt word vectors, and the modal prompt word vectors to obtain semantic prompt word vectors; The training module is used to train the initial trajectory generation model using the traffic environment vector and the semantic cue word vector to obtain the target trajectory generation model, which is used to generate the motion prediction trajectory of the obstacle. The semantic cue word vector includes the unified cue word vector, the category cue word vector, and the modality cue word vector. When there are multiple semantic cue word vectors for the obstacle, the unified cue word vector among the multiple semantic cue word vectors is the same.

9. A trajectory generation device, characterized in that, The device includes: The acquisition module is used to acquire driving information of the autonomous vehicle, including obstacle information around the autonomous vehicle. A generation module is used to predict the motion trajectory of the obstacle based on the target trajectory generation model and the driving information, and to obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the method described in any one of claims 1 to 6.

Images