Training method and device of trajectory generation model and trajectory generation method and device

Abstract

The invention discloses a trajectory generation model training method and device and a trajectory generation method and device, and belongs to the field of intelligent driving. The method comprises the following steps: acquiring historical data of an autonomous vehicle; processing the historical data based on the initial trajectory generation model to obtain at least one of a unified cue word vector, a category cue word vector and a modal cue word vector and a traffic environment vector; based on a setting template of a semantic cue word vector, selecting at least one of the unified cue word vector, the category cue word vector and the modal cue word vector for processing to obtain a semantic cue word vector; and training the initial trajectory generation model by using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model. The semantic prompt word vector of the obstacle is constructed by setting the template, so that the requirements of each obstacle are met, multiplexing of partial units in the initial generation model is realized, the generalization and expandability of the model are enhanced, and the difficulty and cost of model design and model maintenance are reduced.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent driving, and in particular to a method and device for training a trajectory generation model and generating a trajectory. Background Art

[0002] With the continuous development of the field of intelligent transportation, autonomous driving vehicles can obtain information about obstacles around them, predict the movement trajectory of obstacles based on the obstacle information, and provide autonomous driving vehicles with predicted movement trajectories of obstacles. Autonomous driving vehicles can plan their routes in advance to ensure safe driving.

[0003] At present, the motion trajectory of different types of obstacles is generally predicted by repeatedly expanding the trajectory generation model, where the obstacles include vehicle obstacles and pedestrian obstacles. The repeated expansion of the trajectory generation model leads to high expansion and maintenance costs of the model, and different types of obstacles have certain differences in the prediction requirements for the motion trajectory. The motion trajectory of different types of obstacles is predicted by repeatedly expanding the trajectory generation model, resulting in a large difference between the predicted motion trajectory of the obstacle and the actual motion trajectory. Summary of the invention

[0004] The embodiment of the present application provides a method and device for training a trajectory generation model and generating a trajectory, which can be used to solve the problems in the related technology. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides a method for training a trajectory generation model, which may include: obtaining historical data of an autonomous driving vehicle, the historical data including obstacle information around the autonomous driving vehicle; processing the historical data based on an initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector, and a traffic environment vector, the unified prompt word vector being used to represent scene information, the category prompt word vector being used to represent category information of the obstacle, and the modal prompt word vector being used to represent modal information of the obstacle; based on a setting template of a semantic prompt word vector, selecting at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector for processing to obtain a semantic prompt word vector; using the traffic environment vector and the semantic prompt word vector to train the initial trajectory generation model to obtain a target trajectory generation model, the target trajectory generation model being used to generate a motion prediction trajectory of the obstacle.

[0006] In a possible implementation, the processing of the historical data based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector may include: using the initial trajectory generation model to extract features of the historical data to obtain initial features; determining semantic features from the initial features, encoding the semantic features based on a first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector; determining traffic environment features from the initial features, encoding the traffic environment features based on a second encoding unit of the initial trajectory generation model to obtain the traffic environment vector.

[0007] In a possible implementation, after extracting features from the historical data using the initial trajectory generation model to obtain initial features, the method may further include: determining the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned based on the historical data, and generating a learning vector set based on the unified prompt word vector, the category prompt word vector, and the modal prompt word vector to be learned; encoding the semantic features by the first encoding unit based on the initial trajectory generation model, including: encoding the semantic features by the first encoding unit based on the initial trajectory generation model and the learning vector set.

[0008] In a possible implementation, the semantic prompt word vector includes the unified prompt word vector, the category prompt word vector and the modality prompt word vector. When there are multiple semantic prompt word vectors for the obstacle, the unified prompt word vector in the multiple semantic prompt word vectors is the same.

[0009] In a possible implementation, the using the traffic environment vector and the semantic cue word vector to train the initial trajectory generation model to obtain the target trajectory generation model may include: using a decoding unit of the initial trajectory generation model to decode the traffic environment vector and the semantic cue word vector to obtain an obstacle prediction trajectory; calculating a loss value between the obstacle prediction trajectory and the actual trajectory, and adjusting the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model, wherein the loss value of the target trajectory generation model is less than or equal to a reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0010] In a possible implementation, the decoding unit using the initial trajectory generation model decodes the traffic environment vector and the semantic cue word vector to obtain an obstacle prediction trajectory, which may include: using the decoding unit to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and a confidence level, wherein the confidence level is the probability that the obstacle will move to a corresponding mode in the future; and screening the initial obstacle prediction trajectory based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to a reference confidence level.

[0011] In a possible implementation, before processing the historical data based on the initial trajectory generation model, the method may further include: preprocessing the historical data to obtain preprocessed data, wherein the preprocessing includes standardizing the historical data, removing abnormal data, and determining at least one of a preset range of the autonomous driving vehicle.

[0012] On the other hand, an embodiment of the present application provides a method for trajectory generation, which may include: obtaining driving information of an autonomous driving vehicle, wherein the driving information includes obstacle information around the autonomous driving vehicle; predicting the motion trajectory of the obstacle based on a target trajectory generation model and the driving information to obtain a predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the above method.

[0013] In a possible implementation, after predicting the motion trajectory of the obstacle based on the target trajectory generation model and the driving information and obtaining the predicted motion trajectory of the obstacle, the method may further include: generating assisted driving information based on the predicted motion trajectory of the obstacle, and the assisted driving information is used to assist in planning the driving path of the autonomous driving vehicle.

[0014] On the other hand, an embodiment of the present application provides a training device for a trajectory generation model, which may include: an acquisition module for acquiring historical data of an autonomous driving vehicle, wherein the historical data includes obstacle information around the autonomous driving vehicle; an encoding module for processing the historical data based on an initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector, and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent category information of the obstacle, and the modal prompt word vector is used to represent modal information of the obstacle; a setting module for selecting at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector for processing based on a setting template of a semantic prompt word vector to obtain a semantic prompt word vector; a training module for training the initial trajectory generation model using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model, wherein the target trajectory generation model is used to generate a motion prediction trajectory of the obstacle.

[0015] In a possible implementation, the encoding module is used to extract features from the historical data using the initial trajectory generation model to obtain initial features; determine semantic features from the initial features, encode the semantic features based on a first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector; determine traffic environment features from the initial features, encode the traffic environment features based on a second encoding unit of the initial trajectory generation model to obtain the traffic environment vector.

[0016] In one possible implementation, the encoding module is used to determine the unified prompt word vector, the category prompt word vector and the modal prompt word vector to be learned based on the historical data, and generate a learning vector set based on the unified prompt word vector, the category prompt word vector and the modal prompt word vector to be learned; the encoding module is also used to encode the semantic feature based on the first encoding unit of the initial trajectory generation model and the learning vector set.

[0017] In a possible implementation, the semantic prompt word vector includes the unified prompt word vector, the category prompt word vector and the modality prompt word vector. When there are multiple semantic prompt word vectors for the obstacle, the unified prompt word vector in the multiple semantic prompt word vectors is the same.

[0018] In a possible implementation, the training module is used to use the decoding unit of the initial trajectory generation model to decode the traffic environment vector and the semantic prompt word vector to obtain the obstacle prediction trajectory; calculate the loss value between the obstacle prediction trajectory and the actual trajectory, and adjust the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model, the loss value of the target trajectory generation model is less than or equal to the reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0019] In a possible implementation, the training module is used to use the decoding unit to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and a confidence level, wherein the confidence level is the probability that the obstacle will move to a corresponding mode in the future; and to screen the initial obstacle prediction trajectory based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to a reference confidence level.

[0020] In a possible implementation, the acquisition module is further used to preprocess the historical data to obtain preprocessed data, and the preprocessing includes at least one of standardizing the historical data, removing abnormal data, and determining a preset range of the autonomous driving vehicle.

[0021] On the other hand, an embodiment of the present application provides a device for trajectory generation, which may include: an acquisition module for acquiring driving information of an autonomous driving vehicle, wherein the driving information includes obstacle information around the autonomous driving vehicle; a generation module for predicting the motion trajectory of the obstacle based on a target trajectory generation model and the driving information to obtain a predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the above-mentioned method.

[0022] In a possible implementation, the generation module is further used to generate assisted driving information based on the predicted motion trajectory of the obstacle, and the assisted driving information is used to assist in planning the driving path of the autonomous driving vehicle.

[0023] On the other hand, an embodiment of the present application provides a computer device, comprising a processor and a memory, wherein the memory stores at least one program code, and the at least one program code is loaded and executed by the processor so that the computer device implements any of the above-mentioned trajectory generation model training methods or trajectory generation methods.

[0024] On the other hand, a computer-readable storage medium is also provided, in which at least one program code is stored, and the at least one program code is loaded and executed by a processor so that a computer implements any of the above-mentioned trajectory generation model training methods or trajectory generation methods.

[0025] On the other hand, a computer program or a computer program product is also provided, wherein at least one computer instruction is stored in the computer program or the computer program product, and the at least one computer instruction is loaded and executed by a processor so that a computer implements any of the above-mentioned trajectory generation model training methods or trajectory generation methods.

[0026] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0027] The technical solution provided in the embodiment of the present application uses the semantic cue word vector and traffic environment vector of the obstacle to train the initial trajectory generation model to obtain the target trajectory generation model. By setting a template to construct the semantic cue word vector of the obstacle, the needs of each obstacle can be met, and the reuse of some units in the initial generation model can be achieved, which enhances the scalability of the model and reduces the difficulty and cost of model design and model maintenance to a certain extent. The initial trajectory generation model is trained based on the semantic cue word vector, and the target trajectory generation model has a strong learning ability and better generalization. In addition, the target trajectory generation model uses the semantic cue word vector and the traffic environment vector to generate the motion prediction trajectory of the obstacle, which can improve the accuracy of the motion prediction trajectory to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0029] Figure 1 It is a schematic diagram of an implementation environment of a trajectory generation model training or trajectory generation method provided in an embodiment of the present application;

[0030] Figure 2 A flowchart of a method for training a trajectory generation model provided in an embodiment of the present application;

[0031] Figure 3 A flowchart of obtaining at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector provided in an embodiment of the present application;

[0032] Figure 4A flow chart of obtaining a target trajectory generation model provided in an embodiment of the present application;

[0033] Figure 5 A flowchart of obtaining an obstacle prediction trajectory provided in an embodiment of the present application;

[0034] Figure 6 A flow chart of trajectory generation provided in an embodiment of the present application;

[0035] Figure 7 A block diagram of a training device for a trajectory generation model provided in an embodiment of the present application;

[0036] Figure 8 A block diagram of a trajectory generation device provided in an embodiment of the present application;

[0037] Fig. 9 A structural block diagram of a terminal device provided in an embodiment of the present application;

[0038] Fig.10 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] It should be noted that the terms "first", "second", etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] Figure 1 Schematic diagram of the implementation environment of a trajectory generation model training or trajectory generation method provided in an embodiment of the present application. Figure 1 As shown, the implementation environment includes: a terminal device 101 and a server 102 .

[0042] The training method of the trajectory generation model and the method of trajectory generation provided in the embodiment of the present application can be executed by the terminal device 101, can be executed by the server 102, or can be jointly executed by the terminal device 101 and the server 102, and the embodiment of the present application does not limit this. For the case where the training method of the trajectory generation model and the method of trajectory generation provided in the embodiment of the present application are jointly executed by the terminal device 101 and the server 102, the server 102 undertakes the main computing work, and the terminal device 101 undertakes the secondary computing work; or, the server 102 bears the secondary computing work, and the terminal device 101 undertakes the main computing work; or, the server 102 and the terminal device 101 use a distributed computing architecture for collaborative computing.

[0043] It should be noted that the execution device of the trajectory generation model training method and the execution device of the trajectory generation method can be the same or different, and the embodiments of the present application do not limit this. Exemplarily, the execution device of the trajectory generation model training method is the terminal device 101, and the execution device of the trajectory generation method is the server 102; or, the execution device of the trajectory generation model training method and the execution device of the trajectory generation method are both the terminal device 101.

[0044] Optionally, the terminal device 101 can be any electronic product that can perform human-computer interaction with a user through one or more methods such as a keyboard, a touchpad, a touch screen, a remote controller, voice interaction, or a handwriting device. The terminal device 101 includes but is not limited to a mobile phone, a computer, an intelligent voice interaction device, a vehicle-mounted terminal, an aircraft, etc. The server 102 is a server, or a server cluster composed of multiple servers, or any one of a cloud computing platform and a virtualization center, which is not limited in the embodiments of the present application. The server 102 is connected to the terminal device 101 through a wired network or a wireless network. The server 102 has a data receiving function, a data processing function, and a data sending function. Of course, the server 102 may also have other functions, which are not limited in the embodiments of the present application.

[0045] Those skilled in the art should understand that the above-mentioned terminal device 101 and server 102 are only for illustration, and other existing or future terminal devices or servers, if applicable to the present application, should also be included in the scope of protection of the present application and are included here by reference.

[0046] The present application embodiment provides a method for training a trajectory generation model, which can be applied to the above implementation environment. Figure 2 Taking the flowchart of a method for training a trajectory generation model provided in an embodiment of the present application as an example, the method can be executed by an electronic device, and the electronic device can be Figure 1 The terminal device 101 in Figure 1The server 102 in the present application is not limited to this. Figure 2 As shown, the method includes the following steps 110 to 140.

[0047] In step 110, historical data of the autonomous driving vehicle is obtained, where the historical data includes obstacle information around the autonomous driving vehicle.

[0048] In the exemplary embodiment of the present application, the autonomous driving vehicle is equipped with a data acquisition device, wherein the data acquisition device may include GPS (Global Positioning System), INS (Inertial Navigation System), odometer, camera, radar, etc. The radar may include laser radar, millimeter wave radar, ultrasonic radar, etc. The autonomous driving vehicle may obtain the information of the autonomous driving vehicle itself and the information around the autonomous driving vehicle through the data acquisition device.

[0049] Exemplarily, historical data of the autonomous driving vehicle is obtained within a set historical time period, and the historical data of the autonomous driving vehicle includes data collected by the autonomous driving vehicle through a data collection device. For example, the historical data may include information about the autonomous driving vehicle itself, and the information about the autonomous driving vehicle itself may include the speed, acceleration, position, time, and driving direction of the vehicle.

[0050] For example, historical data may also include information around the autonomous driving vehicle, and the information around the autonomous driving vehicle may include obstacle information and traffic environment information, wherein the obstacle information may include at least one of the position, moving speed, and moving trajectory of the obstacle. Obstacles may include dynamic obstacles and static obstacles around the autonomous driving vehicle, dynamic obstacles may include at least one of pedestrians, motor vehicles, and non-motor vehicles, and static obstacles may include at least one of buildings, curbs, and railings. Traffic environment information may include traffic information and environmental information, and traffic information may include at least one of map information, lane line information, and traffic light information; environmental information may include weather information, brightness information, and at least one of the vehicle's driving environment. Among them, the information around the autonomous driving vehicle may include multiple types of obstacle information, traffic information, and environmental information. For example, historical data can be obtained by collecting driving trajectory data of different scenarios and different obstacle types.

[0051] It should be noted that the content of the historical data of the autonomous driving vehicle in this application is for illustrative purposes only. The historical data may also include other information, and this application is not limited to this.

[0052] In step 120, the historical data is processed based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector. The unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent category information of obstacles, and the modal prompt word vector is used to represent modal information of obstacles.

[0053] In an exemplary embodiment of the present application, before the historical data is processed using the initial trajectory generation model, the historical data may be preprocessed to obtain preprocessed data. Preprocessing the historical data may include standardizing the historical data, removing abnormal data, and determining a preset range of the autonomous driving vehicle. By standardizing the historical data and removing abnormal data, more accurate historical data can be obtained, which is conducive to improving the efficiency and accuracy of subsequent data processing processes.

[0054] In an exemplary embodiment of the present application, the preprocessing may also include determining a preset range based on the environmental information of the autonomous driving vehicle. The corresponding preset range may be different depending on the driving environment of the autonomous driving vehicle. For example, the driving environment of the autonomous driving vehicle may include urban roads, highways, rainy days, sunny days, foggy days, accident-prone sections, etc. When the driving environment of the autonomous driving vehicle is a highway, in general, the driving speed of the autonomous driving vehicle and the surrounding motor vehicles is faster, and a larger preset range can be set; when the driving environment of the autonomous driving vehicle is foggy, the visibility is low and the speed is slow, and a smaller preset range can be set. After determining the preset range based on the environmental information, the obstacles around the autonomous driving vehicle can be screened based on the preset range, and the obstacles outside the preset range can be removed. Since the obstacles within the preset range have a greater impact on the driving safety of the autonomous driving vehicle, the historical data of the obstacles need to be processed. The obstacles outside the preset range have little impact on the driving safety of the autonomous driving vehicle, so the historical data of the obstacles outside the preset range are removed. By preprocessing the historical data of the obstacles within the preset range, the number of obstacles to be processed can be reduced while ensuring the driving safety of the autonomous driving vehicle, computing resources can be saved, and the efficiency and accuracy of subsequent model training can be improved.

[0055] In an exemplary embodiment of the present application, preprocessed data or historical data may also be processed based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector.

[0056] Figure 3 A flowchart of obtaining at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector is provided in an embodiment of the present application. Figure 3As shown, obtaining at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector may include steps 121 to 123.

[0057] In step 121, the initial trajectory generation model is used to extract features from the historical data to obtain initial features.

[0058] Exemplarily, the initial trajectory generation model can extract features from historical data or preprocessed data based on a convolutional neural network to obtain initial features, where the initial features include features of obstacles around the autonomous driving vehicle and features of the traffic environment.

[0059] In step 122, semantic features are determined in the initial features, and the semantic features are encoded based on the first encoding unit of the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modality prompt word vector.

[0060] In the exemplary embodiment of the present application, the unified prompt vector is used to represent the scene information, the class-specific prompt vector is used to represent the class information of the obstacle, and the modality-specific prompt vector is used to represent the modality information of the obstacle. The modality information may be the direction of the obstacle's motion trajectory. It should be noted that the scene information in the unified prompt may be the scene information of the obstacle. Since the scene of the autonomous driving vehicle is the same as the scene of the obstacle around the autonomous driving vehicle, the scene information may be obtained based on the historical data corresponding to the autonomous driving vehicle or the obstacle. The present application takes the scene information of the obstacle as an example for explanation.

[0061] In an exemplary embodiment of the present application, the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned can be determined based on historical data, and the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned are used to set a learning vector set, and the learning vector set includes the unified prompt word vector, category prompt word vector and modal prompt word vector that can be learned by the initial trajectory generation model. Among them, the present application can also set the length of the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned, and the length of the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned can be the same or different.

[0062] Exemplarily, the category prompt word vectors in the learning vector set are used to represent the category information of obstacles. For example, the category prompt word vectors in the learning vector set can be set to three categories: motor vehicles, non-motor vehicles, and pedestrians, that is, three learnable category prompt word vectors are set; the category prompt word vectors can also be set to seven categories: large vehicles, medium-sized vehicles, small vehicles, motorcycles, electric vehicles, bicycles, and pedestrians, that is, seven learnable category prompt word vectors are set.

[0063] Exemplarily, the modal prompt word vector in the learning vector set is used to characterize the modal information of the obstacle. For example, the modal prompt word vector can be set to six modes: left turn, left front, straight, right front, right turn, and U-turn, that is, six learnable modal prompt word vectors are set; the modal prompt word vector can also divide the motion trajectory of the autonomous driving vehicle into N sector areas, N is a positive integer greater than or equal to a preset value, and the angles corresponding to the N sector areas can be the same, and the preset value is the minimum number of divided sector areas. For example, the angle corresponding to each sector area is 360 / N, and each sector area can represent a mode, that is, N learnable modal prompt word vectors are set.

[0064] Exemplarily, the unified prompt word vector in the learning vector set is used to characterize the scene information of the obstacle. The same obstacle or different obstacles with different category prompt word vectors and / or modal prompt word vectors can share a unified prompt word vector, that is, when there are multiple semantic prompt word vectors for the obstacle, the unified prompt word vectors in the multiple semantic prompt word vectors are the same. The unified prompt word vector can be a description of the obstacle in general scene information, and the unified prompt word vector can contain one sub-vector or multiple sub-vectors. The sub-vectors of the unified prompt words that can be learned include: highway sections, urban sections, rural sections, accident-prone sections, driving in rainy days, driving in foggy days, driving in sunny days, driving during the day, driving at night, etc.

[0065] In an exemplary embodiment of the present application, after setting the unified prompt word vector, category prompt word vector and modal prompt word vector that can be learned in the learning vector set, the semantic features in the initial features are encoded based on the first encoding unit of the initial trajectory generation model and the learning vector set to obtain the corresponding unified prompt word vector, category prompt word vector and modal prompt word vector, wherein the semantic features include scene features, category features and modal features. For example, the semantic features can be determined in the initial features using a target recognition algorithm or a clustering algorithm. It should be noted that the method for determining semantic features in the initial features of the present application is an exemplary description, and the present application is not limited to this.

[0066] Exemplarily, a set of initialized modal prompt word vectors is input into the initial trajectory generation model, and the initial trajectory generation model optimizes the initialized modal prompt word vectors based on the modal features to obtain the corresponding modal prompt word vectors. It should be noted that the process of obtaining the unified prompt word vector and the category prompt word vector is similar, and will not be described one by one here.

[0067] In step 123, a traffic environment feature is determined in the initial feature, and the traffic environment feature is encoded based on the second encoding unit of the initial trajectory generation model to obtain a traffic environment vector.

[0068] In an exemplary embodiment of the present application, traffic environment features may include time information at each historical moment, static obstacle information, the state of traffic lights, the position of the autonomous driving vehicle, the posture of the autonomous driving vehicle, the position of obstacles, the posture of obstacles, lane lines, traffic flow, road signs, etc. The traffic environment features are encoded by the second encoding unit of the initial trajectory generation model to obtain a traffic environment vector corresponding to the traffic environment features, wherein the second encoding unit can convert the traffic environment features into a traffic environment vector of a fixed length. For example, the traffic environment features can be determined in the initial features using a target recognition algorithm or a clustering algorithm. It should be noted that the method of determining the traffic environment features in the initial features of the present application is an exemplary description, and the present application is not limited thereto.

[0069] In step 130, based on the setting template of the semantic prompt word vector, at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector is selected for processing to obtain the semantic prompt word vector.

[0070] In an exemplary embodiment of the present application, the setting template of the semantic prompt word vector can be set to <unified prompt word vector, category prompt word vector, modal prompt word vector>, and the semantic prompt word vector corresponding to each obstacle is determined by using the setting template of the semantic prompt word vector, wherein each obstacle can correspond to at least one semantic prompt word vector. For example, the semantic prompt word vectors of obstacles around the autonomous driving vehicle may include <daytime urban road driving, motor vehicle, left turn>, <daytime urban road driving, non-motor vehicle, straight>, etc.

[0071] According to the exemplary embodiment of the present application, by using a unified semantic hint word vector setting template, corresponding semantic hint word vectors are constructed for different types of obstacles, which can meet the needs of different types of obstacles, and between different types of obstacles, the first encoding unit in the initial generation model can be reused, thereby enhancing the scalability of the model. In addition, the semantic hint word vector is concise, which can reduce the difficulty and cost of model design and model maintenance to a certain extent.

[0072] In step 140, the initial trajectory generation model is trained using the traffic environment vector and the semantic cue word vector to obtain a target trajectory generation model, which is used to generate a motion prediction trajectory of the obstacle.

[0073] In an exemplary embodiment of the present application, after obtaining the traffic environment vector and the semantic prompt word vector, the traffic environment vector and the semantic prompt word vector are used to train the initial trajectory generation model to obtain the target trajectory generation model. Figure 4 A flow chart of obtaining a target trajectory generation model provided in an embodiment of the present application. Figure 4 As shown, obtaining the target trajectory generation model may include step 141 and step 142.

[0074] In step 141, the traffic environment vector and the semantic cue word vector are decoded by using the decoding unit of the initial trajectory generation model to obtain the obstacle prediction trajectory.

[0075] In an exemplary embodiment of the present application, taking an obstacle around an autonomous driving vehicle as an example, the traffic environment vector and semantic cue word vector corresponding to the obstacle are selected, and the traffic environment vector and semantic cue word vector of the obstacle are decoded using the decoding unit of the initial trajectory generation model to obtain the movement trajectory of the obstacle.

[0076] Figure 5 A flowchart of obtaining an obstacle prediction trajectory is provided in an embodiment of the present application. Figure 5 As shown, the obstacle prediction trajectory may include step 1411 and step 1412.

[0077] In step 1411, a decoding unit is used to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and confidence, where the confidence is the probability that the obstacle will move to the corresponding mode in the future.

[0078] Exemplarily, the traffic environment vector and semantic cue word vector of the obstacle are divided into a prediction vector and a reference vector based on time information, wherein both the prediction vector and the reference vector contain the traffic environment vector and the semantic cue word vector, the prediction vector is used to predict the obstacle trajectory, and the reference vector is used to verify the obstacle trajectory. The second time period corresponding to the reference vector is located after the first time period corresponding to the prediction vector, and the length of the second time period corresponding to the reference vector is less than the length of the first time period corresponding to the prediction vector.

[0079] Exemplarily, the decoding unit of the initial trajectory generation model is used to decode the traffic environment vector and the semantic cue word vector of the first time period, and the movement trajectory of the obstacle in the second time period is predicted using the decoding result of the first time period, so as to obtain the initial obstacle prediction trajectory of the obstacle in the second time period in each mode and the corresponding confidence. Among them, the confidence can represent the probability that the obstacle will move to the corresponding mode in the future, and the number of initial obstacle prediction trajectories is the same as the number of modes, that is, each mode corresponds to an initial obstacle prediction trajectory.

[0080] Exemplarily, the decoding unit of the initial trajectory generation model is used to decode the traffic environment vector and the semantic prompt word vector of the second time period to obtain the actual trajectory of the obstacle in the second time period.

[0081] In step 1412, the initial obstacle prediction trajectory is screened based on the confidence level to obtain an obstacle prediction trajectory, and the confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

[0082] Exemplarily, the confidence corresponding to the initial obstacle prediction trajectory is compared with the reference confidence, and the initial obstacle prediction trajectory with a confidence greater than or equal to the reference confidence is used as the obstacle prediction trajectory. It should be noted that the present application may also select the initial obstacle prediction trajectory corresponding to the maximum confidence among the confidences as the obstacle prediction trajectory.

[0083] In step 142, the loss value between the obstacle prediction trajectory and the actual trajectory is calculated, and the parameters of the initial trajectory generation model are adjusted based on the loss value to obtain a target trajectory generation model. The loss value of the target trajectory generation model is less than or equal to the reference loss value. The loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0084] Exemplarily, the loss value between the predicted obstacle trajectory and the actual trajectory is calculated based on the loss function. The loss value can characterize the degree of difference between the predicted obstacle trajectory and the actual trajectory. Then, based on the size of the loss value, the parameters of the initial trajectory generation model are adjusted until the loss value is less than or equal to the reference loss value, and the target trajectory generation model is obtained. The target trajectory generation model can be used to generate motion prediction trajectories of obstacles around the autonomous driving vehicle.

[0085] In an exemplary embodiment of the present application, the initial trajectory generation model is trained using the semantic cue word vector and traffic environment vector of the obstacle to obtain the target trajectory generation model. The relationship between different types of semantic cue word vectors and the corresponding historical data is clear, and can have a stronger semantic cue effect. Therefore, the semantic cue word vector has a stronger data generalization ability. The initial trajectory generation model is trained based on the semantic cue word vector, and the target trajectory generation model has a strong learning ability and better generalization. And the target trajectory generation model uses the semantic cue word vector and the traffic environment vector to generate the motion prediction trajectory of the obstacle, which can improve the accuracy of the motion prediction trajectory to a certain extent.

[0086] The present application also provides a trajectory generation method. Figure 6 A flow chart of trajectory generation provided in an embodiment of the present application. The method can be executed by an electronic device, which can be Figure 1 The terminal device 101 in Figure 1 The server 102 in the present application is not limited to this. Figure 6 As shown, trajectory generation may include step 210 and step 220 .

[0087] In step 210, driving information of the autonomous driving vehicle is obtained, where the driving information includes obstacle information around the autonomous driving vehicle.

[0088] In the exemplary embodiment of the present application, a data acquisition device is used to collect driving information during the driving process of the autonomous driving vehicle, wherein the driving information may include the information of the autonomous driving vehicle itself and the information around the autonomous driving vehicle, and the information around the autonomous driving vehicle includes the obstacle information around the autonomous driving vehicle. The relevant contents of the information of the autonomous driving vehicle itself and the information around the autonomous driving vehicle have been described in detail above, and will not be repeated here.

[0089] In step 220, the motion trajectory of the obstacle is predicted based on the target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the above method.

[0090] In an exemplary embodiment of the present application, the driving information is processed using a trained target trajectory generation model to obtain semantic cue word vectors and traffic environment vectors of obstacles around the autonomous driving vehicle. The semantic cue word vectors and traffic environment vectors of the obstacles are decoded using a decoder to obtain the motion prediction trajectory of each obstacle around the autonomous driving vehicle. After obtaining the motion prediction trajectory of the obstacles around the autonomous driving vehicle, the motion prediction trajectory can also be processed to obtain assisted driving information. For example, the motion prediction trajectory of obstacles that may affect the driving safety of the autonomous driving vehicle is selected from multiple motion prediction trajectories to generate assisted driving information. The assisted driving information can be a recommended driving path for the autonomous driving vehicle. The assisted driving information can generate a driving path during the driving of the autonomous driving vehicle to ensure the driving safety of the autonomous driving vehicle.

[0091] According to an exemplary embodiment of the present application, driving information of an autonomous driving vehicle is collected and processed through a target trajectory generation model to obtain a predicted motion trajectory of obstacles around the autonomous driving vehicle, and a driving path is generated during the driving of the autonomous driving vehicle to ensure the driving safety of the autonomous driving vehicle.

[0092] The present application also provides a training device for a trajectory generation model. Figure 7 A block diagram of a training device for a trajectory generation model provided in an embodiment of the present application. Figure 7 As shown, the training device of the trajectory generation model may include:

[0093] The acquisition module 310 is used to acquire historical data of the autonomous driving vehicle, where the historical data includes obstacle information around the autonomous driving vehicle.

[0094] The encoding module 320 is used to process the historical data based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent category information of obstacles, and the modal prompt word vector is used to represent modal information of obstacles.

[0095] The setting module 330 is used to select at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector for processing based on the setting template of the semantic prompt word vector to obtain the semantic prompt word vector.

[0096] The training module 340 is used to train the initial trajectory generation model using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model, and the target trajectory generation model is used to generate a motion prediction trajectory of the obstacle.

[0097] In one possible implementation, the acquisition module 310 is also used to preprocess the historical data to obtain preprocessed data, where the preprocessing includes standardizing the historical data, removing abnormal data, and determining at least one of a preset range of the autonomous driving vehicle.

[0098] In one possible implementation, the encoding module 320 is used to determine the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned based on historical data, and to generate a learning vector set based on the unified prompt word vector, category prompt word vector and modal prompt word vector to be learned; the encoding module 320 is also used to encode semantic features based on the first encoding unit of the initial trajectory generation model and the learning vector set.

[0099] In a possible implementation, the encoding module 320 is used to extract features from historical data using an initial trajectory generation model to obtain initial features; determine semantic features from the initial features, encode the semantic features based on a first encoding unit of the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector; determine traffic environment features from the initial features, encode the traffic environment features based on a second encoding unit of the initial trajectory generation model to obtain a traffic environment vector.

[0100] In a possible implementation, the semantic prompt word vector includes a unified prompt word vector, a category prompt word vector, and a modality prompt word vector. When there are multiple semantic prompt word vectors for an obstacle, the unified prompt word vectors in the multiple semantic prompt word vectors are the same.

[0101] In a possible implementation, the training module 340 is used to use the decoding unit of the initial trajectory generation model to decode the traffic environment vector and the semantic prompt word vector to obtain the obstacle prediction trajectory; calculate the loss value between the obstacle prediction trajectory and the actual trajectory, and adjust the parameters of the initial trajectory generation model based on the loss value to obtain the target trajectory generation model, the loss value of the target trajectory generation model is less than or equal to the reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

[0102] In one possible implementation, the training module 340 is used to use a decoding unit to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and a confidence level, where the confidence level is the probability that the obstacle will move to the corresponding mode in the future; the initial obstacle prediction trajectory is screened based on the confidence level to obtain an obstacle prediction trajectory, where the confidence level of the obstacle prediction trajectory is greater than or equal to the reference confidence level.

[0103] In an exemplary embodiment of the present application, the initial trajectory generation model is trained using the semantic cue word vector and traffic environment vector of the obstacle to obtain the target trajectory generation model. The correspondence between different types of semantic cue word vectors and historical data is clear, which can have a stronger semantic cue effect, and the semantic cue word vector has a stronger data generalization ability. The initial trajectory generation model is trained based on the semantic cue word vector, and the target trajectory generation model has a strong learning ability and better generalization. And the target trajectory generation model uses the semantic cue word vector and the traffic environment vector to generate the motion prediction trajectory of the obstacle, which can improve the accuracy of the motion prediction trajectory to a certain extent.

[0104] The present application also provides a trajectory generation device. Figure 8 A block diagram of a trajectory generation device provided in an embodiment of the present application. Figure 8 As shown, the trajectory generating device may include:

[0105] The acquisition module 350 is used to obtain driving information of the autonomous driving vehicle, where the driving information includes obstacle information around the autonomous driving vehicle.

[0106] The generation module 360 ​​is used to predict the motion trajectory of the obstacle based on the target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle. The target trajectory generation model is trained based on the above method.

[0107] In a possible implementation, the generation module 360 ​​is also used to generate assisted driving information based on the predicted motion trajectory of the obstacle, and the assisted driving information is used to assist in planning the driving path of the autonomous driving vehicle.

[0108] According to an exemplary embodiment of the present application, driving information of an autonomous driving vehicle is collected and processed through a target trajectory generation model to obtain a predicted motion trajectory of obstacles around the autonomous driving vehicle, and a driving path is generated during the driving of the autonomous driving vehicle to ensure the driving safety of the autonomous driving vehicle.

[0109] It should be noted that the specific functions of the various modules of the trajectory generation model training device and the trajectory generation device have been described in detail above and will not be repeated here.

[0110] It should be understood that the above-mentioned device only uses the division of the above-mentioned functional modules as an example to illustrate when implementing its functions. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0111] Fig. 9 This is a block diagram of a terminal device provided in an embodiment of the present application. The terminal device 1100 may be a portable mobile terminal, such as a smart phone, a tablet computer, a laptop computer, or a desktop computer. The terminal device 1100 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal, an intelligent voice interaction device, a vehicle-mounted terminal, or other names.

[0112] Typically, the terminal device 1100 includes: a processor 1101 and a memory 1102 .

[0113] The processor 1101 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 1101 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 1101 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1101 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1101 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0114] The memory 1102 may include one or more computer-readable storage media, which may be non-transitory. The memory 1102 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1102 is used to store at least one instruction, which is executed by the processor 1101 to implement the present application. Figure 2 The method embodiment shown in the figure provides a method for training a trajectory generation model, or to implement the present application Figure 6 The illustrated method embodiment provides a method for trajectory generation.

[0115] In some embodiments, the terminal device 1100 may further optionally include: a peripheral device interface 1103 and at least one peripheral device. The processor 1101, the memory 1102 and the peripheral device interface 1103 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 1103 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 1104, a display screen 1105, a camera assembly 1106, an audio circuit 1107 and a power supply 1109.

[0116] The peripheral device interface 1103 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 1101 and the memory 1102. In some embodiments, the processor 1101, the memory 1102, and the peripheral device interface 1103 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1101, the memory 1102, and the peripheral device interface 1103 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0117] The radio frequency circuit 1104 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 1104 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1104 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 1104 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 1104 can communicate with other terminal devices through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 1104 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0118] The display screen 1105 is used to display a UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 1105 is a touch display screen, the display screen 1105 also has the ability to collect touch signals on the surface or above the surface of the display screen 1105. The touch signal can be input to the processor 1101 as a control signal for processing. At this time, the display screen 1105 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 1105 can be one, which is set on the front panel of the terminal device 1100; in other embodiments, the display screen 1105 can be at least two, which are respectively set on different surfaces of the terminal device 1100 or are folded; in other embodiments, the display screen 1105 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal device 1100. Even, the display screen 1105 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 1105 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0119] The camera assembly 1106 is used to capture images or videos. Optionally, the camera assembly 1106 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal device 1100, and the rear camera is arranged on the back of the terminal device 1100. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 1106 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0120] The audio circuit 1107 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 1101 for processing, or input them into the radio frequency circuit 1104 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal device 1100. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 1101 or the radio frequency circuit 1104 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 1107 may also include a headphone jack.

[0121] The power supply 1109 is used to power various components in the terminal device 1100. The power supply 1109 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 1109 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0122] In some embodiments, the terminal device 1100 further includes one or more sensors 1110 , including but not limited to: an acceleration sensor 1111 , a gyroscope sensor 1112 , a pressure sensor 1113 , an optical sensor 1115 , and a proximity sensor 1116 .

[0123] The acceleration sensor 1111 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal device 1100. For example, the acceleration sensor 1111 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 1101 can control the display screen 1105 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 1111. The acceleration sensor 1111 can also be used to collect game or user motion data.

[0124] The gyroscope sensor 1112 can detect the body direction and rotation angle of the terminal device 1100, and the gyroscope sensor 1112 can cooperate with the acceleration sensor 1111 to collect the user's 3D actions on the terminal device 1100. The processor 1101 can implement the following functions based on the data collected by the gyroscope sensor 1112: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0125] The pressure sensor 1113 can be set on the side frame of the terminal device 1100 and / or the lower layer of the display screen 1105. When the pressure sensor 1113 is set on the side frame of the terminal device 1100, the user's holding signal of the terminal device 1100 can be detected, and the processor 1101 performs left and right hand recognition or shortcut operations according to the holding signal collected by the pressure sensor 1113. When the pressure sensor 1113 is set on the lower layer of the display screen 1105, the processor 1101 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1105. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0126] The optical sensor 1115 is used to collect the ambient light intensity. In one embodiment, the processor 1101 can control the display brightness of the display screen 1105 according to the ambient light intensity collected by the optical sensor 1115. Specifically, when the ambient light intensity is high, the display brightness of the display screen 1105 is increased; when the ambient light intensity is low, the display brightness of the display screen 1105 is reduced. In another embodiment, the processor 1101 can also dynamically adjust the shooting parameters of the camera assembly 1106 according to the ambient light intensity collected by the optical sensor 1115.

[0127] The proximity sensor 1116, also known as a distance sensor, is usually arranged on the front panel of the terminal device 1100. The proximity sensor 1116 is used to collect the distance between the user and the front of the terminal device 1100. In one embodiment, when the proximity sensor 1116 detects that the distance between the user and the front of the terminal device 1100 is gradually decreasing, the processor 1101 controls the display screen 1105 to switch from the screen-on state to the screen-off state; when the proximity sensor 1116 detects that the distance between the user and the front of the terminal device 1100 is gradually increasing, the processor 1101 controls the display screen 1105 to switch from the screen-off state to the screen-on state.

[0128] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation on the terminal device 1100, and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0129] Fig.10The schematic diagram of the structure of the server provided in the embodiment of the present application, the server 1200 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Unit, CPU) 1201 and one or more memories 1202, wherein the one or more memories 1202 store at least one program code, and the at least one program code is loaded and executed by the one or more processors 1201 to implement Figure 2 The method embodiment shown in the figure provides a method for training a trajectory generation model, or to implement the present application Figure 6 The method for generating a trajectory provided by the method embodiment shown. Of course, the server 1200 may also have components such as a wired or wireless network interface, a keyboard, and an input / output interface for input and output, and the server 1200 may also include other components for realizing device functions, which will not be described in detail here.

[0130] In an exemplary embodiment, a computer-readable storage medium is also provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to enable a computer to implement the above Figure 2 The method embodiment shown in the figure provides a method for training a trajectory generation model, or to implement the present application Figure 6 The illustrated method embodiment provides a method for trajectory generation.

[0131] Optionally, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0132] In an exemplary embodiment, a computer program or a computer program product is also provided, wherein at least one computer instruction is stored in the computer program or the computer program product, and the at least one computer instruction is loaded and executed by a processor to enable a computer to implement the above Figure 2 The method embodiment shown in the figure provides a method for training a trajectory generation model, or to implement the present application Figure 6 The illustrated method embodiment provides a method for trajectory generation.

[0133] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the historical data and driving information of the autonomous driving vehicle involved in this application are all obtained with full authorization.

[0134] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0135] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a trajectory generation model, characterized in that: The method comprises: Acquire historical data of the autonomous driving vehicle, wherein the historical data includes obstacle information around the autonomous driving vehicle; Processing the historical data based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, and a modal prompt word vector and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent category information of the obstacle, and the modal prompt word vector is used to represent modal information of the obstacle; Based on the setting template of the semantic prompt word vector, at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector is selected for processing to obtain a semantic prompt word vector; The initial trajectory generation model is trained using the traffic environment vector and the semantic cue word vector to obtain a target trajectory generation model, and the target trajectory generation model is used to generate a motion prediction trajectory of the obstacle.

2. The method according to claim 1, characterized in that The historical data is processed based on the initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector, including: Extracting features from the historical data using the initial trajectory generation model to obtain initial features; Determining a semantic feature in the initial feature, encoding the semantic feature based on a first encoding unit of the initial trajectory generation model to obtain at least one of the unified prompt word vector, the category prompt word vector, and the modality prompt word vector; A traffic environment feature is determined in the initial feature, and a second encoding unit based on the initial trajectory generation model encodes the traffic environment feature to obtain the traffic environment vector.

3. The method according to claim 2, characterized in that After extracting features from the historical data using the initial trajectory generation model to obtain initial features, the method further includes: Determine the unified prompt word vector, the category prompt word vector and the modal prompt word vector to be learned based on the historical data, and generate a learning vector set based on the unified prompt word vector, the category prompt word vector and the modal prompt word vector to be learned; The first encoding unit based on the initial trajectory generation model encodes the semantic feature, including: encoding the semantic feature based on the first encoding unit of the initial trajectory generation model and the learning vector set.

4. The method according to claim 1, characterized in that The semantic prompt word vector includes the unified prompt word vector, the category prompt word vector and the modality prompt word vector. When there are multiple semantic prompt word vectors for the obstacle, the unified prompt word vector in the multiple semantic prompt word vectors is the same.

5. The method according to any one of claims 1 to 4, characterized in that: The method of training the initial trajectory generation model by using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model includes: Decoding the traffic environment vector and the semantic cue word vector using a decoding unit of the initial trajectory generation model to obtain an obstacle prediction trajectory; A loss value between the obstacle prediction trajectory and the actual trajectory is calculated, and parameters of the initial trajectory generation model are adjusted based on the loss value to obtain the target trajectory generation model, wherein the loss value of the target trajectory generation model is less than or equal to a reference loss value, and the loss value is used to characterize the degree of difference between the obstacle prediction trajectory and the actual trajectory.

6. The method according to claim 5, characterized in that The decoding unit using the initial trajectory generation model decodes the traffic environment vector and the semantic prompt word vector to obtain an obstacle prediction trajectory, including: Using the decoding unit to decode the traffic environment vector and the semantic cue word vector to obtain an initial obstacle prediction trajectory and a confidence level, wherein the confidence level is a probability that the obstacle will move to a corresponding mode in the future; The initial obstacle prediction trajectory is screened based on the confidence level to obtain the obstacle prediction trajectory, wherein the confidence level of the obstacle prediction trajectory is greater than or equal to a reference confidence level.

7. The method according to any one of claims 1 to 4, characterized in that: Before processing the historical data based on the initial trajectory generation model, the method further includes: The historical data is preprocessed to obtain preprocessed data, wherein the preprocessing includes at least one of standardizing the historical data, removing abnormal data, and determining a preset range of the autonomous driving vehicle.

8. A method for generating a trajectory, characterized in that: The method comprises: Acquire driving information of the autonomous driving vehicle, wherein the driving information includes obstacle information around the autonomous driving vehicle; The motion trajectory of the obstacle is predicted based on the target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the method described in any one of claims 1 to 7.

9. A training device for a trajectory generation model, characterized in that: The device comprises: An acquisition module, used to acquire historical data of the autonomous driving vehicle, wherein the historical data includes obstacle information around the autonomous driving vehicle; an encoding module, configured to process the historical data based on an initial trajectory generation model to obtain at least one of a unified prompt word vector, a category prompt word vector, a modal prompt word vector, and a traffic environment vector, wherein the unified prompt word vector is used to represent scene information, the category prompt word vector is used to represent category information of the obstacle, and the modal prompt word vector is used to represent modal information of the obstacle; A setting module, for selecting at least one of the unified prompt word vector, the category prompt word vector, and the modal prompt word vector for processing based on a setting template of the semantic prompt word vector to obtain a semantic prompt word vector; A training module is used to train the initial trajectory generation model using the traffic environment vector and the semantic prompt word vector to obtain a target trajectory generation model, and the target trajectory generation model is used to generate the motion prediction trajectory of the obstacle.

10. A trajectory generation device, characterized in that: The device comprises: A collection module, used to obtain driving information of the autonomous driving vehicle, wherein the driving information includes obstacle information around the autonomous driving vehicle; A generation module is used to predict the motion trajectory of the obstacle based on a target trajectory generation model and the driving information to obtain the predicted motion trajectory of the obstacle, wherein the target trajectory generation model is trained based on the method described in any one of claims 1 to 7.

Images