Driving track prediction method and automatic driving vehicle

Abstract

The embodiment of the invention provides a driving track prediction method and device and an automatic driving vehicle, and relates to the technical field of automatic driving. The accuracy of vehicle driving track prediction can be improved; the method comprises the following steps: acquiring environment data, wherein the environment data comprises data of a target vehicle; obtaining intention information of the target vehicle according to the environment data; according to the intention information, constructing an intention prompt word; and obtaining an intention driving track of the target vehicle according to the environment data and the intention prompt word.

Description

Technical field

[0001] The embodiments of the present application relate to the field of autonomous driving technology, and in particular to a driving trajectory prediction method and an autonomous driving vehicle. [Background Technology]

[0002] In complex traffic scenarios, autonomous vehicles must not only avoid static objects but also collisions with moving vehicles. Accurately predicting the trajectories of surrounding vehicles can help autonomous vehicles make timely and appropriate decisions to avoid obstacles. Therefore, predicting vehicle trajectories and ensuring their accuracy and diversity are currently hot topics in the autonomous driving field.

[0003] Existing technologies for predicting surrounding vehicle trajectories rely heavily on scene-related data, such as the positional relationship between the current vehicle and a still object, the distance between the still object and the current vehicle, the positional relationship between other vehicles and the current vehicle, the distance between other vehicles and the current vehicle, and the distribution of roads. However, due to the uncertainty inherent in real-world driving scenarios, different vehicles make different choices in the same scenario, making it insufficient to rely solely on scene data to predict the paths of surrounding vehicles. [Summary of the invention]

[0004] The embodiments of the present application provide a driving trajectory prediction method and an autonomous driving vehicle, which can improve the accuracy of vehicle driving trajectory prediction.

[0005] In a first aspect, an embodiment of the present application provides a driving trajectory prediction method, which is applied to an autonomous driving vehicle. The method includes: obtaining environmental data, wherein the environmental data includes data of a target vehicle; obtaining intention information of the target vehicle based on the environmental data; constructing intention prompt words based on the intention information; and obtaining the intended driving trajectory of the target vehicle based on the environmental data and the intention prompt words.

[0006] The driving trajectory prediction proposed in the embodiment of the present application analyzes the possible directions chosen by surrounding obstacle vehicles in the scene, determines the intention prompt words that represent the driving intention of the obstacle vehicle, and uses the intention prompt words to visualize the uncertain reactions of different vehicles to the scene; the intended driving trajectory of the target vehicle is generated according to the directionality of the intention prompt words, which conforms to the description range of the intention prompt words, avoids the interference of uncertainty brought by complex scenes, and improves the accuracy of the trajectory.

[0007] In one possible implementation, obtaining the intended driving trajectory of the target vehicle according to the environmental data and the intention prompt word includes:

[0008] obtaining an intended driving track of the target vehicle according to the environment data, the intention prompt word and regular anchor information, wherein the regular anchor information is related to an optional driving path of the target vehicle at a corresponding position at a current time.

[0009] In one possible implementation, the environment data includes geometry data and image data, the geometry data includes vehicle state data and traffic environment information of the target vehicle within one or more preset time periods before the current time, and the image data includes image information of the target vehicle.

[0010] In one possible implementation, the obtaining of the intention information of the target vehicle according to the environment data includes:

[0011] extracting geometry features from the geometry data;

[0012] extracting image features from the image data;

[0013] performing feature compression, splicing and full-link mapping on the geometry features and the image features to obtain the intention information.

[0014] In one possible implementation, the constructing of the intention prompt word according to the intention information includes:

[0015] setting a plurality of prompt words;

[0016] inputting the plurality of prompt words and binary features corresponding to the prompt words into a latent embedding structure to train a natural language embedding model for dimension reduction of the binary features to obtain the prompt words;

[0017] inputting the intention information into the natural language embedding model;

[0018] querying, in the natural language embedding model, an intention prompt word matching the intention information, the intention prompt word being in natural language and representing a vehicle driving state and a vehicle driving state change.

[0019] In one possible implementation, the obtaining of the intended driving track of the target vehicle according to the environment data, the intention prompt word and the regular anchor information includes:

[0020] obtaining intention anchor information according to the intention prompt word and the regular anchor information;

[0021] generating the intended driving track according to a path direction and a path turning indicated by the intention anchor information.

[0022] In one possible implementation, the intention driving trajectory is generated according to the path direction and path turning indicated by the intention anchor point information, including:

[0023] Each anchor point information in the intention anchor point information is obtained as a query factor based on a cross-attention structure of a decoder in a motion prediction architecture based on attention;

[0024] Valid data having a preset correlation degree with each query factor is queried in the environment data, wherein the environment data of the target vehicle is a value vector of the cross-attention structure, and the encoding of the environment data is a key vector of the cross-attention structure;

[0025] Each valid data is decoded by the decoder in the motion prediction architecture based on attention to obtain the intention driving trajectory.

[0026] In one possible implementation, the geometric features are extracted from the geometric data, including:

[0027] According to the occurrence time of the geometric data, time parameters corresponding to different geometric data are obtained;

[0028] According to the spatial position corresponding to the traffic environment information, spatial parameters corresponding to different geometric data are obtained;

[0029] The geometric data corresponding to different time parameters is encoded in the time dimension by using a decomposition attention structure, and the geometric data corresponding to different spatial parameters is encoded in the spatial dimension.

[0030] In one possible implementation, the image information of the target vehicle includes N historical frames of images of the target vehicle, and the image features are extracted from the image data, including:

[0031] The N historical frames of images of the target vehicle are input into an image feature encoder to output N one-dimensional features, the one-dimensional features being the image features, and the image feature encoder being Resnet-15.

[0032] In a second aspect, the embodiments of the present application provide an automatic driving intelligent vehicle, the automatic driving intelligent vehicle comprising a memory and a processor, the processor being configured to acquire and execute code in the memory to execute the driving trajectory prediction method provided in the first aspect.

[0033] It should be understood that the second aspect of the embodiments of the present application is consistent with the technical solutions of the first aspect of the embodiments of the present application, and the beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar, which will not be repeated.

DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. Obviously, the drawings described in the following description are only some embodiments of the present application, and all other drawings obtained by those of ordinary skill in the art without creative labor based on these drawings are within the protection scope of the present application.

[0035] Figure 1 is a schematic diagram of an application scenario of a driving trajectory prediction method of an embodiment of the present application;

[0036] Figure 2 is a step flowchart of the driving trajectory prediction method proposed by the embodiment of the present application;

[0037] Figure 3 is a schematic diagram of obstacle distribution of an example automatic driving vehicle driving scenario at a first time;

[0038] Figure 4 is a schematic diagram of obstacle distribution of an example automatic driving vehicle driving scenario at a second time;

[0039] Figure 5 is a schematic diagram of an example based on a decomposed attention structure to encode geometric data;

[0040] Figure 6 is a schematic diagram of the structure of an example intent decoder set by the present application;

[0041] Figure 7 is a schematic diagram of an example trajectory decoder using intent anchor point information to decode to generate a driving trajectory and a conventional driving trajectory;

[0042] Figure 8 is a flowchart of an example cross-attention structure decoding given by the embodiment of the present application;

[0043] Figure 9 is a flowchart of an example executing a driving trajectory prediction method to output a trajectory.

DETAILED DESCRIPTION

[0044] In order to better understand the technical solutions of the present application, the following will describe the embodiments of the present application in detail in combination with the drawings.

[0045] It should be clear that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the protection scope of the present application.

[0046] The terminology used in the embodiments of the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the specification. As used in the embodiments of the present application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0047] The target vehicle refers to an obstacle vehicle in a driving scene of an autonomous vehicle.

[0048] The driving trajectory prediction method proposed in the embodiments of the present application is applied to an autonomous vehicle, Figure 1 is a schematic diagram of an application scenario of the driving trajectory prediction method of the embodiments of the present application, as Figure 1 shown, the autonomous vehicle A driving around the road includes traffic lights, signs, railings, forward vehicles, pedestrians, reverse vehicles, piers and other obstacles; wherein the forward vehicles and the reverse vehicles can be identified as target vehicles.

[0049] The autonomous vehicle cooperates with artificial intelligence, visual computing, radar, monitoring devices and global positioning, etc., in the driving process, perceives road information and vehicle information through the perception layer, and calculates the perceived information through the decision layer, outputs the decision of the vehicle operation action, and then controls the vehicle to run according to the decision through the execution layer, to realize the automatic driving of the vehicle.

[0050] In an example, the autonomous vehicle A makes a decision as Figure 1 shown: driving route Sa, controlling the autonomous vehicle A to drive according to Sa, which can realize the accurate avoidance of the target vehicle B by the autonomous vehicle A in a complex traffic environment. The autonomous vehicle A executes the driving trajectory prediction method proposed in the embodiments of the present application according to the perception information, accurately predicts the driving trajectory Sb of the target vehicle B, and provides support for the autonomous vehicle A to make a reasonable immediate decision.

[0051] Figure 2 is a step flowchart of the driving trajectory prediction method proposed in the embodiments of the present application, as Figure 2 shown, the driving trajectory prediction method includes the following steps:

[0052] S1: obtaining environment data, wherein the environment data includes data of a target vehicle.

[0053] The environment data refers to data of a driving scene of an autonomous vehicle. When an obstacle vehicle that hinders the driving of the vehicle is identified, the obstacle vehicle can be determined as a target vehicle, and data of the target vehicle is collected, such as driving data and image data of the target vehicle, so as to obtain information for analyzing the target vehicle.

[0054] The autonomous vehicle can collect road information and obstacle information through a camera, ultrasonic radar, satellite positioning, inertial navigation, a vehicle networking network, and the like, and obtain environmental data based on the road information and the obstacle information.

[0055] S2: Obtain intention information of the target vehicle based on the environmental data.

[0056] The intention feature is used to represent planning of a driving trajectory of the target vehicle within a preset time period. For example, the intention information represents a possible driving direction, a driving action, a driving action combination, or a vehicle state change of the target vehicle at a future time.

[0057] The planning of the driving trajectory of the target vehicle within the preset time period can represent a reaction of the target vehicle to a current scene from a perspective of the autonomous vehicle. For example, the target vehicle makes a reaction of stopping running in response to a traffic signal indicating stopping, the target obstacle vehicle makes a reaction of switching lanes or turning in response to an obstacle appearing in a forward direction, and the like.

[0058] Performing step S2, the data of a driving scene of the autonomous vehicle and the data of the target vehicle are analyzed, and a reaction of the target vehicle to adapt to the scene at a future time is calculated, so as to avoid generating a driving trajectory that does not match a current environment. For example, the environment is a narrow one-way street, the intention information obtained by analyzing the environmental data includes information representing straight driving and right turning, and does not include information representing U-turn, so that the generated driving trajectory also does not include a U-turn trajectory, thereby avoiding generating a driving trajectory that does not match the current environment.

[0059] S3: Construct an intention prompt word based on the intention information.

[0060] The essence of constructing the intention prompt word is to describe a future driving trajectory of the target vehicle using a natural language. The intention prompt word can include a word or a word group.

[0061] For example, the intention prompt word can include: reversing, adjusting a posture, U-turn, lane changing, stopping, starting, turning, multi-point U-turn, and the like. The intention prompt word can also be: first reversing, adjusting a posture, and then advancing to complete a U-turn, or first slowing down and then multi-point U-turn.

[0062] Compared with a representation form of the intention feature designed by using computer rules, the intention prompt word as the intention feature has strong expandability and does not need to introduce additional rule information by manual design.

[0063] S4: Obtain an intended driving trajectory of the target vehicle based on the environmental data and the intention prompt word.

[0064] The embodiment of the present application constructs an intention prompt word expression result, adopts a natural language description of a future driving track of a target vehicle, and generates a driving track based on the natural language description. The process not only reflects that the driving track should be generated based on a driving intention in a series connection relationship, but also shows that the intention guides the generation of the track, so that the corresponding track has a better effect in the corresponding intention scene.

[0065] The above process analyzes possible selection directions of surrounding obstacle vehicles of an autonomous driving vehicle in a scene, determines an intention prompt word representing a driving intention of the obstacle vehicle, and visualizes uncertain reactions of different vehicles to the scene according to the intention prompt word. The intention driving track of the target vehicle is generated according to the directionality of the intention prompt word, which is within the description range of the intention prompt word, avoids the interference of uncertainty caused by a complex scene, and improves the accuracy of the track.

[0066] The embodiment of the present application also proposes an optional way of constructing an intention prompt word, which is implemented by performing the following sub-steps S31 to S34 in step S3:

[0067] S31: Set multiple prompt words.

[0068] The characteristics of vehicle driving are extracted in advance, and multiple prompt words are set to cover all possible driving rules of the vehicle. For example, the prompt words include but are not limited to: reversing, adjusting posture, turning around, changing lanes, stopping, starting, turning, multi-point turning, accelerating, decelerating, etc.

[0069] S32: Input the multiple prompt words and the binary features corresponding to the prompt words into a latent embedding structure to train a natural language embedding model for dimension reduction of the binary features to obtain the prompt words.

[0070] During the training process of the latent embedding structure, a sample pair of a specific prompt word and a binary feature corresponding to the specific prompt word can be formed, input into the latent embedding structure, and the latent embedding is trained.

[0071] The natural language embedding model obtained by training the latent embedding structure by using the above method is a latent embedding established based on the intention prompt word. Since the binary features corresponding to different intention prompt words are input during the training stage, the latent embedding can learn to identify the association between different specific prompt words and binary features, so as to train the latent embedding to identify the binary features of the specific prompt word and optimize the identified binary features, and then perform dimension reduction on the binary features to output the corresponding intention prompt word.

[0072] The natural language embedding model trained by the above process has the function of identifying the binary features of the input and querying the prompt words corresponding to the binary features.

[0073] Therefore, for the natural language embedding model as a whole, the result obtained after dimensionality reduction processing of the binary features of the input is equivalent to mapping the input binary features to the corresponding prompt words in the one-to-one correspondence between the binary features and the prompt words.

[0074] S33: input the intention information into the natural language embedding model.

[0075] S34: query the intention prompt word matching the intention information in the natural language embedding model, and the intention prompt word is expressed in natural language to describe the vehicle driving state and the change of the vehicle driving state.

[0076] In an example of the present application, the pre-set prompt words include reversing, adjusting posture, turning around, changing lanes, stopping, and starting, and the binary features corresponding to each of the set prompt words are obtained: a1, a2, a3, a4, and a5, obtaining sample pairs {reversing, a1}, {turning around, a2}, {changing lanes, a3}, {stopping, a4}, and {starting, a5}. The sample pairs are input into the latent embedding for training until the latent embedding can accurately identify a1, a2, a3, a4, and a5 and output the prompt words corresponding to each of a1, a2, a3, a4, and a5, determine that the latent embedding has the function of identifying the intention signal and querying the intention prompt word corresponding to the signal, and obtain the natural language embedding model.

[0077] In the process of applying the natural language embedding model to the driving trajectory prediction method proposed in the present application, the intention information is input into the natural language embedding model in step S3, and the intention prompt word is found by the natural language embedding model. The process of finding the intention prompt word by the natural language embedding model includes: the natural language embedding model performs dimensionality reduction processing on the intention features, maps to obtain the intention prompt word associated with the intention features, and outputs the intention prompt word corresponding to the intention features.

[0078] After obtaining the intention prompt word, in the process of step S4, the data associated with the intention prompt word can be extracted from the environment data according to the intention prompt word, so as to predict the intention driving trajectory according to the data associated with the intention prompt word:

[0079] For example, assuming that the intention prompt word is "slow down first and then turn around more", in order to ensure that the generated driving trajectory conforms to the description of "slow down first and then turn around more", in the process of predicting the driving trajectory, the corresponding data will be selected for feature extraction with bias, or some data that does not conform to "slow down first and then turn around more" will be ignored, so as to avoid the generation of driving trajectories such as "change lanes and then turn", "stop and then go forward", and generate driving trajectories that conform to the description of "slow down first and then turn around more".

[0080] Continuing the above example, performing S2 obtains intention information including a i , performing step S3 inputs a i into the natural language embedding model, the natural language embedding model finds the intention prompt word, and outputs the intention prompt word adjustment posture-turn around, performing step S4, forming an intention driving trajectory representing adjustment posture-turn around according to the intention prompt word, the formed driving estimate is associated with adjustment posture-turn around, avoiding the generation of driving trajectories that do not conform to the environment requirements, and ensuring the accuracy of the predicted driving trajectory.

[0081] The embodiment of the application also provides an implementation manner of obtaining environment data, and the environment data includes image data and geometric data. In addition to collecting image data at the current time to represent the position and state of the target vehicle in the current scene, different time point corresponding vehicle state data, scene parameter data, and different space position corresponding scene parameter data are also collected as geometric data to represent the change trend of the current scene with time by using different time point corresponding vehicle state data and scene parameter data, and to represent the change of scene information with different positions by using different space corresponding scene parameter data.

[0082] For example, the switching frequency of the traffic light information, the time of the road congestion state, the historical position change of the target obstacle vehicle, etc., can all be used as geometric data.

[0083] The image data includes image information of the target vehicle.

[0084] After the autonomous vehicle recognizes the target vehicle, the image information of the target obstacle vehicle can be collected through a camera, or the image information uploaded by a roadside device can be downloaded from a vehicle networking server to obtain the image data.

[0085] In order to further analyze the characteristics of the change of the driving state of the target vehicle with time, N historical frames of image data corresponding to different time points can be collected, wherein the time interval between each adjacent two time points is the same.

[0086] The geometric data includes vehicle state data and traffic environment information of the target vehicle in one or more preset time periods before the current time.

[0087] The vehicle state data can include vehicle driving speed, vehicle acceleration, etc., and the traffic environment information can include traffic signals, sign information, etc.

[0088] Therefore, the geometry data can refer to a data set of target vehicle related data in a certain dimension, which can be a time dimension and a space dimension; for example, the geometry data can be a set of speeds of the target vehicle at different time points, a set of signals corresponding to traffic lights at different times in the target vehicle driving scene, and a set of obstacle information corresponding to different positions in the target vehicle driving scene.

[0089] An example of the present application is to select a Wayformer (based on a motion prediction architecture) structure for driving trajectory prediction method implementation. The Wayformer structure is composed of a scene encoder and a decoder based on attention. The collected data is encoded, and then the encoded features are decoded to obtain the target calculation result.

[0090] An example of the present application is to combine the above features of the Wayformer structure with the requirements of driving trajectory prediction, and set the encoder of the Wayformer structure to an image encoder as an image feature encoder to encode image information to obtain image features. On the other hand, another encoder of the Wayformer structure is set to obtain a set feature encoder to encode geometry data to obtain geometry features. Another decoder of the Wayformer structure is set to obtain a trajectory decoder to decode geometry features and image features to form a driving trajectory.

[0091] Further combining the embodiments of the present application to construct an intention prompt word, the intention prompt word can be input into the above-mentioned trajectory decoder to obtain the intended driving trajectory of the target vehicle according to the environment data and the intention prompt word. Another decoder of the Wayformer structure can also be set to obtain an intention decoder to decode geometry features and image features to obtain intention information.

[0092] The process of encoding image information using the above-mentioned image encoder includes:

[0093] The image information of the target vehicle includes the historical N frames of images of the target vehicle, and the image features are extracted from the image data, including:

[0094] The historical N frames of images of the target vehicle are input into an image feature encoder to output N one-dimensional features, and the one-dimensional features are the image features. The image feature encoder is Resnet-15.

[0095] The process of encoding the geometry data by using the above-mentioned geometry encoder includes:

[0096] M11: Obtain time parameters corresponding to different geometry data according to the occurrence time of the geometry data.

[0097] M12: Obtain space parameters corresponding to different geometry data according to the space position corresponding to the traffic environment information.

[0098] Since the geometry data includes a data set in the time dimension and a data set in the space dimension, the geometry data can be regarded as the description information of different obstacles and roads in the current scene.

[0099] M13: Encode the geometry data corresponding to different time parameters in the time dimension and encode the geometry data corresponding to different space parameters in the space dimension by using the decomposition attention structure. In an example of the present application, the target vehicle information, the information of vehicles around the target vehicle, the traffic light information, and the surrounding map information corresponding to different times are collected by ultrasonic radar, inertial navigation, and Internet of Vehicles network devices. The obstacle type information, the speed information of the target obstacle vehicle, the obstacle position information, the traffic light display signal, the road type, the road direction, and the road position at times T1-t, T1-2t, …, and T1-nt are collected respectively, to obtain the time parameters corresponding to different geometry data. The position coordinates of the traffic environment information are collected by satellite positioning devices, to obtain the space parameters corresponding to different geometry data. The traffic environment information includes the traffic light display signal, the traffic light position information, the road type, the road position, and the obstacle type information.

[0100] The embodiments of the present application are directed to the characteristics that the geometry data has regularity and correlation in time and space, and encode the geometry data in the time dimension and the space dimension respectively, to mine the regularity of obstacles in time and space in the current scene.

[0101] Figure 3 is an obstacle distribution schematic diagram of an example automatic driving vehicle driving scene at a first time of the present application, Figure 4 is an obstacle distribution schematic diagram of an example automatic driving vehicle driving scene at a second time of the present application. Referring to Figure 3 , the scene includes obstacles and roads, and the roads include straight lanes, road intersections, turning lanes, and sidewalks, etc. The obstacles include obstacle vehicles, traffic signal lights, indicating roadblocks, and pedestrians, etc. The position of the traffic signal light a is (x4, y1), the position of the indicating roadblock a is (x1, y3), the position of the indicating roadblock b is (x1, y2), and the position of the pedestrian is (x1, y4). The position of the target obstacle vehicle is (x b , y b), the position of the obstacle vehicle a is (x b ,y c ), it can be seen that the geometric data collected for obstacles have regularity and correlation in space. Figure 3 and Figure 4 , the position of the target obstacle vehicle at the first time T1 is (x b ,y b ), the position at the second time T2 is (x b+1 ,y b+2 ), it can be seen that the geometric data collected for obstacles have regularity and correlation in time.

[0102] In one example, the geometric data can be represented as: T1-{M1}, T2-{M2}...T N -{M N}, where M N Represents the geometric data set collected at time N. The elements contained in M ​​may include the specific values ​​of the target obstacle vehicle speed, target obstacle vehicle location information, obstacle vehicle a's location, obstacle vehicle a's speed, traffic light display signal, traffic light location information, road type through lane, and other information.

[0103] According to the time of collecting the above geometric data, the time parameters corresponding to different geometric data can be obtained, including: T1, T2...T N .

[0104] In this embodiment, separate geometric feature encoders are configured for different geometric data. The geometric data is pre-classified based on the scene parameter data corresponding to the geometric data, or based on the obstructing vehicle corresponding to the geometric data, and the classified geometric data is input into different geometric feature encoders. For example, a target obstructing vehicle and a traffic light each have different geometric feature encoders, and the geometric data corresponding to the target obstructing vehicle is input into the target obstructing vehicle's geometric feature encoder.

[0105] The geometric data can be expressed as: T1-{K1(obstacle vehicle a)}, T2-{K2(obstacle vehicle a)}...T N -{K N (obstacle vehicle a)}, where the elements contained in the set K are the positions of the obstacle vehicle a at different times, the speeds of the obstacle vehicle a at different times, etc. For example, T1-{k1(traffic light display signal)}, T2-{k2(traffic light display signal)}...T N -{k N (traffic light display signal)}, where the elements contained in the set k are the display signals of the traffic light corresponding to different moments.

[0106] The above-mentioned collected geometric data all carry respective spatial positions, and therefore extracting position information corresponding to the geometric data can obtain spatial parameters corresponding to the scene parameter data and spatial parameters of the target obstacle vehicle.

[0107] Reference Figure 3 , the position (x1, y1) of the traffic signal lamp a, the position (x1, y3) of the roadblock a, and the position (x b , y c ) of the obstacle vehicle a are extracted to obtain spatial parameters corresponding to the traffic signal lamp a: (x1, y1), (x1, y3), and (x b , y c ); and the position (x b , y b ) of the target obstacle vehicle is extracted to obtain spatial parameters corresponding to the target obstacle vehicle: (x b , y b ).

[0108] After the above processing, spatial parameters corresponding to different geometric data and time parameters corresponding to different geometric data are obtained, and therefore the geometric data can be encoded in the time dimension and the spatial dimension respectively to obtain geometric features.

[0109] Figure 5 is an example schematic diagram of encoding geometric data based on a decomposition attention structure according to the present application. After the geometric data is input into the encoder, self-attention operations are separately performed in the spatial dimension and the time dimension, thereby decomposing the attention in the time dimension and the spatial dimension. The encoding of the geometric data in the time dimension and the encoding of the geometric data in the spatial dimension are cross-performed, so that the features can be adjusted by self-attention weights in all dimensions.

[0110] Compared with directly encoding the geometric data by using a traditional self-attention structure, the encoder will superimpose possible dimensions in the learning process. Assuming that S P represents the spatial dimension, T P represents the time dimension, and the self-attention structure does not perform attention calculation in the unknown attention dimension, the attention weights learned in the previous process will be superimposed. The calculation amount of the self-attention structure originally performing self-attention calculation is O(S P 2 ×T P 2 ). Based on the premise that the geometric data has a correlation relationship in the spatial position and the time, the encoder of the decomposition attention structure is set in the embodiments of the present application, the attention is decomposed into the known time dimension and the known spatial dimension, and the calculation amount is O(S P 2 ×T P 2) to O(S P 2 +T P 2 )。

[0111] The spatial dimension encoder and the time dimension encoder each comprise a multi-head attention mechanism layer (Multi-head Attention), a summation and normalization layer (Add&Norm), and a multi-layer perceptron (MLP) network layer.

[0112] After obtaining the environment data through other embodiments of the present application, the step S2 of obtaining the intention information of the target vehicle according to the environment data can be performed through the following sub-steps:

[0113] S21: extracting geometric features from the geometric data.

[0114] S22: extracting image features from the image data.

[0115] The encoder of the Wayformer structure can also be set in different ways to obtain an image encoder and a geometric encoder, which respectively encode the geometric data to obtain geometric features and encode the image data to obtain image features.

[0116] S23: performing feature compression, splicing, and full link mapping on the geometric features and the image features to obtain the intention information.

[0117] The intention decoder obtained by setting a decoder of the Wayformer structure is used in the embodiments of the present application to decode the geometric features and the image features, and the geometric features and the image features are subjected to feature compression, splicing, and full link mapping to obtain the intention information.

[0118] Figure 6 is a structural schematic diagram of an intention decoder of an example setting of the present application, as shown in Figure 6 The geometric features obtained by encoding the geometric data are input into a multi-layer perceptron (MLP) network 1, the multi-layer perceptron network 1 outputs a one-dimensional real vector R1 1XD to a splicing layer C, the image features obtained by encoding the image data are input into a multi-layer perceptron network 2, the multi-layer perceptron network 2 outputs a one-dimensional real vector R2 1XD to the splicing layer C, R1 1XD and R2 1XD are spliced to obtain a two-dimensional real vector R3 2XD , R3 2XD is input into a multi-layer perceptron network 3, and the multi-layer perceptron network 3 performs full link mapping on the two-dimensional real vector R3 2XDFull link mapping is performed, and an intention feature is output.

[0119] The embodiment of the present application also provides an implementation manner of obtaining the intended driving track of the target vehicle according to the environment data and the intention prompt word. Based on the usage mode of the conventional anchor point in the general driving track generation scheme, the intention prompt word is combined with the conventional anchor point information in view of the role of the conventional anchor point in determining the overall direction of the driving track, the anchor point information is expanded by the intention prompt word, and then the driving track is guided to be generated according to the intention by the anchor point information.

[0120] The step S4 of obtaining the intended driving track of the target vehicle according to the environment data and the intention prompt word can be implemented by the following specific steps:

[0121] The intended driving track of the target vehicle is obtained according to the environment data, the intention prompt word and the conventional anchor point information, wherein the conventional anchor point information is related to the optional driving path of the target vehicle at the corresponding position at the current time.

[0122] For example, there are four roads at a certain traffic intersection, and the four roads correspond to four drivable paths respectively. The finally generated driving track must reach one of the four drivable paths.

[0123] Based on the role of the conventional anchor point in predicting the driving track, the embodiment of the present application combines the intention prompt word with the conventional anchor point information, and then analyzes the environment data to generate the driving track, thereby further ensuring the accuracy of the predicted driving track.

[0124] Further, the embodiment of the present application also provides an implementation manner of combining the intention prompt word with the conventional anchor point information. The step of obtaining the intended driving track of the target vehicle according to the environment data, the intention prompt word and the conventional anchor point information comprises:

[0125] M22: determining the original anchor point information according to the optional driving path.

[0126] M23: obtaining the intended anchor point information according to the intention prompt word and the conventional anchor point information.

[0127] The optional driving path fuses the intention prompt word to give multiple path selections, and gives multiple constraint conditions for path formation, so that the path formation is more accurate.

[0128] M24: generating the intended driving track according to the path direction and path turning indicated by the intended anchor point information.

[0129] An example of the present application sets up a decoder constructed with a cross-attention structure for the Wayformer structure to obtain a trajectory decoder; sets up two encoders of the Wayformer structure to obtain an image encoder and a geometry encoder, encodes image data in the environment data with the image encoder to obtain image features, encodes geometry data in the environment data with the geometry encoder to obtain geometry features, inputs the geometry features, the image features, and the intention prompt word into the trajectory decoder, and then analyzes and decodes the intention prompt word, the geometry features, and the image features with the trajectory decoder to generate the intention driving trajectory.

[0130] The execution of M24: "generating the intention driving trajectory according to the path direction and path turning indicated by the intention anchor point information" can be performed by running the following sub-steps:

[0131] M241: obtaining each anchor point information in the intention anchor point information as a query factor based on the cross-attention structure of the decoder in the attention motion prediction architecture.

[0132] M242: querying valid data with a preset correlation degree from each query factor in the environment data, wherein the environment data of the target vehicle is a value vector of the cross-attention structure, and the encoding of the environment data is a key vector of the cross-attention structure.

[0133] M243: decoding each valid data with the decoder in the attention motion prediction architecture to obtain the intention driving trajectory. Figure 7 is a comparison diagram of the intention anchor point decoding trajectory generated by an example trajectory decoder of the present application and a conventional driving trajectory; as Figure 7 shown, the optional driving path includes road 1 and road 2, the image features and the geometry features input into the trajectory decoder are decoded with the original anchor point information, only constraints in the L1 and L2 directions can be given, and the driving trajectory is adjusted in a small range based on the right turn L1 and the U-turn L2.

[0134] The intention prompt word input into the trajectory decoder includes: multi-point U-turn, left turn first and right turn later, the intention anchor point information obtained by fusing the original anchor point information and the intention prompt word can also give constraints in the multi-point U-turn L3 and the left turn first and right turn later L4 directions, and more diversified path options: L3 and L4, in addition to L1 and L2, are given, the left turn first and right turn later driving trajectory direction is given based on the right turn driving trajectory direction under the constraint of the intention prompt word, and is more accurate.

[0135] The embodiment of the present application also gives an implementation manner of decoding a plurality of valid data with an intention anchor point information by a trajectory decoder;

[0136] The trajectory decoder adopts the cross attention structure. Figure 8 This is a flowchart of a cross-attention structure decoding process given in the embodiment of the present application, refer to Figure 8 ,A variety of valid data include geometric features and image features. ,The geometric features and image features are spliced ​​at the splicing layer and ,input into the cross-attention structure. ,After the conventional anchor information and intent prompt words are ,input into the cross-attention structure, the conventional anchor points and ,intent prompt words are combined to obtain the intent anchor information.

[0137] The intention anchor point information provides a sequential query factor Q (Query) of the cross-attention structure, and multiple valid data provide another sequential key-value pair K, V (Key, Value) of the cross-attention structure. The valid data is a value vector (Value, V), and each valid data is encoded as a key vector (Key, K). The intention anchor point information is used to query multiple valid data for decoding to form a driving trajectory.

[0138] S81: Use each anchor point in the intention anchor point as the query factor of the cross-attention structure.

[0139] Valid data and intent prompt words are input into the cross-attention structure. During the decoding process of the cross-attention structure, the intent prompt words focus on the valid data, capture the correlation between the intent anchor point information and the image features and geometric features, thereby fusing the intent anchor point information with the image features and geometric features, and making corrections based on the driving direction given by the intent anchor point information to form a driving trajectory that can avoid various obstacles and conform to the characteristics of the scene.

[0140] S82: Searching for associated valid data having a preset correlation degree with each query factor among the multiple valid data.

[0141] Searching for relevant valid data having a preset correlation degree with each query factor in the plurality of valid data may be achieved by the following process:

[0142] S821: Calculating weight scores associated with different valid data for each query factor;

[0143] S822: Obtain associated valid data corresponding to each query factor according to the weight scores of the single query factor and different valid data.

[0144] by Figure 7 For example, given the L3 direction intention anchor point information query valid data, the associated valid data in the L3 direction is obtained, so as to optimize the details of the L3 direction and output the driving trajectory.

[0145] S823: The trajectory decoder decodes the associated valid data of each query factor to obtain the driving trajectory.

[0146] The autonomous driving vehicle executes the above steps to obtain the intention prompt word of the target vehicle, and then uses the trajectory decoder to calculate the intention prompt word. In the process of generating the driving trajectory, the autonomous driving vehicle can predict the driving trajectory based on the intention information in addition to the scene information and vehicle data, which limits the uncertainty of the target vehicle and improves the accuracy of the prediction.

[0147] For example, in response to complex scenarios, the target vehicle may make a variety of responses, such as driving in a straight line, turning, steering, changing lanes, continuous steering, starting after stopping, and pulling over after changing lanes, and the possible responses of the target vehicle are unknown.

[0148] Therefore, there is a certain deviation in predicting the target obstacle vehicle's driving trajectory based only on scene information or vehicle information; the autonomous driving vehicle predicts the target vehicle's intention prompt words in advance, which has a constraining effect on the possible reactions of the above-mentioned target vehicle. The effective intention data is extracted based on the environmental data with a higher probability of occurrence. The process of the trajectory decoder generating trajectory features, under the influence of the intention prompt words, reduces the interference of reaction-related trajectories with a lower probability of occurrence, and has higher accuracy.

[0149] The embodiment of the present application uses a cross-attention structure to construct a trajectory decoder, fuses the intent prompt words and the original anchor point information, and obtains the intent anchor point that can provide more accurate driving directions. There is no need to introduce other calculation methods to process the intent features and other features, such as the relationship between image features. The intent anchor point information is directly used as the query factor to query the features related to itself in the image features and geometric features, and decode to obtain a more diverse and accurate driving trajectory.

[0150] Figure 9 This is a flow chart of an example of executing the driving trajectory prediction method to output the trajectory of the present application, such as Figure 9 As shown, this example not only outputs the driving trajectory formed based on the intention prompt word, but also outputs the regular trajectory without considering the intention prompt word prediction, providing more reference for the autonomous driving vehicle to plan the driving route.

[0151] This example sets up a geometry encoder, an image encoder, a trajectory decoder, and an intent decoder. Executing the trajectory prediction method requires running the following process:

[0152] D101: Collects image data of the target vehicle, as well as geometric data of the target vehicle and the scene it is in. The geometric data is input into a geometry encoder, which encodes the geometric data to obtain geometric features. The image data is input into an image encoder, which encodes the image data to obtain image features.

[0153] D102: geometry features and image features are input into an intention decoder and a trajectory decoder respectively.

[0154] D103: the intention decoder decodes the geometry features and the image features: decoding in the time dimension and the space dimension respectively, to obtain intention information.

[0155] D104: the intention information is input into a natural language embedding model, the natural language embedding model queries to obtain an intention prompt word corresponding to the intention information, and the target intention prompt word is input into the trajectory decoder.

[0156] D105: the trajectory decoder fuses the intention prompt word on the basis of the conventional anchor point information to obtain intention anchor point information, and calculates the geometry features and the image features by using the intention anchor point information to form a driving trajectory, and outputs the driving trajectory and the conventional trajectory.

[0157] An automatic driving intelligent vehicle is provided, which comprises a memory and a processor, the processor is used for acquiring and executing code in the memory to execute Figures 1 to 9 A corresponding driving trajectory prediction method.

[0158] The above describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different than the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous or possible.

[0159] In the description of the embodiments of the present application, the description referring to the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present specification. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0160] In addition, the terms "first", "second", etc. are used herein only to describe different instances, and do not imply or suggest relative importance or a number of the technical features indicated. Thus, the features defined as "first", "second", etc. can explicitly or implicitly include at least one of the features. In the description of the present specification, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise explicitly specified.

[0161] Any process or method descriptions or blocks in flow charts herein, or described elsewhere in this specification, can be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing the specified logical functions or steps, and alternate implementations are possible. The preferred embodiments of this specification encompass additional implementations in which the functions described in the processes or methods are performed in different orders, including substantially simultaneously, or in reverse order, or in an order that is different from the order shown or discussed.

[0162] Depending on the context, the word "if" as used herein can be interpreted to mean "when" or "while" or "in response to determining" or "in response to detecting." Similarly, the phrase "if it is determined" or "if a stated condition or event occurs" can be interpreted to mean " whenever it is determined" or "in response to determining" or "whenever a stated condition or event occurs" or "in response to detecting a stated condition or event."

[0163] In several embodiments provided in the present specification, it should be understood that the disclosed system, device and method can be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division of the units is only a logical function division, and in actual implementation, another division manner can be adopted, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0164] In addition, each functional unit in each embodiment of the present specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional units.

[0165] The above only describes the preferred embodiments of the present specification, and does not limit the present specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the protection scope of the present specification.

Claims

1. A driving trajectory prediction method, characterized in that: The method comprises: Acquiring environmental data, the environmental data including data of a target vehicle; Obtaining intention information of the target vehicle based on the environmental data; Constructing an intention prompt word according to the intention information; The intended driving trajectory of the target vehicle is obtained according to the environmental data and the intention prompt word.

2. The method according to claim 1, characterized in that The step of obtaining the intended driving trajectory of the target vehicle according to the environmental data and the intention prompt word includes: The intended driving trajectory of the target vehicle is obtained according to the environmental data, the intention prompt word and conventional anchor point information, wherein the conventional anchor point information is related to the optional driving path of the target vehicle at the corresponding position at the current time.

3. The method according to claim 1, characterized in that The environmental data includes geometric data and image data. The geometric data includes vehicle status data and traffic environment information of the target vehicle in one or more preset time periods before the current time; the image data includes image information of the target vehicle.

4. The method according to claim 3, characterized in that The obtaining, based on the environmental data, the intention information of the target vehicle includes: extracting geometric features from the geometric data; extracting image features from the image data; The geometric features and the image features are subjected to feature compression, splicing, and full-link mapping to obtain the intent information.

5. The method according to any one of claims 1 to 4, characterized in that: The step of constructing an intention prompt word according to the intention information includes: Set multiple prompt words; Inputting the plurality of prompt words and the binary features corresponding to the prompt words into a latent embedding structure to train a natural language embedding model for reducing the dimension of the binary features to obtain the prompt words; Inputting the intent information into the natural language embedding model; Intention prompt words matching the intention information are searched in the natural language embedding model, where the intention prompt words are expressed in natural language to describe the vehicle driving state and the change in the vehicle driving state.

6. The method according to claim 2, characterized in that The step of obtaining the intended driving trajectory of the target vehicle according to the environmental data, the intention prompt word, and conventional anchor point information includes: Obtaining intention anchor information according to the intention prompt word and the conventional anchor information; The intended driving trajectory is generated according to the path direction and path turning indicated by the intended anchor point information.

7. The method according to claim 6, characterized in that Generating the intended driving trajectory according to the path direction and path turning indicated by the intended anchor point information includes: Obtaining each anchor point information in the intention anchor point information as a query factor of a cross-attention structure of a decoder in an attention motion prediction architecture; Searching the environmental data for valid data having a preset correlation with each query factor, wherein the environmental data of the target vehicle serves as a value vector of the cross-attention structure, and an encoding of the environmental data serves as a key vector of the cross-attention structure; The decoder in the attention-based motion prediction architecture is used to decode each valid data to obtain the intended driving trajectory.

8. The method according to claim 4, characterized in that The extracting geometric features from the geometric data includes: According to the occurrence time of the geometric data, obtaining time parameters corresponding to different geometric data; Obtaining spatial parameters corresponding to different geometric data according to the spatial position corresponding to the traffic environment information; A decomposed attention structure is used to encode the geometric data corresponding to different temporal parameters in the temporal dimension, and to encode the geometric data corresponding to different spatial parameters in the spatial dimension.

9. The method according to claim 4, characterized in that The image information of the target vehicle includes N historical frames of images of the target vehicle, and extracting image features from the image data includes: Input N historical frames of images of the target vehicle into an image feature encoder to output N one-dimensional features, where the one-dimensional features are the image features, and the image feature encoder is Resnet-15.

10. An autonomous driving intelligent vehicle, comprising a memory and a processor, characterized in that: The processor is configured to obtain and execute the code in the memory to perform the method according to any one of claims 1 to 9.

Images