A decision-oriented intelligent vehicle trajectory prediction method and prediction model

Abstract

The present invention discloses a decision-oriented intelligent vehicle trajectory prediction method and prediction model, which includes the following two core contents: trajectory prediction based on time series and trajectory prediction based on conditions. By introducing time series to construct a prediction model serving multi-period trajectory prediction tasks, temporal consistency within each prediction period is achieved, including information reuse of historical periods and prediction modal continuation of future periods; by introducing a conditional prediction model, scene consistency of trajectory prediction is achieved, that is, scene-level multimodal trajectory prediction is achieved. In response to the problem that the existing mainstream trajectory prediction model constructs the prediction algorithm as an independent task, which is single, closed, and cannot be unified with downstream decision-making and planning tasks, the present invention constructs a trajectory prediction model that introduces time series and conditional prediction, and achieves the unification of trajectory prediction tasks and downstream decision-making and planning tasks in the algorithm process.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and specifically, to a decision-making-oriented intelligent vehicle trajectory prediction method and prediction model. Background Art

[0002] The safe and autonomous operation of intelligent vehicles in a driving environment relies on the coordinated work of various modules. The prediction module and the decision-making planning module are crucial components for achieving environmental awareness and decision-making. As an upstream task of the decision-making module, the prediction module needs to predict the behavioral modes of the intelligent vehicle's neighboring agents at a certain frequency and transmit the results to the decision-making module. This is a spatiotemporally continuous task. However, existing mainstream trajectory prediction algorithms design their models based on treating the prediction task as an isolated task. The design of the prediction model and the representation of the prediction results are not centered on the decision-making task. They lack a consistent coupling strategy tailored to decision-making requirements, ignore the correlation between multi-period trajectory prediction tasks, and thus reduce the task matching between prediction and decision-making. Summary of the Invention

[0003] The purpose of the present invention is to solve the deficiencies in the prior art. This application provides a prediction model for a decision-making intelligent vehicle trajectory prediction method. First, a relative state quantity parameterization method based on time series decomposition is proposed to decouple state parameters from absolute spatial coordinate information. By introducing time series, a prediction model serving multi-period trajectory prediction tasks is constructed, and a multi-target spatiotemporal relative interaction behavior encoding within the observation interval is constructed to achieve temporal consistency within each prediction period, including information reuse of historical periods and prediction modal continuation of future periods. Secondly, the scene consistency of trajectory prediction is achieved by introducing a conditional prediction model, and the causal relationship between edge distribution and conditional distribution under large-scale scenes is obtained, thereby realizing scene-level multimodal trajectory prediction. The above invention content realizes the unification of the algorithm process between trajectory prediction tasks and downstream decision-making planning tasks, reduces the overall computing resource consumption, and improves the stability of decision result output.

[0004] Technical solution: The present invention includes the following two parts

[0005] S1. According to the first aspect of the present disclosure, a multi-period trajectory prediction method using a time series is provided, comprising:

[0006] S11. Constructing a relatively invariant parametric encoding method for scene elements, including:

[0007] Obtain the original state of various elements in the driving scene, including intelligent body elements, road map elements, etc.

[0008] The agent element will obtain the trajectory state quantity under the historical observation period, such as position, speed, heading angle, category, etc. relative to the world coordinate system. Taking the i-th agent in the driving scene as an example, its coordinates relative to the world coordinate system at time t are The heading angle is recorded as Speed ​​is recorded as

[0009] Road map elements will obtain their positions relative to the world coordinate system and related semantic information, such as lane lines, lane centerlines, crosswalks, etc.

[0010] For the agent, the spatial position of the effective agent in the driving scene relative to the world coordinate system is obtained in units of one time step, and an independent space-time coordinate system is constructed for each time step of each agent. Taking the i-th agent at time t as an example, its local space-time coordinate system will be composed of time t, space coordinates and heading angle Sure;

[0011] Align the state information at other moments to the corresponding space-time coordinate system. This step can convert the absolute position information into a relative vector to avoid the change of the reference coordinate system caused by the change of the encoding subject. The relative information includes relative position, relative heading angle, etc. Taking the i-th agent at time t as an example, its absolute space-time state information is Select another element, the absolute spatiotemporal state information of the jth agent at time s is Then the relative position will be described as The relative heading angle is described as

[0012] The agent state encoding module is constructed using a multi-layer perceptron to encode the relative state of the agent at each time step, thereby obtaining the relative state embedding vector of each agent at each time step, which is recorded as Where A is the number of agents, T is the time step span, and D is the encoding dimension;

[0013] For road map elements, since road map elements do not change over time, it is only necessary to construct a spatial coordinate system that is unchanged relative to a single road map element within multiple prediction cycles;

[0014] Use a line segment group to represent a single road map element, obtain the starting and ending points of the line segment group, and use the coordinates of the starting point as the spatial position of the current road map element in the world coordinate system; connect the starting point and the ending point, and use the vector direction from the starting point to the ending point as the spatial direction of the current road map element in the world coordinate system;

[0015] Construct an independent spatial coordinate system for each road map element according to the position and direction of the currently selected road map element;

[0016] Align the state quantities of all constituent points in the currently selected road map element to the independent spatial coordinate system corresponding to the current road map element, calculate the relative coordinates of each point relative to the independent spatial coordinate system, and use them as relative state quantities for the subsequent state quantity encoding process;

[0017] A road map element encoding module is constructed using a multi-layer perceptron to encode the relative state of each road map element in the above driving scenario, thereby obtaining the state embedding vector of each road map element constructed for its own local coordinate system but unchanged relative to the global world coordinate system, and recorded as Where M is the number of road map elements and D is the encoding dimension;

[0018] After obtaining the relative state embedding codes of the agent and road map elements respectively, the relative position and posture information between elements will be lost because the relative state code of each element in the driving scene is obtained based on its own local independent coordinate system.

[0019] To accurately model the relative position and posture of elements, compensate for the relative position and posture information between elements in driving scenarios, and determine the interaction relationship between elements, a spatial relative posture encoding module is introduced, which uses a multi-layer perceptron to encode and represent the relative relationship between elements of two independent intelligent agents.

[0020] The spatial relative pose encoding module will model the relative pose relationship between any two interacting elements in the driving scene, such as between intelligent agents, between intelligent agents and road map elements, and between road map elements; the two interacting elements will be marked as source elements and target elements respectively, and the relative position and relative pose of the source element relative to the target element will be obtained. A parameter-independent relative quantity encoding module will be constructed to encode the relative position and pose, thereby obtaining the encoding result of the relative pose relationship between the two interacting elements.

[0021] The above-mentioned S11 link solves the problem of repeated encoding of state information of the same element in the driving scene due to changes in the scene center in multi-cycle prediction tasks, which needs to be repeated in each prediction cycle. It realizes the reuse of the encoding results of a single scene element in the driving scene and reduces the consumption of computing resources.

[0022] S12. Build a historical encoding feature memory repository to achieve the continuation and preservation of key historical state information, including:

[0023] Construct a memory repository that caches historical trajectory encoding results for a fixed number of prediction cycles for a certain number of agents in the driving scenario. It also updates and manages the internally stored information through a sliding window. When an agent disappears for a period of time, a new agent will be replaced. At the same time, for agents with longer observation times, unimportant early features will be replaced with important new spatiotemporal features.

[0024] Memory repository initialization method: At the initial stage of construction, if the number of observed agents is less than the maintenance limit of the repository, the perceived adjacent agents will fill the memory repository in sequence until the maintenance limit is reached; the state embedding encoding results of the agents in different observation cycles will be used to initialize the memory repository;

[0025] After initialization, the memory repository will maintain the state embedding encoding result sequence of the intelligent agents that have not died in the continuous space-time within the sliding window period;

[0026] When a new prediction task begins, the agent's encoding results will be integrated with other environmental data within the driving scenario and obtain prior information from the historical encoding result sequence stored in the memory repository, including:

[0027] First, the attention scoring mechanism is used to calculate the correlation between the agent encoding results in the current prediction cycle and the encoding sequence stored in the historical prediction cycle, and the correlation mechanism and similarity scoring mechanism are used to maintain the data fusion and update of the memory repository. Memory = {a i}, stores M groups of historical frame embedding codes, where a i To achieve the historical frame embedding coding of the target agent and road map element information fusion based on the attention mechanism, when the embedding coding of the new observation is obtained k Then, the cosine distance between the feature vectors is used to calculate the similarity score between the current code and the historical code. The calculation formula is as follows:

[0028]

[0029] Based on the similarity, the encoding result sequence under the historical prediction period is used to update the encoding result of the current prediction period. The historical encoding with the highest similarity will be used as prior information to enrich the current feature encoding and decoding process, and the memory storage library will be updated at the same time.

[0030] Constructing a similarity evaluation metric involves calculating the similarity between the agent's encoding results and the information stored in the memory bank during the current prediction cycle. This is used to determine whether the memory bank should be updated with the results of the current prediction cycle. If the similarity is low, the memory bank is updated; otherwise, it is not updated.

[0031] The agent codes stored in the memory repository are divided into two categories: one category, representing codes closer to the current prediction period, with fast data updates and denser storage; the other category, representing codes farther from the current prediction period, with slower data updates and less storage. The design of this classification structure helps achieve a balance between storage and computational efficiency and data dissemination accuracy.

[0032] S13. Implementing the transfer of prior information of trajectory prediction modal results, including:

[0033] At the beginning of the current prediction cycle, obtain the historical trajectory state of the intelligent agent in the driving scene;

[0034] Based on the agent tracking information in the adjacent cycles, the agent state data appearing in the current prediction cycle is extracted from the previous prediction cycle, including historical trajectories and multimodal prediction trajectories;

[0035] Calculate the relative movement of each agent in the current cycle relative to the previous cycle;

[0036] Select a specific number of reasonable modes for the target agent based on the mode prediction probability and relative motion in the previous prediction cycle;

[0037] Construct a behavioral modality encoding network, which encodes the predicted behavioral modality that best matches the current driving behavior trajectory of each agent in the previous prediction cycle, thereby obtaining prior information on the predicted modality result and passing it into the encoding process of the current prediction cycle;

[0038] The predicted modality prior information will be used in the multimodal trajectory prediction of the agent in the current prediction cycle. The prior information will be fused with the embedded feature encoding of the agent in the current prediction cycle through a neural network, and the loss function will be designed to guide the target agent to generate a behavioral trajectory similar to the predicted modality of the previous cycle.

[0039] S2. According to the second aspect of the present disclosure, a scene-level multimodal trajectory prediction based on conditional prediction is provided, comprising:

[0040] S21. Construction of driving scenario interaction diagram, including:

[0041] Obtaining state information of elements in the driving scene around the smart car, including the position, speed, and semantic attributes of the intelligent body elements, and the position and semantic attributes of the road map elements;

[0042] A spatiotemporal interaction graph is introduced to represent the driving scenario. Each valid adjacent agent in the driving scenario will serve as a node of the interaction graph, and the edges between the nodes are bidirectional edges.

[0043] In the initial state, the spatiotemporal interaction graph will be initialized as a fully connected graph; the node encoding features of the spatiotemporal interaction graph will be initialized using the agent state embedding code, and the edge features between nodes will use a neural network to encode the relative state of the source node relative to the target node, and initialized with the encoding results;

[0044] The interaction relationship between agents is determined using a spatiotemporal interaction graph. The relative relationship between nodes is decoded based on the spatiotemporal interaction graph nodes and edge state quantities. For nodes i and j, if the interaction relationship between node j and node i is determined, the embedded feature codes of agents i and j, as well as the j->i edge code, are first obtained. A neural network, such as a multi-layer perceptron with a maximum pooling operation, is then used to fuse these three sets of codes to obtain the feature code of the node interaction relationship. This feature code will be used to decode the interaction relationship.

[0045] Three types of node interaction relationships are defined: the source node affects the target node's behavior trajectory, the target node affects the source node's behavior trajectory, and the source and target nodes do not affect each other. A multi-layer perceptron network is used to decode the interaction relationship feature encoding obtained above and output the probabilities of the three types of node interaction relationships.

[0046] After completing the determination of the interaction relationships between all intelligent agents in the current driving scene, the fully connected spatiotemporal interaction graph can be decomposed into multiple groups of interaction relationship subgraphs according to the interaction relationship results between the intelligent agents to simplify the generation of trajectory prediction results under conditional prediction.

[0047] S22. Generation of scene-level multimodal trajectory prediction results, including:

[0048] The driving scenario interaction relationship graph generated in S21 based on the relative relationship between the internal nodes of the interaction subset will guide the conditional prediction in the trajectory decoding module;

[0049] Based on the interaction relationship diagram, the prediction levels of the intelligent agents in the driving scenario are divided into groups from high to low levels. The division is based on the yield relationship between the low level and the high level. In the decoding process, the high level to low level intelligent agents will be decoded in sequence. The trajectory of the low level intelligent agents will be decoded based on the characteristics of the high level intelligent agents.

[0050] The interaction relationship graph is represented by G = {V, E}, where V = [N] represents the nodes composed of N dynamic targets in the driving scene, e ij ∈E represents the directed edge between nodes i and j, and represents the interaction relationship between node i and node j. A multi-agent joint trajectory prediction method is adopted, and the joint distribution is decomposed into marginal distribution and conditional distribution for solution. The decomposition formula is as follows:

[0051]

[0052] Where P(Y|X,M) is the joint predicted trajectory distribution under the driving scenario, X and Y are the historical trajectory and predicted trajectory results of the dynamic target respectively, and p G (i) represents the set of upper-level nodes of node i in the interaction graph;

[0053] When predicting the trajectory of the current-level agent, the agent will obtain the coded information of the trajectory prediction results of the previous-level agent in advance. The current-level agent will integrate the predicted trajectory coding information of the agent in the previous level with which it has interaction judgment according to the interaction relationship and update its own feature vector.

[0054] Based on the updated feature vector of the target agent, the trajectory decoding module is used to generate the predicted trajectory of the target agent contained in the current level to achieve conditional prediction;

[0055] The trajectory encoding module encodes the trajectory decoding results of the current layer to provide them for fusion in the next layer;

[0056] Repeat the above steps until decoding reaches the lowest level.

[0057] The highest-level agent in the driving scenario interaction diagram will use edge prediction to generate predicted trajectories through the trajectory decoding module; the lowest-level agent will not need to perform predicted trajectory encoding;

[0058] To generate scene-level multimodal prediction results, S21 will generate a specific number of groups of interaction relationship graphs, each of which will serve a single scene-level modality and represent a driving behavior modality. Ultimately, this method will be used to generate scene-level multimodal trajectory prediction results.

[0059] Beneficial effects of the present invention:

[0060] The present invention provides a decision-making-oriented intelligent vehicle trajectory prediction method, which introduces time series and conditional prediction models respectively to achieve temporal consistency and scene consistency of trajectory prediction. First, temporal consistency better fits the spatiotemporal continuity of intelligent vehicle prediction-decision-making tasks, can better utilize historical effective information, reduce repeated coding operations, and at the same time improve the stability and security of decision-making tasks by ensuring the continuity of prediction modes in future prediction cycles; secondly, spatial consistency allows the proposed prediction model to output with fewer modal results, covering more effective behavior prediction results. Through scene-level conditional prediction, the joint trajectory prediction results of multiple agents in a single scene are made more reasonable, the number of combinations of predicted behavior modes is reduced, and the computing power burden of the decision-making module is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the algorithm model of the present invention.

[0062] Figure 2A schematic diagram of a multi-cycle prediction task provided by an embodiment of the present invention.

[0063] Figure 3 A schematic diagram of a model for achieving time consistency provided by an embodiment of the present invention.

[0064] Figure 4 A schematic diagram of a model for achieving spatial consistency provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] Exemplary embodiments of the present application are described below with reference to the accompanying drawings. For the sake of clarity and conciseness, not all features of actual embodiments are described in this specification. However, it should be understood that in the process of developing any such actual embodiment, many implementation-specific decisions may be made to achieve the developer's specific goals, and these decisions may vary from embodiment to embodiment. The scope of protection of the present invention is not limited to the described embodiments.

[0066] It should be understood that the present application is not limited to the described embodiments due to the description with reference to the accompanying drawings. Where feasible, different embodiments may combine, replace, borrow, or omit modules or features in different embodiments without changing the ideas proposed by the present invention.

[0067] like Figure 1 As shown, the embodiment of the present invention includes two parts: a multi-period trajectory prediction model that introduces time series and a scene-level multimodal trajectory prediction model based on conditional prediction; the multi-period trajectory prediction model that introduces time series is used to implement the content of the following S1 part, and the scene-level multimodal trajectory prediction model based on conditional prediction is used to implement the content of the following S2 part. Among them, the schematic diagram of the multi-period prediction task provided by the embodiment of the present invention is as follows Figure 2 shown.

[0068] S1. According to a first aspect of the embodiment of the present invention, a multi-period trajectory prediction method introducing a time series is provided, such as Figure 3 Shown, including:

[0069] S11. Constructing a relatively invariant parametric encoding method for scene elements, including:

[0070] Obtain the original state of various elements in the driving scene, such as the position and semantic information of the intelligent body, the position, direction, and semantic information of road map elements, etc.

[0071] Specifically, in this embodiment, the agent element will obtain the trajectory state quantity under the historical observation period, such as the position, speed, heading angle, traffic participant category, etc. relative to the world coordinate system. Taking the i-th agent in the driving scene as an example, its coordinates relative to the world coordinate system at time t are marked as The heading angle is recorded as Speed ​​is recorded as x and y represent the x-axis and y-axis of the coordinate system.

[0072] Road map elements will obtain their positions relative to the world coordinate system and related semantic information, such as lane lines, lane centerlines, crosswalks, etc.

[0073] Construct an independent space-time coordinate system for each agent based on the spatial position and heading angle at each time step;

[0074] For the agent, the spatial position of the effective agent in the driving scene relative to the world coordinate system is obtained in units of one time step, and an independent space-time coordinate system is constructed for each time step of each agent. Taking the i-th agent at time t as an example, its local space-time coordinate system will be composed of time t, space coordinates and heading angle Sure;

[0075] Align the state information at other moments to the corresponding space-time coordinate system. This step can convert the absolute position information into a relative vector to avoid the change of the reference coordinate system caused by the change of the encoding subject. The relative information includes relative position, relative heading angle, etc. Taking the i-th agent at time t as an example, its absolute space-time state information is Select another element, the absolute spatiotemporal state information of the jth agent at time s is Then the relative position will be described as The relative heading angle is described as

[0076] Based on multi-layer perceptrons (MLPs) and maximum pooling operation (MaxPooling), the agent state encoding module is constructed. The state encoding module is used to encode the relative state of the agent at each time step, and the relative state embedding vector of each agent at each time step is obtained, which is recorded as Where A is the number of agents, T is the time step span, and D is the encoding dimension;

[0077] For road map elements, since road map elements do not change over time, it is only necessary to construct a spatial coordinate system that is unchanged relative to a single road map element within multiple prediction cycles;

[0078] Use a line segment group to represent a single road map element, obtain the starting and ending points of the line segment group, and use the coordinates of the starting point as the spatial position of the current road map element in the world coordinate system; connect the starting point and the ending point, and use the vector direction from the starting point to the ending point as the spatial direction of the current road map element in the world coordinate system;

[0079] An independent spatial coordinate system is constructed for each road map element based on its position and orientation. The state quantities of all constituent points in the currently selected road map element are aligned to the independent spatial coordinate system corresponding to the current road map element. The relative coordinates of each point relative to the independent spatial coordinate system are calculated and used as relative state quantities in the subsequent state quantity encoding process.

[0080] A road map element encoding module is constructed based on multi-layer perceptrons (MLPs) with a maximum pooling operation. The road map element encoding module encodes each road map element in the driving scenario and obtains the state embedding vector of each road map element relative to the global coordinate, which is recorded as Where M is the number of road map elements and D is the encoding dimension;

[0081] After obtaining the relative state embedding codes of the agent and road map elements respectively, the relative position and posture information between elements will be lost because the relative state code of each element in the driving scene is obtained based on its own local independent coordinate system.

[0082] To accurately model the relative position and posture of elements, compensate for the relative position and posture information between elements in driving scenarios, and determine the interaction relationship between elements, a spatial relative posture encoding module is introduced, which uses a multi-layer perceptron to encode and represent the relative relationship between elements of two independent intelligent agents.

[0083] The spatial relative posture encoding module will model the relative posture relationship between any two interacting elements in the driving scene, such as between intelligent agents, between intelligent agents and road map elements, and between road map elements; the two interacting elements will be marked as source elements and target elements respectively, and the relative position and relative posture of the source element relative to the target element will be obtained. A parameter-independent relative quantity encoding module will be constructed through a multi-layer perceptron to encode the relative position and posture, thereby obtaining the encoding result of the relative posture relationship between the two interacting elements.

[0084] The above-mentioned S11 link solves the problem of repeated encoding of state information of the same element in the driving scene due to changes in the scene center in multi-cycle prediction tasks, which needs to be repeated in each prediction cycle. It realizes the reuse of the encoding results of a single scene element in the driving scene and reduces the consumption of computing resources.

[0085] S12. Build a historical encoding feature memory repository to achieve the continuation and preservation of key historical state information, including:

[0086] Construct a memory repository. The memory repository will cache the historical trajectory encoding results of a fixed number of prediction cycles for a certain number of agents in the driving scenario. At the same time, the internal stored information will be updated and managed in the form of a sliding window. When an agent disappears for a period of time, a new agent will be replaced. At the same time, for agents with a longer observation time, unimportant early features will be replaced with important new spatiotemporal features. In this embodiment, the memory repository will store up to 10 sets of historical encodings for each agent that has not disappeared in the current prediction cycle.

[0087] The agent codes stored in the memory repository include two categories: one is the code results that are closer to the current forecast period, with fast data updates and dense storage, including 8 sets of historical codes; the other is the code results that are farther away from the current forecast period, with slow data updates and low storage, including 2 sets of historical codes;

[0088] The memory repository is initialized using the encoding results of valid agents in the starting scene of trajectory prediction. The feature encoding result sequence within the sliding window period is maintained for the agents that have not yet died in continuous spacetime. In the initial construction stage, if the number of observed agents is less than the maintenance limit of the repository, the perceived adjacent agents will fill the memory repository in sequence until the maintenance limit is reached. The state embedding encoding results of the agents in different observation periods are used to initialize the memory repository.

[0089] When a new round of prediction tasks begins, the agent's encoding results will be integrated with various environmental data of the driving scene and obtain prior information from the one-to-one corresponding historical encoding result sequence stored in the memory repository, including:

[0090] The attention scoring function is used to scale the dot product attention to calculate the correlation between the agent's encoding results in the current prediction cycle and the encoding sequences stored in the historical prediction cycles;

[0091] Based on the calculated weights, the encoding result sequence under the historical prediction period is used to update the encoding result of the current prediction period;

[0092] Further enrich and update the agent encoding results within the current prediction cycle;

[0093] A similarity evaluation metric is constructed, implemented as the cosine distance between the current encoding and the past encoding. This generates a similarity score across all memory locations, representing the similarity between the agent's encoding results in the current prediction cycle and the information stored in the memory repository. This is used to determine whether the memory repository contents need to be updated with the current prediction cycle results. If the similarity is small, the memory repository contents are updated; otherwise, they are not updated. In this embodiment, the specific method is as follows:

[0094] Define Memory = {a i}, stores M groups of historical frame embedding codes, where a i To achieve the historical frame embedding coding of the target agent and road map element information fusion based on the attention mechanism, when the embedding coding of the new observation is obtained k Then, the cosine distance between the feature vectors is used to calculate the similarity score between the current code and the historical code. The calculation formula is as follows:

[0095]

[0096] Based on the similarity, the encoding result sequence under the historical prediction period is used to update the encoding result of the current prediction period. The historical encoding with the highest similarity will be used as prior information to enrich the current feature encoding and decoding process, and the memory storage library will be updated at the same time.

[0097] S13. Implementing the transfer of prior information of trajectory prediction modal results, including:

[0098] At the beginning of the current prediction cycle, the historical trajectory state of the intelligent agent in the driving scene is obtained, such as position, speed, heading angle, semantic attributes, etc.

[0099] Based on the agent tracking information in the adjacent cycles, the agent state data appearing in the current prediction cycle is extracted from the previous prediction cycle, including historical trajectories and multimodal prediction trajectories;

[0100] Calculate the relative movement of each agent in the current cycle relative to the previous cycle;

[0101] Based on the modal prediction probability and relative motion in the previous prediction cycle, a specific number of reasonable modalities are selected for the target agent, such as selecting 3 groups from 6 groups of modal results to reduce the system computing power burden;

[0102] A behavioral modality encoding network composed of MLPs is used to encode the predicted behavioral modality retained by each agent to obtain prior information on the predicted modality results;

[0103] The predicted modality prior information is used to predict the agent's multimodal trajectory in the current prediction cycle. The prior information and loss function are used to guide the target agent to generate a behavioral trajectory similar to the predicted modality in the previous cycle. The agent's feature vector is concatenated with the prior information of different modalities and restored to its original dimension through a linear layer. The updated agent feature vector is then used in the decoder to generate the predicted trajectory.

[0104] S2. According to the second aspect of the embodiment of the present invention, a scenario-level multimodal trajectory prediction based on conditional prediction is provided, such as Figure 4 Shown, including:

[0105] S21. Construction of driving scenario interaction diagram, including:

[0106] Obtaining state information of elements in the driving scene around the smart car, including the position, speed, and semantic attributes of the intelligent body elements, and the position and semantic attributes of the road map elements;

[0107] Construct a fully connected spatiotemporal interaction graph of agent elements, with each agent acting as a node in the graph and bidirectional edges connecting the nodes. Initially, the spatiotemporal interaction graph will be initialized as a fully connected graph. The node encoding features of the spatiotemporal interaction graph will be initialized using the agent state embedding code. The edge features between nodes will be initialized using a neural network encoding the relative state of the source node relative to the target node.

[0108] Based on the state quantity contained in the fully connected spatiotemporal interaction graph, the relative relationship between the encoding and decoding nodes is analyzed. The node relationships include yielding, influencing and ignoring. The encoder-decoder module is composed of MLPs and residual structure.

[0109] Based on the relative relationships between nodes within the interaction subsets, a driving scenario interaction graph is constructed, divided into lower-level yield relationships relative to higher-level yield relationships. This driving scenario interaction graph guides the conditional prediction in the trajectory decoding module.

[0110] S22. Generation of scene-level multimodal trajectory prediction results, including:

[0111] Based on the driving scenario interaction relationship diagram, the prediction levels of the intelligent agents in the driving scenario are divided into groups from high-level to low-level. In the decoding process, the high-level to low-level intelligent agents will be decoded in sequence. The trajectory decoding of the low-level intelligent agents will be based on the integration of the high-level intelligent agent features.

[0112] When predicting the trajectory of the current-level agent, the agent will obtain the coded information of the trajectory prediction results of the previous-level agent in advance. The current-level agent will fuse the predicted trajectory coding information of the agent in the previous level with which it has interaction judgment according to the interaction relationship, and use the cross-attention mechanism to update its own feature vector;

[0113] Based on the updated feature vector of the target agent, the trajectory decoding module composed of MLPs is used to generate the predicted trajectory of the target agent contained in the current level to achieve conditional prediction;

[0114] The trajectory encoding module composed of MLPs encodes the trajectory decoding results of the current layer to provide them for fusion in the next layer;

[0115] Repeat the above steps until decoding reaches the lowest level.

[0116] The highest-level agent in the driving scenario interaction diagram will use edge prediction to generate predicted trajectories through the trajectory decoding module; the lowest-level agent will not need to perform predicted trajectory encoding;

[0117] To generate scene-level multimodal prediction results, S21 will generate a specific number of groups of interaction relationship graphs, each of which will serve a single scene-level modality and represent a driving behavior modality. This method is used to generate scene-level multimodal trajectory prediction results.

[0118] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. Any equivalent methods or changes that do not deviate from the technology of the present invention should be included in the scope of protection of the present invention.

Claims

1. A decision-oriented intelligent vehicle trajectory prediction method, characterized in that: It consists of two parts: The first part constructs a relative state parameterization method based on time series decomposition, decoupling state parameters from absolute spatial coordinate information. By introducing time series, a prediction method serving multi-period trajectory prediction tasks is constructed, and the spatiotemporal relative interaction behavior encoding of multiple targets within the observation interval is constructed to achieve temporal consistency within each prediction period, including information reuse of historical periods and prediction mode continuation of future periods. The method includes the following steps: S11 constructs a relatively invariant parametric encoding method for scene elements; S12 builds a historical coding feature memory repository to achieve the continuation and retention of historical state information; S13, realizing the transmission of prior information of trajectory prediction modal results; The second part is to achieve scene consistency of trajectory prediction by constructing conditional prediction, obtain the causal relationship between edge distribution and conditional distribution in large-scale scenes, and realize scene-level multimodal trajectory prediction. It includes the following steps: S21. Constructing a driving scenario interaction diagram; S22. Generation of scene-level multimodal trajectory prediction results, specifically including: S22.1 Based on the driving scenario interaction relationship diagram, the prediction levels of the intelligent agents in the driving scenario are divided into groups from high to low levels, and the division is based on the yield relationship between the lower level and the higher level; Different levels of agents use a multi-agent joint trajectory prediction method and decompose the joint distribution into marginal distribution and conditional distribution for solution; S22.2 will decode from high level to low level in the decoding process. The low-level agent will decode the trajectory based on the integration of the high-level agent features. When predicting the trajectory of the current-level agent, the agent will obtain the coded information of the trajectory prediction results of the previous-level agent in advance. The current-level agent will integrate the predicted trajectory coding information of the agent in the previous level with which it has interaction judgment according to the interaction relationship and update its own feature vector. S22.3 Based on the updated feature vector of the target agent, the trajectory decoding module is used to generate the predicted trajectory of the target agent contained in the current level to achieve conditional prediction; S22.4 encodes the trajectory decoding result of the current layer to provide it to the next layer for fusion; S22.5 Repeat the above steps S22.2-S22.4 until decoding reaches the lowest level.

2. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 1, characterized in that: The implementation of the first part includes the following: S11 constructs a relatively invariant parametric encoding method for scene elements, including: S11.1 Obtain the original state of various elements in the driving scene, including intelligent agent elements and road map elements; S11.2 For each agent, obtain the spatial position of each valid agent in the driving scene relative to the world coordinate system at each time step, and construct an independent space-time coordinate system for each time step of each agent at each time step; Align the state information at other moments to the corresponding space-time coordinate system, and convert the absolute position information into relative state quantities to avoid changes in the reference coordinate system caused by changes in the encoding subject; S11.3 uses a multi-layer perceptron to construct an agent state encoding module to encode the relative state of the agent at each time step, thereby obtaining the relative state embedding code of each agent at each time step, which is recorded as Where A is the number of agents, T is the time step span, and D is the encoding dimension; S11.4 For road map elements, construct an independent spatial coordinate system that is invariant relative to the individual road map elements during multiple sets of prediction periods; S11.5 aligns the state quantities of all constituent points in the currently selected road map element to the independent spatial coordinate system corresponding to the current road map element, and calculates the relative coordinates of each point relative to the independent spatial coordinate system as the relative state quantity for use in the subsequent state quantity encoding process; S11.6 uses a multi-layer perceptron to construct a road map element encoding module to encode the relative state of each road map element in the above driving scenario, thereby obtaining the relative state embedding code of each road map element constructed for its own local coordinate system and unchanged relative to the global world coordinate system, and denoted as Where M is the number of road map elements and D is the encoding dimension; After S11.7 obtains the relative state embedding encoding of the agent and road map elements, a spatial relative pose encoding module is introduced to accurately model the relative position and posture between elements, compensate for the relative position and posture information between elements in the driving scenario, and determine the interaction relationship between elements. A multi-layer perceptron encoding is used to represent the relative relationship between the two independent agent elements. S12 builds a historical encoding feature memory repository to achieve the continuation and preservation of historical state information, including: S12.1 Build and initialize the memory repository; The memory repository caches the historical trajectory encoding results of a fixed number of prediction cycles for a certain number of agents in the driving scene. At the same time, it updates and manages the internal storage information in the form of a sliding window. When an agent dies for a period of time, a new agent will replace it. At the same time, for agents with longer observation time, unimportant early features will be replaced by important new spatiotemporal features. Memory repository initialization method: At the initial stage of construction, if the number of observed agents is less than the maintenance limit of the repository, the perceived adjacent agents will fill the memory repository in sequence until the maintenance limit is reached; the state embedding encoding results of the agents in different observation cycles will be used to initialize the memory repository; After initialization, the memory repository will maintain the state embedding encoding result sequence of the intelligent agents that have not died in the continuous space-time within the sliding window period; S12.2 When a new round of prediction tasks begins, the agent's encoding results are integrated with other environmental data in the driving scenario, and prior information is obtained from the historical encoding result sequence stored in the memory repository. The memory repository is updated, including: The attention scoring mechanism is used to calculate the correlation between the agent encoding results in the current prediction cycle and the encoding sequence stored in the historical prediction cycle, and the correlation mechanism and similarity scoring mechanism are used to maintain the data fusion and update of the memory repository. Memory = {a i }, stores M groups of historical frame embedding codes, where a i To achieve the historical frame embedding coding of the target agent and road map element information fusion based on the attention mechanism, when the embedding coding of the new observation is obtained k Then, the cosine distance between the feature vectors is used to calculate the similarity score between the current code and the historical code. The calculation formula is as follows: The encoding result sequence under the historical prediction cycle is used to update the encoding result of the current prediction cycle. The historical encoding with the highest similarity will be used as prior information to enrich the current feature encoding and decoding process and update the memory storage library; S13. Implementing the transfer of prior information of trajectory prediction modal results, including: S13.1 At the beginning of the current prediction cycle, obtain the historical trajectory state of the intelligent agent in the driving scene; S13.2 extracts the agent state data appearing in the current prediction cycle from the previous prediction cycle based on the agent tracking information in the adjacent cycle, including the historical trajectory and the multimodal prediction trajectory; S13.4 Calculate the relative motion of each agent in the current cycle relative to the previous cycle; S13.5 selects a specific number of reasonable modes for the target agent based on the mode prediction probability and relative motion in the previous prediction cycle; S13.6 Build a behavioral modality encoding network, which encodes the predicted behavioral modality that best matches the current driving behavior trajectory of each agent in the previous prediction cycle, thereby obtaining prior information on the predicted modality result and passing it into the encoding process of the current prediction cycle; S13.7 The predicted modal prior information will be used to predict the multimodal trajectory of the agent in the current prediction cycle. The prior information will be fused with the relative state quantity embedding code of the agent in the current prediction cycle through a neural network, and the loss function design will be used to guide the target agent to generate a behavioral trajectory similar to the predicted modality of the previous cycle.

3. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 2, characterized in that: In S11.1, the original state quantity of the agent element includes the trajectory state quantity under the historical observation period, including the position, velocity, and heading angle relative to the world coordinate system; The original state of a road map element includes its position relative to the world coordinate system and related semantic information.

4. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 2, characterized in that: In S11.2, the relative state of the agent includes relative position and relative heading angle. For the i-th agent at time t, its absolute spatiotemporal state information is Select another element, the absolute spatiotemporal state information of the jth agent at time s is Then the relative position will be described as The relative heading angle is described as 5. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 2, characterized in that: In S11.4, the method for constructing an independent spatial coordinate system is as follows: using a line segment group to represent a single road map element, obtaining a starting position point and an ending position point of the line segment group, and using the coordinates of the starting point as the spatial position of the current road map element in the world coordinate system; connecting the starting point and the ending point, and using the vector direction from the starting point to the ending point as the spatial direction of the current road map element in the world coordinate system; An independent spatial coordinate system is constructed for each road map element according to the position and direction of the currently selected road map element.

6. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 2, characterized in that: In S11.7, the spatial relative posture encoding module will model the relative posture relationship between any two interacting elements in the driving scene; the two interacting elements will be marked as source elements and target elements respectively, the relative position and relative posture of the source element relative to the target element will be obtained, and a parameter-independent relative quantity encoding module will be constructed to encode the relative position and posture, thereby obtaining the encoding result of the relative posture relationship between the two interacting elements.

7. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 1, characterized in that: The second part S21, constructing a driving scenario interaction relationship diagram, is implemented by: S21.1 Obtaining state information of elements in the driving scene surrounding the intelligent vehicle, including the position, speed, and semantic attributes of intelligent body elements, and the position and semantic attributes of road map elements; S21.2 introduces a spatiotemporal interaction graph to represent driving scenarios. Each valid adjacent agent in the driving scenario will be used as a node in the spatiotemporal interaction graph, and the edges between nodes are bidirectional edges. The interaction relationship graph is represented by G = {V, E}, where V = [N] represents the nodes composed of N dynamic targets in the driving scenario, and e ij ∈E represents the directed edge between nodes i and j, representing the interaction relationship between node i and node j; S21.3 uses a spatiotemporal interaction graph to determine the interaction relationship between agents. Based on the spatiotemporal interaction graph, the nodes and edge state variables are used to decode the relative relationship between nodes. For nodes i and j, if the interaction relationship between node j and node i is determined, the embedded feature codes of agents i and j, as well as the edge code from j to i, are first obtained. These codes are then fused using a multi-layer perceptron and a maximum pooling operation to finally obtain the feature code of the node interaction relationship. Three types of node interaction relationships are defined: source nodes affect the behavior trajectory of target nodes, target nodes affect the behavior trajectory of source nodes, and source and target nodes do not affect each other. A multi-layer perceptron network is used to decode the feature encoding of the node interaction relationships obtained above and output the probabilities of the three types of node interaction relationships. After S21.4 completes the determination of the interaction relationships between all intelligent agents in the current driving scenario, it decomposes the spatiotemporal interaction graph into multiple groups of interaction relationship subgraphs according to the interaction relationship results between the intelligent agents to simplify the generation of trajectory prediction results under conditional prediction.

8. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 7, characterized in that: In S21.2, in the initial state, the spatiotemporal interaction graph is initialized as a fully connected graph; the node encoding features of the spatiotemporal interaction graph will be initialized using the agent state embedding coding, and the features of the edges between nodes will use the neural network to encode the relative state of the source node relative to the target node, and initialized using the encoding results.

9. The decision-making-oriented intelligent vehicle trajectory prediction method according to claim 7, characterized in that: In the above S22, the intelligent agent at the highest level in the driving scene interaction relationship diagram generates a predicted trajectory using an edge prediction method; the intelligent agent at the lowest level does not need to perform predicted trajectory encoding; S21 is used to generate multiple groups of interaction relationship diagrams, each of which will serve a single scene-level modality and represent a driving behavior modality, ultimately realizing the generation of scene-level multimodal trajectory prediction results.

10. A decision-oriented intelligent vehicle trajectory prediction model, characterized in that: The prediction model is a multi-period trajectory prediction model that introduces time series and / or a scene-level multimodal trajectory prediction model based on conditional prediction; The multi-period trajectory prediction model that introduces time series can realize the content of the first part of the decision-oriented intelligent vehicle trajectory prediction method of claim 1; The scenario-level multimodal trajectory prediction model based on conditional prediction can implement the content of the second part of the decision-oriented intelligent vehicle trajectory prediction method of claim 1.

Images