Adaptive iteration trajectory prediction method and system based on attention mechanism

Abstract

The invention discloses a self-adaptive iteration trajectory prediction method and system based on an attention mechanism, and the method comprises the steps: obtaining the historical trajectories of a plurality of road participants through a sensor module in the process that a vehicle runs on a road, and carrying out the coding of the historical trajectories, and obtaining an intelligent agent network; obtaining and coding a high-definition map to obtain a lane map network; the intelligent agent network and the lane map network are aggregated to obtain a symmetric scene network, the relative position between the vehicle and any road participant is calculated, coding is carried out through a multi-layer perceptron to obtain the relative position, and the relative position is embedded and aggregated into the symmetric scene network; and carrying out data decoding through a decoder based on scene self-adaption to obtain a motion track of the road participant at a future moment. According to the method, the problem that redundant calculation exists during online prediction due to the fact that input needs to be subjected to re-normalization and re-coding when an observation window slides forwards in a modeling scheme taking an intelligent agent as a center in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of trajectory prediction, and in particular to an adaptive iterative trajectory prediction method and system based on an attention mechanism. Background Art

[0002] Due to the rapid development of science and technology, driverless cars are gradually becoming the protagonist of industry competition. As a key link in intelligent driving vehicles, the trajectory prediction module plays a connecting role, so it is very important to improve its prediction accuracy.

[0003] Currently, trajectory prediction in the research community is mainly based on vectorized scene representation, and on this basis, agent-centric trajectory prediction is proposed. However, the agent-centric modeling scheme requires renormalization and re-encoding of the input when the observation window slides forward, resulting in redundant calculations during online prediction. Summary of the invention

[0004] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an adaptive iterative trajectory prediction method and system based on an attention mechanism, aiming to solve the problem that the agent-centric modeling scheme in the prior art needs to renormalize and re-encode the input when the observation window slides forward, resulting in redundant calculations during online prediction.

[0005] A first aspect of the present invention is to provide an adaptive iterative trajectory prediction method based on an attention mechanism, the method comprising: When the vehicle is driving on the road, the historical trajectories of several road participants are obtained through the preset sensor module and the historical trajectories are encoded to obtain an intelligent agent network; The sensor module is used to obtain a high-definition map of the surrounding area during driving and encodes the map to obtain a lane map network; Aggregating the agent network with the lane map network to obtain a symmetric scene network, and calculating the relative position between the vehicle and any road participant, encoding through a multi-layer perceptron to obtain a relative position embedding, and aggregating it into the symmetric scene network; The data is decoded by a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future moment.

[0006] According to one aspect of the above technical solution, when a vehicle is driving on a road, the steps of obtaining the historical trajectories of several road participants through a preset sensor module and encoding the historical trajectories to obtain an intelligent agent network include: When a vehicle is driving on the road, the historical trajectories of several road participants are obtained through the sensor modules preset on the vehicle; each road participant is an intelligent agent; The historical trajectory encoder is used to encode the features of the historical trajectory of a single agent at different historical moments, and linear aggregation is performed to obtain an agent network.

[0007] According to one aspect of the above technical solution, the step of obtaining a high-definition map of the periphery during driving and encoding it through the sensor module to obtain a lane map network includes: When the vehicle is driving on the road, a high-definition map of the surrounding area is obtained through the preset sensor module on the vehicle; According to the high-definition map, each road is divided into a plurality of lanes, and each lane is divided into a plurality of lane segments; The feature encoding of multiple lane segments in the same road is performed through a node encoder and linear aggregation is performed to obtain a lane graph network.

[0008] According to one aspect of the above technical solution, the agent network is aggregated with the lane map network to obtain a symmetric scene network, and the relative position between the vehicle and any road participant is calculated, and the relative position is embedded by encoding through a multi-layer perceptron, and the steps of aggregating them into the symmetric scene network include: Obtain the heading difference, relative azimuth and distance between the vehicle and the road participant, and determine the relative position between the vehicle and the road participant based on the heading difference, relative azimuth and distance; The relative position is encoded by a multi-layer perceptron to obtain a relative position embedding; wherein the relative azimuth is represented by a sine value and a cosine value; Outputting agent features according to the agent network, and outputting lane environment features according to the lane map network; The agent features, lane environment features and relative position embedding are fused by vectorization to obtain a symmetric scene network.

[0009] According to one aspect of the above technical solution, the step of fusing the agent features, lane environment features and relative position embedding by vectorization to obtain a symmetric scene network includes: Using context features as keys and values ​​and the extended input agent as a query, information is passed from context features to agent features based on a multi-head attention mechanism for information update.

[0010] According to one aspect of the above technical solution, the step of decoding data by a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future time includes: Generate several adaptive trajectory anchor points in a data-driven manner through a scene-based adaptive decoder; The trajectory anchor point is selected through an adaptive optimization iteration module to obtain the motion trajectory of the road participant at a future moment.

[0011] According to one aspect of the above technical solution, the method further includes: The motion trajectory is used to adaptively select the trajectory anchor point, retrieve key context elements, and generate an offset of each trajectory segment in the motion trajectory by encoding context to optimize each trajectory segment to obtain a final motion trajectory.

[0012] A second aspect of the present invention is to provide an adaptive iterative trajectory prediction system based on an attention mechanism, which is applied to the method described in the above technical solution, and the system comprises: The first network building module is used to obtain the historical trajectories of several road participants through a preset sensor module and encode the historical trajectories to obtain an intelligent agent network when the vehicle is driving on the road; A second network construction module is used to obtain a high-definition map of the periphery during driving through the sensor module and encode it to obtain a lane map network; An aggregation module is used to aggregate the agent network and the lane map network to obtain a symmetric scene network, calculate the relative position between the vehicle and any road participant, encode the relative position through a multi-layer perceptron to obtain relative position embedding, and aggregate it into the symmetric scene network; The motion trajectory prediction module is used to decode data through a scene-adaptive decoder to obtain the motion trajectory of the road participant at a future moment.

[0013] A third aspect of the present invention is to provide a readable storage medium having a computer program stored thereon, which implements the method described in the above technical solution when executed by a processor.

[0014] The fourth aspect of the present invention is to provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in the above technical solution when executing the computer program.

[0015] Compared with the prior art, the adaptive iterative trajectory prediction method and system based on the attention mechanism shown in the present invention has the following beneficial effects: The present invention builds a decoder based on an agent network, a map network, and scene adaptation, and effectively embeds map topology information into the attention mechanism through a lane map network, producing excellent results. At the same time, the symmetric scene network is used, combined with global attention and relative position embedding updates, so that the design is more compact, and the decoder based on scene adaptation is combined with adaptive optimization iteration to achieve a dynamic and reasonable allocation between accuracy and computational consumption. In the actual vehicle deployment work, it can greatly improve the utilization rate of each module and achieve efficient output of multimodal trajectory prediction, thereby improving the riding safety and comfort of autonomous driving vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which: Figure 1 Schematic diagram of a flow chart of an adaptive iterative trajectory prediction method based on an attention mechanism in one embodiment of the present invention; Figure 2 It is a structural block diagram of an adaptive iterative trajectory prediction system based on the attention mechanism in one embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings. Several embodiments of the present invention are shown in the accompanying drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive.

[0018] It should be noted that when an element is referred to as being "fixed to" another element, it may be directly on the other element or there may be a central element. When an element is considered to be "connected to" another element, it may be directly connected to the other element or there may be a central element at the same time. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0020] Embodiment 1 See also Figure 1The first embodiment of the present invention provides an adaptive iterative trajectory prediction method based on an attention mechanism, the method comprising steps S10 to S40: Step S10, when the vehicle is driving on the road, the historical trajectories of several road participants are obtained through a preset sensor module and the historical trajectories are encoded to obtain an intelligent agent network.

[0021] In this embodiment, the vehicle refers to the self-vehicle, which is equipped with a sensor module, including a camera, a lidar, a millimeter-wave radar, etc. During the driving process of the vehicle, the sensor module is used to obtain the historical trajectory of peripheral road participants during the driving process of the vehicle, that is, the historical movement trajectory of the road participants at historical moments is obtained.

[0022] Among them, road participants include vehicles on the road and outside the road, cyclists, pedestrians, and movable obstacles, which can be stationary or moving at the historical moment. For example, a vehicle that is a road participant is traveling in the same direction as the adjacent vehicle.

[0023] Furthermore, when the vehicle is driving on the road, the historical trajectories of several road participants are obtained through a preset sensor module and the historical trajectories are encoded to obtain the steps of the intelligent agent network, including: When a vehicle is driving on the road, the historical trajectories of several road participants are obtained through the sensor modules preset on the vehicle; each road participant is an intelligent agent; The historical trajectory encoder is used to encode the features of the historical trajectory of a single agent at different historical moments, and linear aggregation is performed to obtain an agent network.

[0024] Specifically, when the vehicle is driving on the road, it will identify peripheral road participants through sensor modules such as cameras, lidar, and millimeter-wave radar on the vehicle, and obtain the historical trajectories of road participants at historical moments. Each road participant is an independent intelligent agent. The historical trajectory encoder is then used to encode the features of the historical trajectories of individual intelligent agents at different moments, and linearly aggregate them to obtain an intelligent agent network. Finally, the intelligent agent network is used to manage the interactions between agents in the current driving scene.

[0025] Step S20, obtaining a high-definition map of the surrounding area during driving through the sensor module and encoding it to obtain a lane map network.

[0026] In this embodiment, during the driving process of the vehicle, a high-definition map of the surrounding area during the driving process is also obtained through the above-mentioned sensor module, and then the roads, lanes or lane sections are encoded and output in the form of a vector map to obtain a lane map network.

[0027] Furthermore, the step of obtaining a high-definition map of the periphery during driving by the sensor module and encoding it to obtain a lane map network includes: When the vehicle is driving on the road, a high-definition map of the surrounding area is obtained through the preset sensor module on the vehicle; According to the high-definition map, each road is divided into a plurality of lanes, and each lane is divided into a plurality of lane segments; The feature encoding of multiple lane segments in the same road is performed through a node encoder and linear aggregation is performed to obtain a lane graph network.

[0028] Specifically, during the driving process of the vehicle, the sensor module will simultaneously identify and obtain the historical trajectories of road participants and the high-definition map of the surrounding environment, and then construct the intelligent agent network and the lane map network respectively. When constructing the lane map network, each road is divided into multiple lanes according to the high-definition map, and each lane is divided into multiple lane segments. Then, the node encoder is used to divide the lane segments as units for feature encoding, and linear aggregation is performed to obtain the lane map network.

[0029] Step S30, aggregating the agent network and the lane map network to obtain a symmetric scene network, and calculating the relative position between the vehicle and any road participant, encoding through a multi-layer perceptron to obtain relative position embedding, and aggregating it into the symmetric scene network.

[0030] In this embodiment, the agent network and the lane map network are aggregated to obtain a symmetric scene network, and the relative position between the vehicle and any road participant is calculated, and the relative position embedding is obtained by encoding through a multi-layer perceptron, and the steps of aggregating them into the symmetric scene network include: Obtain the heading difference, relative azimuth and distance between the vehicle and the road participant, and determine the relative position between the vehicle and the road participant based on the heading difference, relative azimuth and distance; The relative position is encoded by a multi-layer perceptron to obtain a relative position embedding; wherein the relative azimuth is represented by a sine value and a cosine value; Outputting agent features according to the agent network, and outputting lane environment features according to the lane map network; The agent features, lane environment features and relative position embedding are fused by vectorization to obtain a symmetric scene network.

[0031] The step of fusing the agent features, lane environment features and relative position embedding by vectorization to obtain a symmetric scene network includes: Using context features as keys and values ​​and the extended input agent as a query, information is passed from context features to agent features based on a multi-head attention mechanism for information update.

[0032] Specifically, in this embodiment, the relative position of the agent is calculated in pairs using heading difference, relative azimuth and distance, and further encoded by a multi-layer perceptron to obtain relative position embedding. In order to enhance numerical stability, the angle is represented by its sine and cosine values.

[0033] Then, the agent features, surrounding features, and RPE are effectively fused by vectorization, using context features as keys and values, and the extended input agent features as queries. Finally, a multi-head attention mechanism is applied to pass messages from context features to agent features for real-time information updates.

[0034] Step S40, decoding the data using a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future moment.

[0035] In this embodiment, the step of decoding data by a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future time includes: Generate several adaptive trajectory anchor points in a data-driven manner through a scene-based adaptive decoder; The trajectory anchor point is selected through an adaptive optimization iteration module to obtain the motion trajectory of the road participant at a future moment.

[0036] Wherein, the method further comprises: The motion trajectory is used to adaptively select the trajectory anchor point, retrieve key context elements, and generate an offset of each trajectory segment in the motion trajectory by encoding context to optimize each trajectory segment to obtain a final motion trajectory.

[0037] Specifically, the scene-adaptive decoder utilizes a recurrent, anchor-free trajectory proposal module to generate K adaptive trajectory anchors in a data-driven manner, and then uses an adaptive optimization iteration module to select the final predicted trajectory. In the mode-scene attention, a cross-attention layer is used to update the mode query with multiple contexts, including the target agent's history encoding, map encoding, and neighboring agent encodings. After the mode-scene attention, multi-modal queries query each other through mode-mode self-attention to improve the diversity of multiple modes.

[0038] In order to reduce the context retrieval burden of queries and improve the quality of trajectory anchors, the decoder of DETR in object detection is generalized to a loop mode. In the loop process, context-aware mode queries are only decoded through a multi-layer perceptron at the end of each loop step to decode future trajectories. In the subsequent process, these queries become input again and retrieve the scene context related to the prediction of the next few waypoints.

[0039] In addition, this embodiment also uses the initial predicted trajectory to adaptively select trajectory anchors and retrieve key context elements, and then optimizes each trajectory segment by generating an offset for each trajectory segment using the encoded context. Furthermore, the model predicts a quality score that measures the quality of the prediction and adaptively determines the number of refinement iterations.

[0040] More specifically, in this embodiment, the BEV perspective is used to project the positions of each intelligent agent in the current driving scene. i The historical state of an agent , the goal of this embodiment is to predict The future state of the time step and the associated probability p , where Represents the agent at time step T state, which consists of , where represents the coordinates of the center of the agent, Indicates whether the agent lacks historical trajectories (filled with 0 if it does not exist), Indicates the quality of the trace (from 0 to 3), where higher values ​​indicate more comprehensive historical traces. represents the type of the target agent, represents the heading angle relative to the agent, Represents the speed of the agent on the X and Y axes.

[0041] The target agent will be associated with the context element Interact with surrounding agents Historical status and HD maps m For HD maps m , using a vectorized representation, where each lane is defined as a sequence of points along its centerline, each of which is represented by coordinates and semantic information. Therefore, the classic motion prediction task is expressed as ,in f Represents a prediction model.

[0042] In this embodiment, firstly, according to a backbone network Generate initial trajectory and trajectory characteristics , so the classic trajectory prediction method is expressed as Initialize the trajectory Used to select track anchor points and retrieve key context elements , and then compare them with the trajectory features Passed together into the refinement model Since trajectory refinement is a multiple-iteration process, i The trajectory refinement formula for the iteration is: , and finally the generated offset Add to Input Track Trajectory refinement is performed in .

[0043] In this embodiment, the historical trajectory encoder HTE is used to encode the temporal characteristics of the historical trajectory of a single agent, and the historical state of the agent is represented as ,in represents the number of input features. First, use MLP to transform the input features into ,Right now ,in, represents a multi-layer perceptron network, Represents the dimension of the conversion feature, and then uses the stacked Transformer structure to transform the agent T The Transformer structure encodes the features within the time step. j The attention mechanism of the layer is expressed as ,in , and then use the MLP-based aggregator to construct a singular feature vector for each trajectory, thereby obtaining the final output feature of the historical trajectory encoder HTE The agent network AIN is used to identify the differences between agent trajectories and effectively collect information about surrounding agents. AIN also uses the Transformer structure, which is represented as ,in The final output feature is .

[0044] In this embodiment, a node encoder NE is used to encode the node features on the same lane, and further aggregate these nodes into lane-level features. As input, and calculate the aggregate features ,in represents the number of lanes in the scene, represents the number of nodes in each lane, represents the input dimension of the map, Indicates the output dimension produced by the NE network.

[0045] Among them, the node encoder NE and the history trajectory encoder THE have the same structure and data processing method, but have different hyperparameters and weights. Compared with one-dimensional data, map data shows complex connection patterns, including differences in spatial distances and various types of node connections. Traditional encoding methods have difficulty capturing these complex relationships, so a lane graph network LGT is referenced in this embodiment.

[0046] The attention mechanism of the lane graph network LGT model is designed as follows: ; in, , , , , Represents the connection type matrix of the preceding connection, the succeeding connection, the left connection, the right connection and the map topology structure, W is the corresponding weight matrix for each matrix.

[0047] The structure of the LGT model consists of stacked Transformer layers that embed topological information from the map. All layers in this structure share the RPE matrix, taking the output of NE as the input of LGT and obtaining map information. ,in D Representation output The dimension of j The attention matrix of the layer is expressed as .

[0048] In this embodiment, the bias matrix is ​​used B to represent the spatial relativity of the map topology. Since the longitudinal velocity of a vehicle is usually higher than its lateral velocity, the agent exhibits more dependencies in the front-to-back direction.

[0049] In order to capture the differences in traffic scene movement, four connection relationships are defined in this embodiment: predecessor, successor, left neighbor and right neighbor, and four PRE matrices are constructed. In addition, lateral connectivity may affect lane change, so a connection type matrix is ​​constructed , and perform matrix multiplication to represent the lateral connectivity between adjacent channels, using For static map elements, such as lane segments, the centroid of the polyline is used as the trajectory anchor, and the displacement vector between the endpoints is used as the heading angle.

[0050] Specifically, the elements i The anchor position in the global coordinate system of and the heading vector To represent it, we use three quantities to describe the elements.i and elements j The relative position between , relative azimuth and distance .

[0051] To enhance numerical stability, angles are expressed using their sine and cosine values.

[0052] In this embodiment, the heading difference Expressed as , relative azimuth (The displacement vector is ) is expressed as , the relative spatial information is represented as a 5-dimensional vector In addition, the relative position encoding is further encoded through MLP to produce a relative position embedding RPE with shape [N,N,D].

[0053] In this embodiment, once the agent features, map features, and relative position embedding RPE are obtained, the features are updated in a viewpoint-invariant manner using the symmetric scene network. The structure of the symmetric scene network, which includes multiple stacked SFT layers, is similar to the standard Transformer. i and j The characteristics of each agent are expressed as , ,from arrive The RPE vector associated with the edge is represented as Tuple Contains scheduled slave nodes i Passed to the node j Therefore, a simple MLP can be used to encode these features and obtain the node j No. i context vector, represented as ,in represents the concatenation operator, It means that the MLP consists of a linear layer, a layer normalization, and a ReLU activation function. Then, cross attention is performed on the target node and its context, which is expressed as ,in is the standard multi-head attention function, It is a feature j The context vector set of . At the same time, Also included , indicating that each node has a self-loop. As with the standard Transformer, a point-wise feed-forward fully connected layer is integrated after the attention mechanism. In addition, in each layer, The update is performed by re-encoding the context vector using another MLP and then added to the input RPE via a residual connection.

[0054] In this embodiment, the last step of prediction is to use the scene encoding output by the encoder to decode the K future trajectories of each target agent, which is very important because the encoder only returns a set of feature embeddings. Inspired by object detection research, a DETR-like decoder is used to handle such a one-to-many problem, where multiple learnable queries interact to encode and decode trajectories. The query-based decoder utilizes a recurrent, trajectory anchor-free proposal module to generate adaptive trajectory anchors, followed by a scene-adaptive decoder that improves the performance of trajectory prediction models with limited additional computation by adaptively iterating between retrieving key contextual elements and predicting more accurate trajectories.

[0055] In the mode-scene attention module, a cross-attention layer is used to update the mode query with multiple contexts, including the historical trajectory of the target agent, the map encoding, and the encodings of neighboring agents. Thanks to the cross-attention layer, the mode query can retrieve the scene context and quickly narrow the search space of trajectory anchors. Afterwards, K mode queries pay attention to each other through the mode-mode self-attention module to improve the diversity of multiple modes. In order to predict the trajectories of multiple agents in parallel, a set of scene encodings is shared among all target agents in the scene. Since these encodings are retrieved from their local spatiotemporal coordinate system, they need to be projected into the current viewpoint of each agent to achieve the same effect as agent-centric modeling.

[0056] Trajectory refinement requires information-rich trajectory anchor selection and context retrieval. After generating the initial trajectory and trajectory features in the previous decoder, the trajectory refinement module first selects a certain number of trajectory anchors along the trajectory and then retrieves the context around the trajectory anchors with an adaptive radius. In order to strike a balance between context richness and computational efficiency, the trajectory is divided into N lane segments and the end point of each lane segment is selected as the trajectory anchor. The context elements around the trajectory anchor are retrieved. c Previous methods usually use a fixed radius or area around the trajectory anchor point to extract the local context, but this method is very poor when applied to different data sets and scenarios that may require different retrieval ranges. Therefore, in this embodiment, a method of adaptive radius retrieval of trajectory anchor points is proposed, which can be achieved by the following principles, that is, the number of trajectory refinement iterations and the speed of the target intelligent agent v , which is expressed as ,in As the number of trajectory refinement iterations increases, the trajectory becomes more accurate and the radius search range decreases.

[0057] Previous methods usually fuse all embeddings at once and refine the entire future trajectory, which has a long-term cumulative error. To alleviate this problem, a cyclic refinement strategy is adopted to divide the trajectory into N trajectory segments and refine the trajectory N times each time. Here N is equal to the number of trajectory anchors, that is, each trajectory corresponds to one trajectory anchor. For each trajectory segment, only the context retrieved from the corresponding trajectory anchor is used for trajectory refinement to enhance local context fusion. In addition, refining all N trajectory segments completes one trajectory refinement iteration, while this embodiment will perform multiple trajectory refinement iterations.

[0058] Specifically, in each loop refinement, in this embodiment, by refining a trajectory segment, the trajectory anchor point corresponding to the trajectory segment a Embedded around a set of scene context And the future trajectory embedding of the target agent This embodiment adopts a cross-attention mechanism to fuse them, where the trajectory embedding is used as a query to process the The fused trajectory embedding will be used to predict , which is the offset of the waypoint in the trajectory segment, to subtly adjust the original trajectory segment. The updated trajectory embedding will also be used as a new query to refine the next trajectory segment. All are done in one iteration N After time steps, the entire trajectory will be adjusted and the trajectory embedding will be updated with rich contextual elements. Since multiple possible trajectories are predicted, the trajectory embedding will also be used to predict the probability of each trajectory.

[0059] After one iteration, the updated trajectory and trajectory embedding are used to start another refinement iteration. This embodiment first repeats the adaptive trajectory anchor selection and context retrieval mentioned above, and then performs another N In the first refinement iteration, the present embodiment uses a compression network to reduce the hidden dimension of the trajectory embedding of the initial trajectory prediction module to achieve efficient refinement with little performance loss. As the subsequent iterations are entered, the predicted trajectory becomes more accurate. However, after multiple trajectory refinement iterations, the accuracy of trajectory prediction may also decrease.

[0060] In order to balance the performance gain and the additional computational effort, this embodiment proposes an adaptive trajectory refinement strategy, in which the number of refinement iterations is dynamically adjusted according to the current prediction quality and the remaining prediction refinement improvement. Specifically, this embodiment proposes a quality score to quantify the current prediction quality, which is expressed as ,in represents the maximum prediction error among all iterations, In order to enable the model to predict the quality score, this embodiment first uses GRU to repeatedly process the trajectory embeddings of all previous iterations, and then uses MLP to generate the quality score, generating three types of outputs: predicted trajectory, predicted trajectory probability and quality score.

[0061] In addition, the quality score of each refinement iteration will be predicted. Therefore, this embodiment proposes a simple and effective strategy to dynamically decide whether another trajectory refinement iteration is needed. The input includes: the initial trajectory prediction model , trajectory refinement model , Quality Score Decoder 、Agent history trajectory , Scene Context c , quality score threshold , the maximum number of refinement iterations in the trajectory refinement phase ; Its output includes: the future trajectory of the target agent And the corresponding trajectory probability p .

[0062] Compared with the prior art, the adaptive iterative trajectory prediction method based on the attention mechanism shown in this embodiment has the following beneficial effects: This embodiment builds a decoder based on an agent network, a map network, and scene adaptation, and effectively embeds map topology information into the attention mechanism through a lane map network, producing excellent results. At the same time, the symmetric scene network is used, combined with global attention and relative position embedding updates, making the design more compact, and the scene-adaptive decoder combined with adaptive optimization iterations makes a dynamic and reasonable allocation between accuracy and computational consumption. In actual vehicle deployment work, it can greatly improve the utilization of each module and achieve efficient output of multimodal trajectory prediction, thereby improving the riding safety and comfort of autonomous driving vehicles.

[0063] Embodiment 2 See also Figure 2 The second embodiment of the present invention provides an adaptive iterative trajectory prediction system based on an attention mechanism, which is applied to the method described in the above embodiment. The system includes: The first network construction module 10 is used to obtain the historical trajectories of several road participants through a preset sensor module and encode the historical trajectories to obtain an intelligent agent network when the vehicle is driving on the road; A second network construction module 20 is used to obtain a high-definition map of the periphery during driving through the sensor module and encode it to obtain a lane map network; Aggregation module 30, used for aggregating the agent network with the lane map network to obtain a symmetric scene network, and calculating the relative position between the vehicle and any road participant, encoding through a multi-layer perceptron to obtain a relative position embedding, and aggregating it into the symmetric scene network; The motion trajectory prediction module 40 is used to decode data through a scene-adaptive decoder to obtain the motion trajectory of the road participant at a future moment.

[0064] Compared with the prior art, the adaptive iterative trajectory prediction system based on the attention mechanism shown in this embodiment has the following beneficial effects: This embodiment builds a decoder based on an agent network, a map network, and scene adaptation, and effectively embeds map topology information into the attention mechanism through a lane map network, producing excellent results. At the same time, the symmetric scene network is used, combined with global attention and relative position embedding updates, making the design more compact, and the scene-adaptive decoder combined with adaptive optimization iterations makes a dynamic and reasonable allocation between accuracy and computational consumption. In actual vehicle deployment work, it can greatly improve the utilization of each module and achieve efficient output of multimodal trajectory prediction, thereby improving the riding safety and comfort of autonomous driving vehicles.

[0065] Embodiment 3 A third embodiment of the present invention provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method described in the above embodiment is implemented.

[0066] Embodiment 4 A fourth embodiment of the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the method described in the above embodiments when executing the computer program.

[0067] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0068] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An adaptive iterative trajectory prediction method based on attention mechanism, characterized in that: The method comprises: When the vehicle is driving on the road, the historical trajectories of several road participants are obtained through the preset sensor module and the historical trajectories are encoded to obtain an intelligent agent network; The sensor module is used to obtain a high-definition map of the surrounding area during driving and encodes the map to obtain a lane map network; Aggregating the agent network with the lane map network to obtain a symmetric scene network, and calculating the relative position between the vehicle and any road participant, encoding through a multi-layer perceptron to obtain a relative position embedding, and aggregating it into the symmetric scene network; The data is decoded by a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future moment.

2. The adaptive iterative trajectory prediction method based on the attention mechanism according to claim 1, characterized in that: When the vehicle is driving on the road, the historical trajectories of several road participants are obtained through the preset sensor module and the historical trajectories are encoded to obtain the steps of the intelligent agent network, including: When a vehicle is driving on the road, the historical trajectories of several road participants are obtained through the sensor modules preset on the vehicle; each road participant is an intelligent agent; The historical trajectory encoder is used to encode the features of the historical trajectory of a single agent at different historical moments, and linear aggregation is performed to obtain an agent network.

3. The adaptive iterative trajectory prediction method based on the attention mechanism according to claim 1, characterized in that: The steps of obtaining a high-definition map of the periphery during driving by the sensor module and encoding it to obtain a lane map network include: When the vehicle is driving on the road, a high-definition map of the surrounding area is obtained through the preset sensor module on the vehicle; According to the high-definition map, each road is divided into a plurality of lanes, and each lane is divided into a plurality of lane segments; The feature encoding of multiple lane segments in the same road is performed through a node encoder and linear aggregation is performed to obtain a lane graph network.

4. The adaptive iterative trajectory prediction method based on the attention mechanism according to claim 1, characterized in that: The agent network and the lane map network are aggregated to obtain a symmetric scene network, and the relative position between the vehicle and any road participant is calculated, and the relative position embedding is obtained by encoding through a multi-layer perceptron, and the steps of aggregating them into the symmetric scene network include: Obtain the heading difference, relative azimuth and distance between the vehicle and the road participant, and determine the relative position between the vehicle and the road participant based on the heading difference, relative azimuth and distance; The relative position is encoded by a multi-layer perceptron to obtain a relative position embedding; wherein the relative azimuth is represented by a sine value and a cosine value; Outputting agent features according to the agent network, and outputting lane environment features according to the lane map network; The agent features, lane environment features and relative position embedding are fused by vectorization to obtain a symmetric scene network.

5. The adaptive iterative trajectory prediction method based on the attention mechanism according to claim 4 is characterized in that: The step of fusing the agent features, lane environment features and relative position embedding by vectorization to obtain a symmetric scene network includes: Using context features as keys and values ​​and the extended input agent as a query, information is passed from context features to agent features based on a multi-head attention mechanism for information update.

6. The adaptive iterative trajectory prediction method based on attention mechanism according to claim 1, characterized in that: The step of decoding data by a scene-adaptive decoder to obtain the movement trajectory of the road participant at a future time includes: Generate several adaptive trajectory anchor points in a data-driven manner through a scene-based adaptive decoder; The trajectory anchor point is selected through an adaptive optimization iteration module to obtain the motion trajectory of the road participant at a future moment.

7. The adaptive iterative trajectory prediction method based on the attention mechanism according to claim 6 is characterized in that: The method further comprises: The motion trajectory is used to adaptively select the trajectory anchor point, retrieve key context elements, and generate an offset of each trajectory segment in the motion trajectory by encoding context to optimize each trajectory segment to obtain a final motion trajectory.

8. An adaptive iterative trajectory prediction system based on attention mechanism, characterized in that: The method applied to any one of claims 1 to 7, wherein the system comprises: The first network building module is used to obtain the historical trajectories of several road participants through a preset sensor module and encode the historical trajectories to obtain an intelligent agent network when the vehicle is driving on the road; A second network construction module is used to obtain a high-definition map of the periphery during driving through the sensor module and encode it to obtain a lane map network; An aggregation module is used to aggregate the agent network and the lane map network to obtain a symmetric scene network, calculate the relative position between the vehicle and any road participant, encode the relative position through a multi-layer perceptron to obtain relative position embedding, and aggregate it into the symmetric scene network; The motion trajectory prediction module is used to decode data through a scene-adaptive decoder to obtain the motion trajectory of the road participant at a future moment.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images