Key agent discrimination method fusing future motion prediction information

Abstract

The invention discloses a key agent discrimination method fusing future motion prediction information. The key agent discrimination method is characterized by comprising the following steps: selecting a true value of a key agent; constructing a self-vehicle multi-modal feature and an agent feature fused with future motion prediction information; and building a key agent selector model. The method can utilize the deep neural network model to adaptively learn and discriminate the key agent in a complex traffic environment, fully considers the possible interaction condition of a long time sequence in the future, breaks through the short-sight bottleneck of a traditional rule-based discrimination method, and improves the accuracy and rationality of an interaction determination result. According to the method, the key agent is judged by using the time window and the dynamically adjusted distance threshold value, a judgment method in a real driving process is highly simulated, and the causal reasoning capability of a key obstacle selector model is improved, so that powerful support is provided for subsequent trajectory planning.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, and in particular to a key intelligent agent discrimination method that integrates future motion prediction information. Background Technology

[0002] In complex traffic scenarios such as busy intersections and roundabouts, autonomous vehicles need to accurately perceive the status information of surrounding vehicles (such as position, speed, and heading) and further consider the potential dynamic interactions between multiple vehicles. How to effectively identify and select agents that significantly influence the vehicle's trajectory planning from among numerous traffic participants has become a core technical challenge that urgently needs to be addressed. Although existing research often uses neural network models to replace traditional rule-based methods for trajectory prediction and planning, rule-based selection mechanisms are still widely relied upon in the crucial agent identification stage. These methods often fail to fully consider the interactive evolution of spatiotemporal positions and behavioral dependencies, resulting in a lack of sufficient adaptability and generalization ability in dynamic environments.

[0003] Against the backdrop of the rapid development of autonomous driving technology, Chinese patent CN202010214301.X, "Method, System, Terminal and Storage Medium for Determining Dynamic Interaction Scenarios of Unmanned Vehicles," proposes a method for identifying and classifying dynamic interaction scenarios during unmanned vehicle testing. This method can define interactive behavior in both temporal and spatial dimensions, helping to reveal potential problems of intelligent vehicles in dynamic interaction scenarios. However, in the practical application and deployment of intelligent driving algorithms, in addition to identifying the overall scenario type, it is also necessary to further achieve a refined determination of whether there is dynamic interaction between the vehicle and surrounding intelligent agents within the scenario. This is of great significance for improving the decision-making accuracy and operational reliability of the intelligent driving system. Currently, methods for determining vehicle-to-vehicle interactions often rely on rule-based heuristic screening or simple Euclidean distance threshold screening, which is difficult to reliably distinguish traffic participants with potential interaction risks in dynamic and changing traffic flows. These methods often fail to fully explore the implicit correlation between the future movement intentions of intelligent agents and the behavior of the vehicle, easily leading to misjudgment or omission of key interaction objects. Meanwhile, in trajectory planning tasks, treating all environmental agents as equally important interaction objects not only introduces a significant computational burden but may also reduce the real-time performance and reliability of decisions due to information overload. Therefore, how to construct a method that can integrate future motion prediction information and adaptively select key interaction objects from numerous agents has become a key technical issue for improving the interaction capabilities and operational efficiency of autonomous driving systems. Summary of the Invention

[0004] To address the aforementioned technical bottlenecks, this invention proposes a key intelligent agent discrimination method that integrates future motion prediction information. This method can improve the accuracy and rationality of interactive judgment results and enhance causal reasoning capabilities, thereby providing strong support for subsequent trajectory planning.

[0005] In complex traffic scenarios such as busy intersections and roundabouts, the positions, speeds, and headings of each vehicle are constantly changing. Therefore, static assumptions based solely on spatial grid partitioning or simple Euclidean distance are insufficient for accurate judgments. To improve the accuracy of interactive judgments, it is necessary to comprehensively consider the influence of other agents on the vehicle's state at a specific moment within a future time window, and to construct a method that can reasonably characterize the risk area at each future moment. This invention selects ground truth information for posterior guidance based on future spatiotemporal positional relationships, and further outputs a set of key agents based on a deep neural network model, thereby effectively overcoming the limitations of rule-based discrimination methods in long-term interactive modeling.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows: A key agent discrimination method that integrates future motion prediction information, comprising the following steps: A: Selecting the truth values ​​of key agents A1: Obtain multi-source scene information Multi-source traffic scene information is acquired based on the environmental perception and localization modules of autonomous vehicles. This multi-source traffic scene information includes dynamic intelligent agents, static intelligent agents, vectorized map elements, and the vehicle's state. The trajectory points of both the vehicle and the intelligent agents at each moment contain six-dimensional features: their lateral coordinates relative to the vehicle's coordinate system. and vertical coordinates The cosine value of the heading angle relative to the vehicle's coordinate system Sine value and lateral velocity scalar and longitudinal velocity scalar The collected vehicle and agent trajectories are decomposed into historical and future trajectories at specific time steps.

[0007] A2: Determine the truth values ​​of key agents Based on the aforementioned multi-source traffic scene information, the time window method is used to determine the truth value of the key intelligent agent: for any future time... Computational agents in to Trajectory points within the time window and the vehicle Agents whose Euclidean distance to their current location at any given time are considered critical agents, with a risk probability of 1 if the distance is below a predetermined risk threshold, and 0 otherwise. The true risk probability for all agents is expressed as follows: This is used as the truth value for training the key agent selector. and Representing relative to time The historical and future time steps.

[0008] A3: Set risk distance threshold To further refine the risk areas at different time steps, based on The risk distance threshold is set based on the vehicle's speed at different times. Its function is to dynamically adjust the size of the risk area according to the vehicle's speed at different times, thereby making the selection of key intelligent agents more in line with human driving habits. The correction method is as follows: when the vehicle is traveling at high speed, the risk distance threshold increases accordingly; when the vehicle is traveling at low speed, the risk distance threshold decreases accordingly. The risk distance threshold is set as shown in formula (1):

[0009] in Indicates the risk distance threshold. This represents the speed scalar of the vehicle at a specific moment.

[0010] B: Constructing agent features that integrate multimodal characteristics of the vehicle with future motion prediction information. B1: Encoding multi-source traffic scene information The vehicle's state features are encoded by the State Discard Encoder (SDE), denoted as... The encoding of the historical trajectory features of a dynamic intelligent agent is achieved using a Feature Pyramid Network (FPN), denoted as... For vectorized map elements, feature encoding is performed based on the PointNet vector encoder, denoted as... Static agent features are encoded using a multilayer perceptron (MLP), denoted as... .

[0011] B2: Integrating features from multiple traffic scenarios To effectively capture the complex interactions between different features, the encoded features are concatenated to form a unified scene feature representation. It also incorporates global position coding based on Fourier embedding. Learnable embeddings related to semantic attributes As shown in formula (2):

[0012] in This indicates a splicing operation, after which... layer The encoder will transmit the scene features To integrate, the first layer The encoder is shown in Formula 3:

[0013] in, express The encoder includes layer normalization, multi-head attention, and feedforward neural network modules. Indicates passing through the previous layer The scene features aggregated by the encoder. The scene features output by the last layer are defined as follows: and break it down into , and .in, It includes both dynamic and static agent features.

[0014] B3: Constructing multimodal features of the vehicle The method for constructing multimodal features of a vehicle based on segmented reference line anchor points is as follows: First, identify the radius around the current position of the vehicle. Within the defined lane range, a fixed-length lane centerline is extracted as a reference line. Next, each reference line is decomposed into sections at fixed intervals. Each lane segment is encoded using a PointNet vector encoder, denoted as . ,in This represents the encoding dimension. Finally, the segmented reference line features are processed using an MLP. Vehicle status characteristics The fusion is performed as the initial condition for the multimodal features of the vehicle, as shown in formula (4):

[0015] in This represents the multimodal characteristics of the vehicle.

[0016] B4: Integrating Future Motion Prediction Features To fully consider the impact of agents' future behavior on vehicle planning, and to identify agents with a more significant influence on long-term vehicle planning, agent features are defined. It incorporates future motion prediction information. First, it uses MLP to analyze agent features. Decoding is performed to obtain the agent's predicted future trajectory. Next, the predicted future trajectory is encoded using FPN and then combined with the agent's features. Perform weighted fusion to make the agent features Future motion prediction information is incorporated. Finally, this feature is used as input to the key agent selector for subsequent key agent discrimination.

[0017] C: Building a key agent selector model C1: Obtain the attention scores of the agent for different modalities of the vehicle. Through vehicle multimodal features With agent characteristics Cross-attention between modalities yields attention scores for different obstacles, as shown in formula (5):

[0018] in This represents the standard cross-attention mechanism. This represents the attention score of the vehicle's multimodal features to different obstacles.

[0019] C2: Decoding the agent's risk probability Based on MLP, attention scores are encoded into key score features, and agent features are concatenated with key score features and fused with future motion prediction information. Vehicle status characteristics To obtain the risk characteristics of each agent. After decoding this feature based on MLP, through The activation function processes the data and outputs the probability of whether the agent is a key agent. As shown in formulas (6) and (7):

[0020]

[0021] C3: Calculate the loss and optimize the parameters of the key obstacle selector model. The loss between the predicted probability and the true risk probability is calculated using the focal binary classification cross-entropy function. As shown in formula (8):

[0022] in, The cross-entropy function represents the focus binary classification function, which updates the parameters of the key agent selector model through backpropagation; during the inference phase, a risk threshold is set to filter out key agents that exceed the threshold for subsequent interactive decision-making.

[0023] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention can use a deep neural network model to adaptively learn and identify key intelligent agents in complex traffic environments, fully considering possible long-term interaction scenarios, breaking through the short-sightedness bottleneck of traditional rule-based discrimination methods, and improving the accuracy and rationality of interaction judgment results.

[0024] 2. This invention utilizes time windows and dynamically adjusted distance thresholds to determine key intelligent agents, highly simulating the discrimination method in real driving processes, improving the causal reasoning ability of the key obstacle selector model, and thus providing strong support for subsequent trajectory planning. Attached Figure Description

[0025] This invention has a total of appendices Figure 3 Zhang, of which: Figure 1 This is a flowchart of the present invention.

[0026] Figure 2 This is a schematic diagram of selecting the truth value of key agents based on the time window method.

[0027] Figure 3 This is a schematic diagram of the model structure of a key intelligent agent selector that integrates future motion prediction information. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings. Figure 1 This paper presents an overall flowchart of a key agent discrimination method that integrates future motion prediction information, including the following steps: A: Selecting the truth values ​​of key agents A1: Obtain multi-source scene information By deploying an environmental perception system based on a fusion of vision and LiDAR on autonomous vehicles, surrounding intelligent agents can be detected and tracked. The agents are numbered to collect trajectory information. The trajectories of the vehicle and agents are divided into historical trajectories and future trajectories at specific time steps. The future trajectories will serve as a crucial basis for determining the truth value of key agents.

[0029] A2: Determine the truth values ​​of key agents To overcome the short-sighted bottleneck of relying solely on fixed-time selection of key agent truth values, this invention determines key agent truth values ​​based on a time window method, such as... Figure 2 As shown. Vehicle number 0 represents an autonomous vehicle, and vehicles numbered 1 and 2 represent intelligent agents in the environment. It can be observed that at some point in the future... Agent 1 is closer to the vehicle than Agent 2 is. However, within the time window... to Within the vehicle's territory, Agent 1's trajectory did not significantly encroach on the risk zone and showed a clear intention to turn left; therefore, Agent 1 will not be considered a critical agent truth value. Agent 2, despite... Although it is far away from the vehicle at any given time, it clearly intends to accelerate and overtake, and it clearly intrudes into the surrounding risk area of ​​the vehicle within a specific time window. In order to ensure the safety of the vehicle during its driving process, the impact of its above-mentioned behavior on the planning process must be considered. Therefore, Agent 2 is regarded as the truth value of the key agent.

[0030] A3: Set risk distance threshold Figure 2 In This represents the risk distance threshold, where the risk area is a circle centered on the vehicle with a radius of [missing information]. A circular area. The size of the risk area is dynamically adjusted according to the vehicle's speed at different times, making the process of selecting the truth value of the key intelligent agent closer to human driving habits, and the risk distance threshold. The adjustment method is shown in formula (1).

[0031] B: Constructing agent features that integrate multimodal characteristics of the vehicle with future motion prediction information. B1: Constructing multimodal features of the autonomous vehicle Figure 1 The key agent selector shown in the diagram receives inputs that include the vehicle's multimodal features. Specifically, this is achieved through... The encoder re-extracts the vehicle state features from the scene features and fuses them with the segmented reference line anchor point features. Each modality manages a target area and indicates the vehicle's future lateral and longitudinal movement trends.

[0032] B2: Agent characteristics that integrate future motion prediction information Another input to the key agent selector is agent features that incorporate future motion information, obtained via: The encoder outputs scene features to separate the agent's historical trajectory features, decodes the future predicted trajectory through MLP, and then encodes it into future motion prediction features through FPN network. These features are then fused with the original agent features through a residual-like structure, so that the input agent features contain future motion prediction information, which is more conducive to the training and judgment of the key agent selector model.

[0033] C: Construct the key agent selector model structure C1: Obtain the attention scores of the agent for different modalities of the vehicle. The key agent selector model structure is as follows: Figure 3 As shown, attention scores are obtained through cross-attention between the vehicle's multimodal features and the agent's features that are fused with future motion predictions. The attention scores of different modalities to the agent are encoded using an MLP, and the attention score features will be used as one of the risk features in subsequent decoding operations.

[0034] C2: Decoding the agent's risk probability Figure 3 The specific risk probability decoding process is demonstrated. Attention score features are concatenated with vehicle state features and agent features that incorporate future motion prediction information. After processing through MLP and Sigmoid activation functions, the risk probability of whether the agent is a key agent is output.

[0035] C3: Calculate the loss and optimize the parameters of the key obstacle selector model. The model parameters are updated via backpropagation by calculating the cross-entropy loss between the risk probability output by the key agent selector and the ground truth value of the key agent determined based on the future trajectory. During the inference phase, key agents are filtered based on a specific risk threshold and fed into the trajectory decoder for subsequent planning tasks.

[0036] This invention is not limited to this embodiment. Any equivalent concept or modification within the technical scope disclosed in this invention shall be included within the protection scope of this invention.

Claims

1. A key agent discrimination method that integrates future motion prediction information, characterized in that: Includes the following steps: A: Select the truth values ​​of key agents; B: Constructing multimodal features of the vehicle and integrating intelligent agent features with future motion prediction information; C: Build a key agent selector model.

2. The key agent discrimination method that integrates future motion prediction information according to claim 1, characterized in that: The method for selecting the truth value of the key agent described in step A includes the following steps: Multi-source traffic scene information is acquired based on the environmental perception and localization modules of autonomous vehicles. This multi-source traffic scene information includes dynamic intelligent agents, static intelligent agents, vectorized map elements, and the vehicle's state. The trajectory points of both the vehicle and the intelligent agents at each moment contain six-dimensional features: their lateral coordinates relative to the vehicle's coordinate system. and vertical coordinates The cosine value of the heading angle relative to the vehicle's coordinate system Sine value and lateral velocity scalar and longitudinal velocity scalar The collected trajectories of vehicles and intelligent agents are decomposed into historical trajectories and future trajectories through specific time steps; A2: Determine the truth values ​​of key agents Based on the aforementioned multi-source traffic scene information, the time window method is used to determine the truth value of the key intelligent agent: for any future time... Computational agents in to Trajectory points within the time window and the vehicle Agents whose Euclidean distance to their current location at any given time are considered critical agents, with a risk probability of 1 if the distance is below a predetermined risk threshold, and 0 otherwise. The true risk probability for all agents is expressed as follows: This is used as the truth value for training subsequent key agent selectors; among which, and Representing relative to time Historical and future time steps; A3: Set risk distance threshold To further refine the risk areas at different time steps, based on The risk distance threshold is set based on the vehicle's speed at different times. Its function is to dynamically adjust the size of the risk area according to the vehicle's speed at different times, so as to be closer to human driving habits in the process of selecting key intelligent agents. The correction method is: when the vehicle is traveling at high speed, the risk distance threshold increases accordingly; when the vehicle is traveling at low speed, the risk distance threshold decreases accordingly. The risk distance threshold is set as shown in formula (1): in Indicates the risk distance threshold. This represents the speed scalar of the vehicle at a specific moment.

3. The key agent discrimination method according to claim 1, characterized in that: Step B describes the method for constructing multimodal features of the autonomous vehicle and agent features that fuse future motion prediction information, which includes the following steps: B1: Encoding multi-source traffic scene information The vehicle's state features are encoded by the State Discard Encoder (SDE), denoted as... The encoding of the historical trajectory features of a dynamic intelligent agent is achieved using a Feature Pyramid Network (FPN), denoted as... For vectorized map elements, feature encoding is performed based on the PointNet vector encoder, denoted as... Static agent features are encoded using a multilayer perceptron (MLP), denoted as... ; B2: Integrating features from multiple traffic scenarios To effectively capture the complex interactions between different features, the encoded features are concatenated to form a unified scene feature representation. It also incorporates global position coding based on Fourier embedding. Learnable embeddings related to semantic attributes As shown in formula (2): in This indicates a splicing operation, after which... layer The encoder will transmit the scene features To integrate, the first layer The encoder is shown in Formula 3: in, express The encoder includes layer normalization, multi-head attention, and feedforward neural network modules. Indicates passing through the previous layer The scene features aggregated by the encoder; the scene features output from the last layer are defined as follows: and break it down into , and ;in, It includes both dynamic and static agent features; B3: Constructing multimodal features of the vehicle The method for constructing multimodal features of a vehicle based on segmented reference line anchor points is as follows: First, identify the radius around the current position of the vehicle. Within the defined lane range, a fixed-length lane centerline is extracted as a reference line; secondly, each reference line is decomposed into sections at fixed intervals. Each lane segment is encoded using a PointNet vector encoder, denoted as . ,in The encoding dimension is represented; finally, the segmented reference line features are represented by an MLP. Vehicle status characteristics The fusion is performed as the initial condition for the multimodal features of the vehicle, as shown in formula (4): in Represents the multimodal characteristics of the vehicle; B4: Integrating Future Motion Prediction Features To fully consider the impact of agents' future behavior on vehicle planning, and to identify agents with a more significant influence on long-term vehicle planning, agent features are defined. It incorporates future motion prediction information; firstly, it uses MLP to analyze agent features. First, decode the data to obtain the agent's predicted future trajectory; then, encode the predicted future trajectory using FPN and combine it with the agent's features. Perform weighted fusion to make the agent features The system incorporates future motion prediction information; finally, this feature is used as input to the key agent selector for subsequent key agent discrimination.

4. The key agent discrimination method that integrates future motion prediction information according to claim 1, characterized in that: The method for building the key agent selector model described in step C includes the following steps: C1: Obtain the attention scores of the agent for different modalities of the vehicle. Through vehicle multimodal features With agent characteristics Cross-attention between modalities yields attention scores for different obstacles, as shown in formula (5): in This represents the standard cross-attention mechanism. This represents the attention scores of the vehicle's multimodal features to different obstacles; C2: Decoding the agent's risk probability Based on MLP, attention scores are encoded into key score features, and agent features are concatenated with key score features and fused with future motion prediction information. Vehicle status characteristics To obtain the risk characteristics of each agent. After decoding this feature based on MLP, through The activation function processes the data and outputs the probability of whether the agent is a key agent. As shown in formulas (6) and (7): C3: Calculate the loss and optimize the parameters of the key obstacle selector model. The loss between the predicted probability and the true risk probability is calculated using the focal binary classification cross-entropy function. As shown in formula (8): in, The cross-entropy function represents the focus binary classification function, which updates the parameters of the key agent selector model through backpropagation; during the inference phase, a risk threshold is set to filter out key agents that exceed the threshold for subsequent interactive decision-making.

Images