Intelligent agent path decision-making method and device, computer equipment and storage medium

Abstract

The invention relates to an agent path decision method and device, computer equipment and a storage medium. The method comprises the following steps: inputting historical multi-modal data into a multi-modal hierarchical feature fusion network, fusing the historical multi-modal data by adopting a fusion encoder based on a Transform structure, and obtaining future trajectory features of a target agent and surrounding moving obstacles through a decoder; and acquiring future trajectory information of the target agent and the surrounding moving obstacles by adopting a multi-head self-attention mechanism through a multi-trajectory prediction network, and performing an interactive game according to the future trajectory information of the target agent and the surrounding moving obstacles, and generating motion trajectory prediction of the surrounding moving obstacles and a path decision trajectory of the target agent. According to the method, when the trajectory is generated, complex dynamic interaction behaviors such as avoidance, following or vehicle meeting possibly existing in the target agent and the surrounding moving obstacles in the future can be predicted, and the accuracy of trajectory prediction and path decision is improved.

Description

Technical Field

[0001] This application belongs to the field of intelligent driving technology, and specifically relates to an intelligent agent path decision-making method, device, computer equipment, and storage medium. Background Technology

[0002] Intelligent driving refers to technologies where machines assist humans in driving, and in special circumstances, completely replace human drivers. Intelligent driving and electric vehicles are naturally related, with a mutually reinforcing and supportive relationship. Currently, intelligent driving typically employs imitation learning-based decision-making and planning methods for path planning. In these methods, the decision-making model needs to combine information such as the surrounding map environment, historical information, and predicted obstacle trajectories of the autonomous driving vehicle (ADV) to make reasonable planning decisions. The ADV needs to interact with surrounding moving obstacles (Surrounding Vehicles, SVs) in the real environment and complete its planning and navigation tasks.

[0003] Currently, in imitation learning-based decision planning methods, trajectory prediction for the Service Provider (SV) is considered an independent upstream task in path decision planning. Historical interaction information from the scene is used to predict the SV's trajectory, while the Path Provider (ADV) performs real-time path planning based on the predicted SV trajectory and scene information. This prediction method, which only uses historical interaction information, is called Marginal Prediction (MBP). The MBP method assumes that the SV makes planning decisions solely based on historical interaction information from the scene. However, in real-world scenarios, both the SV and ADV make predictions about each other and adjust their decision-making behavior accordingly. Therefore, the MBP method ignores the fact that both the SV and ADV make their own predictions about each other, leading to inaccurate predicted SV trajectories and affecting the accuracy of the ADV's final path planning. Summary of the Invention

[0004] This application provides an agent path decision-making method, apparatus, computer device, and storage medium, which aims to at least partially solve one of the aforementioned technical problems in the prior art.

[0005] To address the above problems, this application provides the following technical solution:

[0006] A path decision-making method for an intelligent agent, comprising:

[0007] Acquire historical multimodal data of the target intelligent agent; the historical multimodal data includes the historical motion characteristics of the target intelligent agent, the historical motion characteristics of surrounding moving obstacles, map lane information, and map zebra crossing area information;

[0008] The historical multimodal data is input into a multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on a Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target intelligent agent and surrounding moving obstacles through a decoder.

[0009] The future trajectory features are input into a multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles. Based on the future trajectory information of the target agent and surrounding moving obstacles, the network performs interactive game to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

[0010] The technical solution adopted in this application embodiment also includes: the historical motion features of the target intelligent agent are represented as [T] h D aDV ], where T h For a historic moment, D ADV The motion characteristics of the target intelligent agent include center coordinates, direction, and vehicle size; the historical motion characteristics of the surrounding moving obstacles are represented as [N]. SV ,T h D SV ], where N SV D is the number of moving obstacles in the surrounding area. SV The motion characteristics of the surrounding moving obstacles; the map lane information is represented as [N] m N p D m ], where N m Np is the number of lane vectors to search, and D is the number of feature points for each lane vector. m The feature point attributes include coordinates and whether the crossing is permitted under the current traffic rules; the map zebra crossing area information is represented as [N] c N p D c ], where N c N represents the number of crosswalk vectors being searched. p D represents the number of feature points in each pedestrian vector. c These are the attributes of the feature points.

[0011] The technical solution adopted in this application embodiment further includes: inputting historical multimodal data into a multimodal hierarchical feature fusion network, wherein the multimodal hierarchical feature fusion network uses a Transformer-based fusion encoder to fuse the historical multimodal data, extracts the spatiotemporal interaction features of the historical multimodal data, and obtains the future trajectory features of the target intelligent agent and surrounding moving obstacles through a decoder, specifically:

[0012] The multimodal hierarchical feature fusion network employs a Transformer structure to analyze the historical motion features of the target agent and the historical motion features of surrounding moving obstacles [N]. SV ,T h D SV The map lane information [N] is encoded and the Vectornet feature extraction algorithm is used to extract the features. m N p D m [N] and zebra crossing area information on the map c N p D c The historical multimodal data is encoded, mapped to the same feature space and stacked. Then, the spatiotemporal interaction features of the historical multimodal data are extracted by a fusion encoder based on a multi-layer Transformer structure. The spatiotemporal interaction features are input into a future decoder to obtain the future trajectory features of the target agent and surrounding moving obstacles.

[0013] The technical solution adopted in this application embodiment further includes: inputting the future trajectory features into a multi-trajectory prediction network, wherein the multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performs interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent, specifically as follows:

[0014] The multi-trajectory prediction network uses SV Predictor and ADV Planner, respectively, to calculate the self-attention score between the trajectories of surrounding moving obstacles and the target agent using a Transformer module based on multi-head self-attention. The self-attention scores are weighted and merged, and the interactive game process between the surrounding moving obstacles and the target agent is modeled using conditional prediction and conditional planning methods. This captures the interaction features between the surrounding moving obstacles and the target agent at different times, and the interaction features are fused with the scene context encoding to update the scene context encoding features for the surrounding moving obstacles and the target agent. Finally, the trajectories of the surrounding moving obstacles and the target agent are mutually constrained for motion trajectory prediction and path decision-making.

[0015] The technical solution adopted in this application embodiment further includes: the modeling of the interactive game process between the surrounding moving obstacles and the target intelligent agent through conditional prediction and conditional programming methods, specifically:

[0016] Using the future trajectory information as the Query, and the map lane information, map zebra crossing area information, and the other party's future trajectory information as the Key and Value, the similarity score between the Query and each Key is calculated, and the Softmax function is applied to convert the similarity score into a probability distribution. The Value vector is weighted and summarized according to the probability distribution to generate a comprehensive feature expression that integrates environmental features and interactive behavior. The comprehensive feature expression is used to predict the motion trajectory of surrounding moving obstacles and make path decisions for the target intelligent agent.

[0017] The technical solution adopted in this application embodiment further includes: inputting the future trajectory features into a multi-trajectory prediction network, wherein the multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performs interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent, and further includes:

[0018] The path decision trajectory of the target intelligent agent is input into the trajectory optimization module. The trajectory optimization module uses a mathematical model to optimize and iterate the path decision trajectory, and then outputs the optimized path decision trajectory.

[0019] The technical solution adopted in this application embodiment further includes: inputting the path decision trajectory of the target intelligent agent into the trajectory optimization module, wherein the trajectory optimization module uses a mathematical model to optimize and iterate the path decision trajectory, and outputs the optimized path decision trajectory, specifically as follows:

[0020] The trajectory optimization module uses a nonlinear optimizer to post-process the path decision trajectory and optimizes the path decision trajectory by combining a safety cost function, a comfort cost function, a traffic rule cost function, and a speed limit cost function.

[0021] The security cost function is:

[0022]

[0023] in, and ω represents the lateral collision cost and longitudinal collision cost when the minimum lateral and longitudinal distances are less than the safe distance, respectively. s and ω d These represent the weights of the lateral collision cost and the longitudinal collision cost, respectively.

[0024] The comfort cost function is:

[0025]

[0026] Where ω represents the weight and s represents the displacement. Represents acceleration. Represents the rate of change of acceleration;

[0027] The traffic rule cost function is:

[0028]

[0029] in Indicates the cost of direction. ω represents the cost of traffic rules. h and ω l They are respectively and The weights;

[0030] The speed limit cost function is:

[0031]

[0032] Where v t v represents the driving speed. lim The current speed limit;

[0033] By linearly summing the safety cost function, comfort cost function, traffic rule cost function, and speed limit cost function, the final cost function optimization objective of the nonlinear optimizer is obtained as follows:

[0034]

[0035] Where, ω i It is allocated to each individual cost item The weight is t, where t represents the time index.

[0036] Another technical solution adopted in this application embodiment is: an intelligent agent path decision-making device, comprising:

[0037] Multimodal data acquisition module: used to acquire historical multimodal data of the target intelligent agent; the historical multimodal data includes the historical motion characteristics of the target intelligent agent, the historical motion characteristics of surrounding moving obstacles, map lane information, and map zebra crossing area information;

[0038] Multimodal data fusion module: used to input the historical multimodal data into the multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on the Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target intelligent agent and surrounding moving obstacles through the decoder.

[0039] The trajectory prediction module is used to input the future trajectory features into a multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performs interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

[0040] Another technical solution adopted in this application embodiment is: a computer device, the computer device including a processor and a memory coupled to the processor, wherein,

[0041] The memory stores program instructions for implementing the agent path decision-making method;

[0042] The processor is used to execute the program instructions stored in the memory to control the agent's path decision-making method.

[0043] Another technical solution adopted in this application embodiment is: a storage medium storing processor-executable program instructions, the program instructions being used to execute the agent path decision method.

[0044] Compared to existing technologies, the beneficial effects of the embodiments of this application are as follows: The intelligent agent path decision-making method, device, computer equipment, and storage medium of the embodiments of this application capture the spatiotemporal interaction features in historical multimodal data, utilize the historical behavior and future trajectory prediction information of the target intelligent agent and surrounding moving obstacles to engage in interactive games and adjust their respective trajectories, so that the decision model can predict the complex dynamic interaction behaviors that the target intelligent agent and surrounding moving obstacles may have in the future, such as avoidance, following, or meeting oncoming traffic, when generating the trajectory, and dynamically adjust the decision based on the prediction results, so that the final generated SV motion trajectory prediction and ADV path decision trajectory are more consistent with the actual driving situation, avoid the accumulation of errors between trajectory prediction and path decision-making, and improve the accuracy of trajectory prediction and path decision. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the prediction-decision planning system framework based on imitation learning and interactive game theory in the embodiments of this application;

[0046] Figure 2 This is a flowchart of the intelligent agent path decision-making method according to an embodiment of this application;

[0047] Figure 3 This is a schematic diagram of the pre-fusion, post-fusion, and hierarchical fusion algorithms of the multimodal hierarchical feature fusion network in the embodiments of this application;

[0048] Figure 4 This is a schematic diagram of the intelligent agent path decision-making device according to an embodiment of this application;

[0049] Figure 5 This is a schematic diagram of the computer device structure according to an embodiment of this application;

[0050] Figure 6 This is a schematic diagram of the structure of the storage medium according to an embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0052] The terms "first," "second," and "third" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," or "third" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. All directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationships and movements between components in a specific orientation (as shown in the figures). If the specific orientation changes, the directional indications also change accordingly. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or computer device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or computer devices.

[0053] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0054] Specifically, please refer to Figure 1 and Figure 2 , Figure 1 This is a schematic diagram of the prediction-decision planning system framework based on imitation learning and interactive game theory in the embodiments of this application. The system framework includes a multimodal feature encoder, a conditional decoder, and a trajectory optimization module. Figure 2 This is a flowchart of the intelligent agent path decision-making method according to an embodiment of this application. Based on Figure 1 The agent path decision-making method of this application embodiment, as shown in the framework, includes the following steps:

[0055] S10: Obtain historical multimodal data of the target agent over a past period of time;

[0056] In this step, the acquired historical multimodal data includes:

[0057] Historical motion characteristics of the target agent ADV [T] h D ADV ], where T h For a historic moment, D ADV The motion characteristics of the target intelligent agent include its center coordinates, orientation, and vehicle size.

[0058] Historical motion characteristics of surrounding moving obstacles SV [N] SV ,T h D SV ], where N SV D represents the number of moving obstacles in the surrounding area. SV The motion characteristics of the surrounding moving obstacles, such as center coordinates, orientation, and vehicle size;

[0059] Map lane information [N] m N p D m ], where N m Np is the number of lane vectors to search, and D is the number of feature points for each lane vector. m These are the attributes of the feature points, which include, but are not limited to, their coordinates and whether they are allowed to pass under the current traffic rules.

[0060] Map zebra crossing area information [N] c N p D c ], where N c N represents the number of crosswalk vectors being searched. p D represents the number of feature points in each pedestrian vector. c These are the attributes of the feature points.

[0061] S20: Input historical multimodal data into the multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on the Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target intelligent agent and surrounding moving obstacles through the decoder.

[0062] In this step, to address the differences in abstraction levels and encoding computational resources among historical multimodal data, this embodiment uses a multimodal hierarchical feature fusion network to fuse historical multimodal data during the encoding stage. First, historical multimodal data is input into the multimodal hierarchical feature fusion network through different encoders. The multimodal hierarchical feature fusion network employs a Transformer structure to analyze the historical motion features [T] of the target agent. h D ADV ] and historical motion characteristics of surrounding moving obstacles [N SV ,T h D SV The map lane information [N] is encoded and the Vectornet (a hierarchical graph neural network) feature extraction algorithm is used to extract the lane information. m N p D m [N] and zebra crossing area information on the map c N p D c The historical multimodal data is encoded by mapping different encoders to the same feature space and stacking them. Then, a fusion encoder based on a multi-layer Transformer structure further extracts the spatiotemporal interaction features of the historical multimodal data to obtain a more comprehensive feature representation. These spatiotemporal interaction features are then input into a Future Decoder to decode and obtain the future trajectory features of the target agent and surrounding moving obstacles. Specifically, the historical multimodal data fusion algorithm in this application includes early fusion, late fusion, and hierarchical fusion, as detailed below. Figure 3The diagram shows the pre-fusion, post-fusion, and hierarchical fusion algorithms of the multimodal hierarchical feature fusion network in this application embodiment. As shown in (a), the pre-fusion algorithm first encodes historical multimodal data using different feature encoders, then stacks the encoded features. This algorithm considers the differences in encoding and computation for each modality, but lacks the interaction between different modalities. As shown in (b), the post-fusion algorithm directly fuses and encodes the stacked historical multimodal data. This algorithm considers the interaction and fusion between multimodal data, but ignores the differences in the encoding resources required by different modalities. As shown in (c), the hierarchical fusion algorithm combines the advantages of the pre-fusion and post-fusion algorithms, but consumes more computational resources compared to the previous two methods. When the number of network layers and parameters is small, the pre-fusion algorithm has stronger encoding capabilities, while as the number of network parameters gradually increases, the hierarchical fusion algorithm has higher encoding capabilities. In practical applications, a suitable fusion algorithm can be selected according to the application scenario.

[0063] In the multimodal hierarchical feature fusion network of this application embodiment, a fusion encoder containing multiple Transformers is used to further extract the spatiotemporal interaction features of multimodal data, so that single-modal features and cross-modal features are fully extracted. When the learning samples are large enough, the trajectory interaction features in the dataset can be fully learned, which is beneficial to help the model learn and understand the driving scene information of ADV.

[0064] S30: Input the future trajectory features into the multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and the surrounding moving obstacles. It then engages in interactive game based on the future trajectory information of the target agent and the surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

[0065] In this step, traditional methods typically treat the prediction of the trajectory of surrounding moving obstacles as an upstream task independent of path decision-making. The path decision of the agent relies more on static environmental information such as maps and its own historical behavior. However, the multi-trajectory prediction network of this application uses a conditional prediction-based decoder, which fully considers the future trajectory information of SV and ADV, and corrects their respective trajectories by combining map lane information and map zebra crossing area information. It combines the game behavior in the interaction process between SV and ADV to generate the trajectory prediction of SV and the path decision trajectory of ADV. This makes the path decision trajectory of ADV not only dependent on static environmental information and its own historical behavior, but also on dynamic features and potential interactions with surrounding moving obstacles, thereby more accurately reflecting the actual driving situation of the agent and providing the target agent with safer and more efficient path decision-making.

[0066] Furthermore, such as Figure 1As shown, after inputting future trajectory features into the multi-trajectory prediction network, the network obtains future trajectory information of surrounding moving obstacles and the target agent through two modules: SV Predictor and ADV Planner. Specifically, SV Predictor and ADV Planner use a Transformer module based on multi-head self-attention to calculate the self-attention score between the SV and ADV trajectories. These self-attention scores are then weighted and merged, and the interactive game process between SV and ADV is modeled using conditional prediction and conditional planning methods. This captures the interaction features between SV and ADV at different times, thereby quantifying the mutual influence and dependency between their trajectories. Finally, the interaction features are fused with the scene context encoding to update the scene context encoding features for SV and ADV. The trajectories of ADV and SV are mutually constrained for motion trajectory prediction and path decision-making, ensuring that the future trajectories of ADV and SV are mutually constrained and contain rich interactive game information.

[0067] Furthermore, the interactive game process between ADV and SV includes: using the obtained future trajectory information as the Query, and using map lane information, map zebra crossing area information, and the opponent's future trajectory information as the Key and Value, calculating the similarity score between the Query and each Key, and applying the Softmax function to convert the similarity score into a probability distribution to capture environmental features and other agent behaviors that generate the target agent's future trajectory information; weighting and summarizing the Value vector based on the obtained probability distribution to generate a comprehensive feature expression that integrates environmental features and interactive behaviors between agents, and using this comprehensive feature expression to predict the SV's motion trajectory and make path decisions for ADV. This embodiment of the application, through interactive game between the target agent and surrounding moving obstacles, enables the model to predict complex dynamic interactive behaviors that the target agent and surrounding moving obstacles may exhibit in the future, such as avoidance, following, or meeting oncoming traffic, and dynamically adjusts the decision-making behavior based on the prediction results, making the final generated ADV path decision trajectory more consistent with actual driving conditions.

[0068] S40: Input the path decision trajectory of the target intelligent agent into the trajectory optimization module. The trajectory optimization module uses a mathematical model to optimize and iterate the path decision trajectory, and then outputs the optimized path decision trajectory.

[0069] In this step, due to the instability of the imitation learning-based decision model, the generated path decision trajectory cannot be guaranteed to fully comply with safety constraints and traffic rule constraints. Therefore, this embodiment of the application iteratively optimizes the path decision trajectory of the target agent by calling the trajectory optimization module, and finally outputs a safe path decision trajectory that complies with traffic rules. This can effectively reduce the amount of computation, and the iteration using the Gauss-Newton method can ensure the stability and convergence of the optimization process.

[0070] Specifically, the optimization algorithm of the trajectory optimization module includes: post-processing the generated path decision trajectory using a nonlinear optimizer, and optimizing the path decision trajectory by combining multiple cost functions such as safety cost function, comfort cost function, traffic rule cost function, and speed limit cost function, thereby avoiding collisions between the path decision trajectory and the surrounding dynamic and static environment, and adjusting the comfort and motion efficiency of the path decision trajectory. The specific cost functions used by the nonlinear optimizer are as follows:

[0071] (1) Security cost function

[0072] The safety cost function constrains the path decision trajectory to maintain a safe distance from other road users, avoiding collisions. By transforming the target agent's path decision trajectory coordinates to the Frenet coordinate system, the path decision trajectory is decomposed into the longitudinal direction along the planned path and the lateral direction perpendicular to the planned path, and the minimum distance in each direction is calculated:

[0073]

[0074] Where d safe_s The minimum vertical distance, d safe_d The minimum horizontal distance, s t and d t Let represent the vertical and horizontal coordinates of ADV at time t, respectively. and SV i Predicted location at the same time.

[0075] δ s δ d Let represent the lateral and longitudinal safety distances, respectively. When the minimum lateral and longitudinal distances are less than the safety distances, it indicates a risk of collision. The collision cost in this case is calculated using the following formulas; otherwise, the collision cost is 0.

[0076]

[0077] Finally, the collision costs in both directions are linearly added together to obtain the final collision cost:

[0078]

[0079] Where, ω s and ω d These represent the weights of the lateral collision cost and the longitudinal collision cost, respectively.

[0080] The embodiments of this application calculate the collision cost of the target intelligent agent when collision problems may exist, ensuring that the target intelligent agent always maintains a safe distance from surrounding moving obstacles, while effectively reducing the amount of computation.

[0081] (2) Comfort Cost Function

[0082] In the ADV's path decision-making process, to avoid frequent acceleration and deceleration affecting ride comfort, it is necessary to calculate the cost of ADV's acceleration and rate of change of acceleration.

[0083]

[0084] Where ω represents the weight and s represents the displacement. Represents acceleration. This represents the rate of change of acceleration. By weighted summing the absolute values ​​of the ADV's acceleration and the rate of change of acceleration, frequent acceleration or deceleration behaviors of the ADV can be limited, thus improving ride comfort.

[0085] (3) Traffic rule cost function

[0086] In ADV (Advanced Driver Assistance) path decision-making, adherence to traffic rules within the road structure is crucial. To ensure absolute safety in path decision-making, a traffic rule cost function is introduced to more explicitly emphasize constraints such as traffic lights and driving directions, thereby optimizing the path decision trajectory. Specifically, to ensure the ADV's driving direction conforms to traffic rules, a direction cost is introduced:

[0087]

[0088] Where, θ t It is the orientation angle of ADV at time t. It is the orientation angle of the point closest to the ADV on the center line of the lane. By reducing the difference in orientation angle between the ADV and the nearest point on the center line of the lane, the ADV can be made to follow the lane direction more closely.

[0089] At the same time, traffic rule costs are introduced to ensure that ADVs comply with traffic light rules:

[0090]

[0091] Among them, s t Let s be the distance traveled by the ADV along the planned path. stop v represents the position of the stop line on the road when encountering a red light.t′ Let be the speed at time t'. If the ADV crosses the stop line at a red light, a large penalty factor will be introduced, forcing the ADV to stop before the stop line.

[0092] Finally, the directional cost and the traffic rule cost are linearly added together to obtain the final traffic rule cost function:

[0093]

[0094] Where, ω h and ω l Directional cost and traffic rules cost The weight.

[0095] (4) Speed ​​Limitation Cost Function

[0096] To ensure that ADV can travel as efficiently as possible without exceeding the speed limit, a speed limit cost function needs to be introduced:

[0097]

[0098] Where v t v represents the driving speed. lim The current speed limit.

[0099] (5) Fusion of multiple cost functions

[0100] By linearly summing the safety cost function, comfort cost function, traffic rule cost function, and speed limit cost function, the final cost function optimization objective of the nonlinear optimizer is obtained as follows:

[0101]

[0102] Where, ω i It is allocated to each individual cost item The weights are denoted by t, which represents the time index. By assigning appropriate weights to different cost terms, a reasonable path decision trajectory can be generated while satisfying multiple objectives such as safety, comfort, and efficiency. After solving the optimization problem using the Newton-Gaussian method, the optimized path decision trajectory is obtained. Then determine the path trajectory The coordinates are projected back into Cartesian space to obtain path planning results applicable to actual driving environments.

[0103] Based on the above, the intelligent agent path decision-making method of this application captures the spatiotemporal interaction features in historical multimodal data, uses the historical behavior and future trajectory prediction information of the target intelligent agent and surrounding moving obstacles to engage in interactive games and adjust their respective trajectories. This enables the decision-making model to predict complex dynamic interaction behaviors that the target intelligent agent and surrounding moving obstacles may have in the future, such as avoidance, following, or meeting oncoming traffic, when generating trajectories. The model also dynamically adjusts its decisions based on the prediction results, making the final generated SV motion trajectory prediction and ADV path decision trajectory more consistent with actual driving conditions. This avoids the accumulation of errors between trajectory prediction and path decision-making, and improves the accuracy of trajectory prediction and path decision-making.

[0104] Please see Figure 4 This is a schematic diagram of the intelligent agent path decision-making device according to an embodiment of this application. The intelligent agent path decision-making method device 40 according to an embodiment of this application includes:

[0105] Multimodal data acquisition module 41: used to acquire historical multimodal data of the target intelligent agent; the historical multimodal data includes historical motion characteristics of the target intelligent agent, historical motion characteristics of surrounding moving obstacles, map lane information, and map zebra crossing area information;

[0106] Multimodal data fusion module 42: used to input the historical multimodal data into the multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on the Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target intelligent agent and surrounding moving obstacles through the decoder.

[0107] Trajectory prediction module 43: used to input the future trajectory features into a multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performs interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

[0108] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0109] The apparatus provided in this application can be applied to the foregoing method embodiments. For details, please refer to the description of the above method embodiments, which will not be repeated here.

[0110] Please see Figure 5This is a schematic diagram of a computer device structure according to an embodiment of this application. The computer device 50 includes:

[0111] Memory 51 storing executable program instructions;

[0112] Processor 52 connected to memory 51;

[0113] The processor 52 is used to call the executable program instructions stored in the memory 51 and perform the following steps: acquiring historical multimodal data of the target agent; the historical multimodal data includes historical motion features of the target agent, historical motion features of surrounding moving obstacles, map lane information, and map zebra crossing area information; inputting the historical multimodal data into a multimodal hierarchical feature fusion network, the multimodal hierarchical feature fusion network using a Transformer-based fusion encoder to fuse the historical multimodal data, extracting the spatiotemporal interaction features of the historical multimodal data, and obtaining the future trajectory features of the target agent and surrounding moving obstacles through a decoder; inputting the future trajectory features into a multi-trajectory prediction network, the multi-trajectory prediction network using a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performing interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

[0114] The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor.

[0115] Please see Figure 6This is a schematic diagram of the structure of the storage medium in an embodiment of this application. The storage medium in this embodiment stores program instructions 61 capable of implementing the following steps: acquiring historical multimodal data of the target intelligent agent; the historical multimodal data includes historical motion features of the target intelligent agent, historical motion features of surrounding moving obstacles, map lane information, and map zebra crossing area information; inputting the historical multimodal data into a multimodal hierarchical feature fusion network, the multimodal hierarchical feature fusion network using a Transformer-based fusion encoder to fuse the historical multimodal data, extracting the spatiotemporal interaction features of the historical multimodal data, and obtaining the future trajectory features of the target intelligent agent and surrounding moving obstacles through a decoder; inputting the future trajectory features into a multi-trajectory prediction network, the multi-trajectory prediction network using a multi-head self-attention mechanism to obtain the future trajectory information of the target intelligent agent and surrounding moving obstacles, and performing interactive game based on the future trajectory information of the target intelligent agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target intelligent agent. The program instructions 61 can be stored in the aforementioned storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network computer device, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program instructions, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or terminal computer devices such as computers, servers, mobile phones, and tablets. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0116] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, apparatuses, or units, and may be electrical, mechanical, or other forms.

[0117] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units. The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made based on the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A path decision-making method for an intelligent agent, characterized in that, include: Acquire historical multimodal data of the target intelligent agent; the historical multimodal data includes the historical motion characteristics of the target intelligent agent, the historical motion characteristics of surrounding moving obstacles, map lane information, and map zebra crossing area information; The historical multimodal data is input into a multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on the Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target agent and surrounding moving obstacles through a decoder. The future trajectory features are input into a multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles. Based on the future trajectory information of the target agent and surrounding moving obstacles, the network performs interactive game to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

2. The agent path decision-making method according to claim 1, characterized in that, The historical motion characteristics of the target agent are represented as [T] h D ADV ], where T h For a historic moment, D ADV The motion characteristics of the target intelligent agent include center coordinates, direction, and vehicle size; the historical motion characteristics of the surrounding moving obstacles are represented as [N]. SV T h D SV ], where N SV D is the number of moving obstacles in the surrounding area. SV The motion characteristics of the surrounding moving obstacles; The map lane information is represented as [N] m N p D m ], where N m N represents the number of lane vectors to search. p The number of feature points for each lane vector, D m The feature point attributes include coordinates and whether the crossing is permitted under the current traffic rules; the map zebra crossing area information is represented as [N] c N p D c ], where N c N represents the number of crosswalk vectors being searched. p D represents the number of feature points in each pedestrian vector. c These are the attributes of the feature points.

3. The agent path decision-making method according to claim 2, characterized in that, The process involves inputting historical multimodal data into a multimodal hierarchical feature fusion network. This network employs a Transformer-based fusion encoder to fuse the historical multimodal data, extracting its spatiotemporal interaction features, and then decoding the data to obtain the future trajectory features of the target agent and surrounding moving obstacles. Specifically: The multimodal hierarchical feature fusion network employs a Transformer structure to analyze the historical motion features of the target agent and the historical motion features of surrounding moving obstacles [N]. SV T h D SV The map lane information [N] is encoded and the Vectornet feature extraction algorithm is used to extract the features. m N p D m [N] and zebra crossing area information on the map c N p D c The historical multimodal data is encoded, mapped to the same feature space and stacked. Then, the spatiotemporal interaction features of the historical multimodal data are extracted by a fusion encoder based on a multi-layer Transformer structure. The spatiotemporal interaction features are input into a future decoder to decode and obtain the future trajectory features of the target agent and surrounding moving obstacles.

4. The agent path decision-making method according to claim 3, characterized in that, The future trajectory features are input into a multi-trajectory prediction network. This network employs a multi-head self-attention mechanism to acquire the future trajectory information of the target agent and surrounding moving obstacles. Based on this information, the network engages in interactive game theory to generate predicted motion trajectories for the surrounding moving obstacles and the path decision trajectory for the target agent. Specifically: The multi-trajectory prediction network uses the SV Predictor and ADV Planner modules based on multi-head self-attention to calculate the self-attention scores between the trajectories of surrounding moving obstacles and the target agent. These self-attention scores are weighted and merged. Conditional prediction and conditional planning methods are used to model the interactive game process between the surrounding moving obstacles and the target agent, capturing the interaction features between them at different times. These interaction features are fused with scene context encoding to update the scene context encoding features for both the surrounding moving obstacles and the target agent. The trajectories of the surrounding moving obstacles and the target agent are mutually constrained for motion trajectory prediction and path decision-making.

5. The agent path decision-making method according to claim 6, characterized in that, The process of modeling the interaction and game between the surrounding moving obstacles and the target agent using conditional prediction and conditional programming methods is as follows: Using the future trajectory information as the Query, and the map lane information, map zebra crossing area information, and the other party's future trajectory information as the Key and Value, the similarity score between the Query and each Key is calculated, and the Softmax function is applied to convert the similarity score into a probability distribution. The Value vector is weighted and summarized according to the probability distribution to generate a comprehensive feature expression that integrates environmental features and interactive behavior. The comprehensive feature expression is used to predict the motion trajectory of surrounding moving obstacles and make path decisions for the target intelligent agent.

6. The agent path decision-making method according to any one of claims 1 to 5, characterized in that, The future trajectory features are input into a multi-trajectory prediction network. This network employs a multi-head self-attention mechanism to acquire the future trajectory information of the target agent and surrounding moving obstacles. Based on this information, it engages in interactive game theory to generate the predicted motion trajectories of the surrounding moving obstacles and the path decision trajectory of the target agent. The process further includes: The path decision trajectory of the target intelligent agent is input into the trajectory optimization module. The trajectory optimization module uses a mathematical model to optimize and iterate the path decision trajectory, and then outputs the optimized path decision trajectory.

7. The agent path decision-making method according to claim 6, characterized in that, The process involves inputting the path decision trajectory of the target intelligent agent into the trajectory optimization module. The trajectory optimization module then iteratively optimizes the path decision trajectory using a mathematical model and outputs an optimized path decision trajectory. Specifically: The trajectory optimization module uses a nonlinear optimizer to post-process the path decision trajectory and optimizes the path decision trajectory by combining a safety cost function, a comfort cost function, a traffic rule cost function, and a speed limit cost function. The security cost function is: in, and ω represents the lateral collision cost and longitudinal collision cost when the minimum lateral and longitudinal distances are less than the safe distance, respectively. s and ω d These represent the weights of the lateral collision cost and the longitudinal collision cost, respectively. The comfort cost function is: Where ω represents the weight and s represents the displacement. Represents acceleration. Represents the rate of change of acceleration; The traffic rule cost function is: in Indicates the cost of direction. ω represents the cost of traffic rules. h and ω l They are respectively and The weights; The speed limit cost function is: Where v t v represents the driving speed. lim The current speed limit; By linearly summing the safety cost function, comfort cost function, traffic rule cost function, and speed limit cost function, the final cost function optimization objective of the nonlinear optimizer is obtained as follows: Where, ω i It is allocated to each individual cost item The weight is t, where t represents the time index.

8. A path decision-making device for an intelligent agent, characterized in that, include: Multimodal data acquisition module: used to acquire historical multimodal data of the target intelligent agent; the historical multimodal data includes the historical motion characteristics of the target intelligent agent, the historical motion characteristics of surrounding moving obstacles, map lane information, and map zebra crossing area information; Multimodal data fusion module: used to input the historical multimodal data into the multimodal hierarchical feature fusion network. The multimodal hierarchical feature fusion network uses a fusion encoder based on the Transformer structure to fuse the historical multimodal data, extract the spatiotemporal interaction features of the historical multimodal data, and obtain the future trajectory features of the target intelligent agent and surrounding moving obstacles through the decoder. The trajectory prediction module is used to input the future trajectory features into a multi-trajectory prediction network. The multi-trajectory prediction network uses a multi-head self-attention mechanism to obtain the future trajectory information of the target agent and surrounding moving obstacles, and performs interactive game based on the future trajectory information of the target agent and surrounding moving obstacles to generate the motion trajectory prediction of the surrounding moving obstacles and the path decision trajectory of the target agent.

9. A computer device, characterized in that, The computer device includes a processor and a memory coupled to the processor, wherein, The memory stores program instructions for implementing the agent path decision-making method according to any one of claims 1-7; The processor is used to execute the program instructions stored in the memory to control the agent's path decision-making method.

10. A storage medium, characterized in that, The system stores processor-executable program instructions for performing the agent path decision method according to any one of claims 1 to 7.

Images