Agent interaction scenario construction method and apparatus, and behavior prediction model training method and apparatus

Abstract

The present disclosure relates to the technical fields of intelligent mines, driverless driving technologies, and maps. Disclosed are an agent interaction scenario construction method and apparatus, and a behavior prediction model training method and apparatus. The agent interaction scenario construction method comprises: acquiring sensor data of a plurality of agents and map data of geographic locations corresponding to the sensor data; for the sensor data of each agent, performing trajectory detection on the detected other agents represented by the sensor data of the agent to obtain a first trajectory; determining a second trajectory of the agent on the basis of the sensor data; on the basis of a scenario data admission condition corresponding to a preset scenario, screening the first trajectory and / or the second trajectory to obtain a target trajectory; and merging the map data and the target trajectory to obtain merged scenario data. The present disclosure can improve the efficiency and accuracy of constructing agent interaction scenarios.

Description

Methods for constructing intelligent agent interaction scenarios, methods and devices for training behavior prediction models Technical Field

[0001] This disclosure relates to the fields of smart mines, autonomous driving technology, and map technology, and in particular to a method for constructing intelligent agent interaction scenarios, a method and apparatus for training behavior prediction models. Background Technology

[0002] In the field of trajectory prediction with interactive scenarios, past technical solutions mainly relied on motion models of independent agents or rule-based methods. These methods performed reasonably well in simple, non-interactive scenarios, but their limitations became significant in complex interactive environments, such as urban traffic, mines, and docks. Traditional methods mainly fall into three categories: 1) Single-agent models: These methods assume that the movement of each agent is independent of other agents and learn the prediction model through historical trajectory data. This assumption ignores the interaction between agents, leading to a significant drop in prediction accuracy when predicting highly interactive scenarios, such as vehicle encounters and pedestrian avoidance. 2) Rule-based methods: Some solutions attempt to simulate the interaction between agents by formulating rules, such as methods based on social force models. Although this method considers the mutual influence between agents to some extent, the rule setting is highly dependent on manual intervention, making it difficult to cover all possible interactive scenarios and exhibiting poor adaptability to new scenarios. 3) Multi-agent models: These models simultaneously predict the movement of all agents in the scenario and learn the prediction model through historical trajectory data.

[0003] However, these existing technical solutions have significant shortcomings. First, single-agent models neglect the interactions between agents, leading to poor prediction accuracy in complex scenarios. Second, while rule-based methods consider interactions, they struggle to cover all possible interaction scenarios and lack flexibility. Finally, although multi-agent models consider the overall scenario, the diverse and complex interaction patterns result in numerous local optima with low distinguishability between them. This makes the output susceptible to any inaccurate prediction, thus affecting the overall accuracy of the scenario prediction. Furthermore, these methods generally suffer from strong data dependence and poor adaptability to new scenarios, making it difficult to meet the high accuracy and robustness requirements for trajectory prediction in practical applications. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and system for constructing intelligent agent interaction scenarios, which improves the efficiency and accuracy of intelligent agent interaction scenario construction methods.

[0005] In a first aspect, this disclosure provides a method for constructing an intelligent agent interaction scenario. The method includes: acquiring sensor data of multiple intelligent agents and map data of the geographical locations corresponding to the sensor data; for each intelligent agent's sensor data, performing trajectory detection on other intelligent agents detected by the sensor data of that intelligent agent to obtain a first trajectory; and determining a second trajectory of the intelligent agent based on the sensor data; filtering the first trajectory and / or the second trajectory according to the scene data admission conditions corresponding to the preset scene to obtain a target trajectory; and merging the map data and the target trajectory to obtain merged scene data.

[0006] Secondly, this disclosure provides a method for training a behavior prediction model, which includes: obtaining a trajectory to be trained from merged scene data; dividing each trajectory to be trained into a first historical trajectory and a second historical trajectory according to time sequence; inputting the first historical trajectory into the encoder of an encoder-decoder model for encoding to obtain an encoding vector; inputting the encoding vector into a multi-level decoder of the encoder-decoder model, and predicting the trajectories of multiple consecutive sub-time periods through the multi-level decoder; wherein the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided; concatenating the predicted trajectories of the multiple consecutive sub-time periods and comparing them with the second historical trajectory; and training the encoder-decoder model based on the comparison results to obtain a multi-agent interaction behavior prediction model.

[0007] Thirdly, this disclosure provides an intelligent agent interaction scene construction device, which includes:

[0008] The acquisition module is configured to acquire sensor data from multiple agents and map data showing the geographical locations corresponding to the sensor data.

[0009] The detection module is configured to perform trajectory detection on other detected agents represented by the sensor data of each agent to obtain a first trajectory; and determine a second trajectory of the agent based on the sensor data.

[0010] The filtering module is configured to filter the first trajectory and / or the second trajectory based on the scene data admission conditions corresponding to the preset scene to obtain the target trajectory;

[0011] The merge module is configured to merge map data and target trajectories to obtain merged scene data.

[0012] Fourthly, this disclosure provides a behavior prediction model training device, comprising:

[0013] The acquisition module is configured to obtain the trajectory to be trained from the merged scene data;

[0014] The segmentation module is set to divide each training trajectory into a first historical trajectory and a second historical trajectory according to the time sequence.

[0015] The encoding module is configured to input the first historical trajectory into the encoder of the encoder-decoder model for encoding to obtain an encoded vector.

[0016] The prediction module is configured to input the encoded vector into the multi-level decoder of the encoder-decoder model, and predict the trajectory of multiple consecutive sub-time periods through the multi-level decoder; wherein, the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided.

[0017] The stitching module is configured to stitch together the predicted trajectories of multiple consecutive sub-time periods and compare them with the second historical trajectory.

[0018] The training module is configured to train the encoder-decoder model based on the comparison results to obtain a multi-agent interaction behavior prediction model.

[0019] Fifthly, this disclosure provides a cloud platform, including: an intelligent agent interaction scene construction device and a behavior prediction model training device.

[0020] The technical solution provided in this disclosure performs trajectory detection on other detected agents represented by the sensor data of each agent to obtain a first trajectory. Simultaneously, a second trajectory for the agent is determined based on the same sensor data. This dual-trajectory processing method effectively improves the comprehensiveness and accuracy of trajectory data, enabling better capture of the interactive behaviors and relationships between agents. Secondly, based on the scene data admission conditions corresponding to the preset scene, the first and / or second trajectories are filtered to obtain the target trajectory, effectively reducing the impact of data noise and outliers on the final scene construction. Finally, the map data and the target trajectory are merged to obtain merged scene data, significantly improving the quality and reliability of the constructed scene. This disclosure not only accurately captures the motion characteristics of individual agents but also reflects the complex interactive relationships between agents. Furthermore, by combining map data, the constructed scene is closer to the actual environment, enhancing the applicability and accuracy of the model in practical applications. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 is a schematic diagram of an embodiment of the intelligent agent interaction scene construction method in this disclosure;

[0023] Figure 2 is a schematic diagram of an embodiment of the behavior prediction model training method in this disclosure;

[0024] Figure 3 is a schematic diagram of the prediction results of a common decoder in an embodiment of this disclosure;

[0025] Figure 4 is a schematic diagram of the prediction results of the decoder of the method in this embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of an embodiment of the intelligent agent interaction scene construction device in this disclosure;

[0027] Figure 6 is a schematic diagram of an embodiment of the behavior prediction model training device in this disclosure. Detailed Implementation

[0028] This disclosure provides a method, apparatus, and system for constructing intelligent agent interaction scenarios. The terms "first," "second," "third," "fourth," etc. (if applicable) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0029] For ease of understanding, the specific process of the embodiments of this disclosure is described below. Please refer to Figure 1. One embodiment of the intelligent agent interaction scene construction method in this disclosure includes:

[0030] Step S101: Obtain sensor data from multiple agents and map data showing the geographical locations corresponding to the sensor data;

[0031] Step S102: For the sensor data of each agent, perform trajectory detection on other agents detected by the sensor data of the agent to obtain the first trajectory; and determine the second trajectory of the agent based on the sensor data.

[0032] Step S103: Based on the scene data admission conditions corresponding to the preset scene, filter the first trajectory and / or the second trajectory to obtain the target trajectory;

[0033] Step S104: Merge the map data and the target trajectory to obtain merged scene data.

[0034] It is understood that the executing entity of this disclosure can be an intelligent agent interaction scenario construction device, or it can be a terminal, an unmanned vehicle, or a server; no specific limitation is made here. This disclosure uses a server as an example for illustration in its embodiments.

[0035] Optionally, the server acquires sensor data from multiple intelligent agents, along with map data showing the corresponding geographical locations. This data originates from devices such as vehicle cameras, LiDAR, and GPS, and includes the agent's own position and speed information, as well as perception data of the surrounding environment. For example, an autonomous vehicle might collect its own GPS coordinates, speed, and the relative positions and movements of surrounding vehicles and pedestrians. For each agent's sensor data, the server performs trajectory detection on other agents detected by that agent, obtaining a first trajectory. This process involves processing the raw data, including semantic segmentation, object detection, and object tracking. Semantic segmentation divides image data into different semantic regions, such as roads, vehicles, and pedestrians. Object detection identifies specific target objects, such as other vehicles or pedestrians. Object tracking tracks the movement trajectories of these target objects using continuous frame data. Simultaneously, based on the agent's own sensor data, a second trajectory is determined. These two steps, combined, consider both the agent's own movement and the movement of other surrounding agents, laying the foundation for subsequent interaction prediction.

[0036] Then, the server filters the first and / or second trajectories according to the preset scenario data admission criteria to obtain the target trajectory. The admission criteria are set based on business constraints on data characteristics, including data health features, trajectory-agent relationship features, and spatiotemporal features. Data health features include data integrity, duration, and trajectory shape; trajectory-agent relationship features include whether they are adjacent, whether there is interaction, and specific interaction behaviors such as following, overtaking, and crossing; spatiotemporal features include average driving speed, acceleration, and trajectory curvature. For example, admission criteria might require data integrity, a duration greater than 9 seconds, trajectory curvature meeting curve requirements, and overtaking trajectories with speeds greater than 30 km / h. These conditions ensure that the data entering subsequent processing has sufficient quality and representativeness. Finally, the server merges the map data and the target trajectory to obtain merged scenario data. This step combines the filtered trajectory data with geographic environmental information to form a complete scenario description. The merged scenario data not only includes the agent's movement trajectory but also environmental information such as road structure and traffic signs, providing rich context for subsequent interaction prediction.

[0037] For example, suppose in a crossroads scenario, the server receives sensor data from multiple vehicles. Vehicle A is proceeding straight through the intersection, while vehicle B is preparing to turn left. Vehicle A's sensor data shows it is traveling straight at 50 km / h, while simultaneously detecting that vehicle B is decelerating and activating its left turn signal. Vehicle B's sensor data shows its speed decreasing from 40 km / h to 20 km / h, and its steering wheel gradually turning to the left. The server processes this raw data to obtain the precise trajectories of the two vehicles. Then, based on acceptance criteria (such as data integrity > 95%, duration > 10 seconds, and involvement of intersection turning operations), these trajectories are deemed valid data. Finally, this trajectory data is merged with the intersection's map data (including road width, traffic light locations, etc.) to form a complete description of the interactive scenario.

[0038] By performing the above steps, for each agent's sensor data, trajectory detection is performed on other agents detected by that agent's sensor data to obtain a first trajectory. Simultaneously, a second trajectory for that agent is determined based on the same sensor data. This dual-trajectory processing method effectively improves the comprehensiveness and accuracy of trajectory data, enabling better capture of interactions and relationships between agents. Secondly, based on the scene data admission criteria corresponding to the preset scene, the first and / or second trajectories are filtered to obtain the target trajectory, effectively reducing the impact of data noise and outliers on the final scene construction. Finally, the map data and target trajectories are merged to obtain merged scene data, significantly improving the quality and reliability of the constructed scene. This disclosure not only accurately captures the motion characteristics of individual agents but also reflects the complex interactive relationships between agents. Furthermore, by combining map data, the constructed scene is closer to the actual environment, enhancing the model's applicability and accuracy in practical applications.

[0039] In one specific embodiment, trajectory detection is performed on other detected intelligent agents represented by the sensor data of the intelligent agent to obtain a first trajectory, specifically including the following steps:

[0040] (1) For the sensor data of each agent, perform semantic segmentation on the sensor data of the agent; perform target detection on the segmented data; and perform target tracking on the detected target agents;

[0041] (2) Based on the target tracking results corresponding to the sensor data of multiple agents, determine the target tracking data corresponding to the same target agent;

[0042] (3) For each target agent, determine the first trajectory corresponding to the target agent based on the target tracking data corresponding to the target agent.

[0043] Optionally, in the intelligent agent interaction scenario construction method, the intelligent agent specifically refers to an autonomous vehicle. For the sensor data of each autonomous vehicle, semantic segmentation is performed. Semantic segmentation technology classifies each pixel in the original image into a specific category, such as road, other vehicles, traffic signs, etc. In the autonomous driving scenario, a deep learning model specifically trained for the road environment, such as DeepLab or PSPNet, is used to convert the original image into a semantically labeled map, where each color represents a specific element in the road scene. Then, object detection is performed on the segmented data. In the autonomous driving scenario, object detection algorithms such as YOLOv5 or Faster R-CNN are optimized for identifying and locating other vehicles. These algorithms output a list containing vehicle types and precise bounding box coordinates. Object detection pays special attention to other autonomous vehicles for subsequent interaction prediction.

[0044] Subsequently, target tracking is performed on the detected autonomous vehicles. Common target tracking algorithms in autonomous driving systems include improved Kalman filters or SORT (Simple Online and Realtime Tracking). These algorithms are specifically optimized for vehicle motion characteristics, enabling stable tracking even under high-speed motion and partial occlusion conditions. Target tracking assigns a unique ID to each detected autonomous vehicle and predicts its position, velocity, and acceleration in consecutive frames.

[0045] After obtaining target tracking results from sensor data of multiple autonomous vehicles, the target tracking data corresponding to the same target autonomous vehicle is determined. Data association uses an improved Hungarian algorithm or the Joint Probabilistic Data Association (JPDA) method to associate the observation results of different sensors (such as cameras, lidar, and millimeter-wave radar) with the same target autonomous vehicle. Data fusion employs methods such as extended Kalman filtering or particle filtering to comprehensively consider information from multiple sensors to obtain a more accurate state estimate of the target autonomous vehicle. Finally, for each target autonomous vehicle, the first trajectory corresponding to that target autonomous vehicle is determined based on its corresponding target tracking data. In autonomous driving scenarios, trajectory generation typically uses vehicle kinematics models combined with curve fitting techniques. For example, using the constant acceleration model (CA) or constant steering rate and acceleration model (CTRV) commonly used in adaptive cruise control (ACC) systems, combined with Bezier curve fitting, discrete tracking points are converted into continuous trajectory curves that conform to the vehicle's motion characteristics. This method not only fills in gaps in the observation data but also reduces the impact of measurement noise, generating smoother trajectories that conform to the actual vehicle motion patterns.

[0046] For example, an autonomous vehicle (vehicle A) is collecting data. Its LiDAR and camera systems generate 10 frames of point cloud and image data per second. First, a semantic segmentation algorithm processes this data, distinguishing roads, other vehicles, guardrails, etc. In one frame, the algorithm identifies three other autonomous vehicles. The object detection algorithm precisely locates these vehicles. For example, for one of the vehicles (target B), the algorithm outputs its bounding box coordinates as (x = 50m, y = 3.5m, length = 4.5m, width = 1.8m) with a confidence score of 0.98. The target tracking algorithm then tracks target B in consecutive frames, recording its position changes. Suppose that in one second (10 frames), the center point coordinates of target B are: (50m, 3.5m), (52m, 3.5m), ..., (70m, 3.5m). The data association step associates these observations from the LiDAR and camera with the same target B. The data fusion step combines data from two sensors to obtain more accurate position and velocity estimates. Finally, a continuous trajectory of target B is generated using a vehicle kinematics model and curve fitting techniques. For example, assuming the use of a CTRV model, a smooth trajectory function can be obtained, which not only accurately represents the observed position but also reasonably estimates the intermediate unobserved positions, while ensuring that the trajectory conforms to the vehicle's motion characteristics. This disclosure can accurately capture the motion trajectories of various autonomous vehicles and provide rich interactive information.

[0047] In one specific embodiment, the first trajectory and / or the second trajectory are filtered according to the scene data admission conditions corresponding to the preset scene to obtain the target trajectory, including the following steps:

[0048] (1) Determine the trajectory feature indicators corresponding to the driving trajectory based on the scenario data access conditions corresponding to the preset scenario;

[0049] (2) For each trajectory of the first trajectory and / or the second trajectory, data feature extraction is performed on each trajectory feature index to obtain the trajectory feature data corresponding to each feature index of the trajectory.

[0050] (3) Based on the admission criteria and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, the trajectory is filtered to obtain the target trajectory.

[0051] Optionally, based on the scenario data admission conditions corresponding to the preset scenario, trajectory feature indicators corresponding to the driving trajectory are determined. These feature indicators include data health feature indicators, trajectory-agent relationship feature indicators, and spatiotemporal feature indicators. Data health feature indicators are used to characterize the completeness, duration, and shape of the trajectory data. Trajectory-agent relationship feature indicators are used to characterize the interaction behavior characteristics of the trajectory-corresponding agent with other agents and its positional relationship with its own trajectory. Spatiotemporal feature indicators are used to characterize the spatiotemporal position characteristics, motion parameter characteristics, and trajectory shape characteristics of the trajectory-corresponding agent.

[0052] For each trajectory of the first and / or second trajectories, data features are extracted for each trajectory feature index to obtain trajectory feature data corresponding to each feature index. This process involves multiple data processing techniques. For data health features, integrity is assessed by calculating the density and continuity of data points, duration is calculated using timestamp differences, and trajectory shape is evaluated through curvature analysis. Extraction of trajectory-agent relationship features involves calculating the relative distance, relative velocity, and acceleration between agents, as well as identifying specific interaction patterns such as following, overtaking, or avoidance behaviors. Extraction of spatiotemporal features includes calculating motion parameters such as average velocity, acceleration, and steering angular rate, as well as the spatial distribution characteristics of the trajectory.

[0053] Finally, based on the admission criteria and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, trajectory filtering is performed to obtain the target trajectory. This involves setting thresholds or ranges for each feature index and evaluating the feature data of each trajectory. Only trajectories that simultaneously meet the requirements of all feature indices will be selected as target trajectories.

[0054] For example, suppose in a highway scenario, the preset data admission criteria require trajectory data to meet the following conditions: data integrity greater than 95%, duration greater than 10 seconds, average speed between 60 km / h and 120 km / h, and including at least one lane change. Based on these conditions, the trajectory feature indicators to be extracted include: data point density, trajectory duration, average speed, and lateral displacement.

[0055] For a specific trajectory, the data processing procedure is as follows: First, the data point density is calculated. Assuming the total trajectory length is 1000 meters and there are 980 valid data points, the data integrity is 98%, which meets the requirements. Second, the difference between the start and end timestamps of the trajectory is calculated, yielding a duration of 15 seconds, which also meets the condition. Then, the average speed is calculated by dividing the total travel distance by the time, resulting in an average speed of 90 km / h, which is within the required range. Finally, the lateral displacement data is analyzed, revealing a lateral movement exceeding 3.5 meters in the middle of the trajectory, which is identified as a lane change.

[0056] Optionally, the trajectory feature indicators include at least one of the following: data health feature indicators, trajectory-agent relationship feature indicators, and spatiotemporal feature indicators;

[0057] Data health feature indicators are used to characterize trajectory data by at least one of the following: data integrity, duration, and the shape of the trajectory; trajectory-agent relationship feature indicators are used to characterize the interaction behavior characteristics of the trajectory-corresponding agent's motion with other agents and / or the positional relationship characteristics with its own trajectory; spatiotemporal feature indicators are used to characterize the trajectory-corresponding agent by at least one of the following spatiotemporal position characteristics, motion parameter characteristics, and trajectory shape characteristics.

[0058] In one specific embodiment, trajectory filtering is performed based on the admission criteria and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory to obtain the target trajectory, including the following steps:

[0059] (1) For each trajectory feature indicator, determine the admission criteria and the judgment criteria for that trajectory indicator; and compare the trajectory feature data corresponding to the trajectory feature indicator extracted for each trajectory with the corresponding judgment criteria.

[0060] (2) The trajectory whose trajectory feature data all meet the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

[0061] Optionally, trajectory feature indicators include data health feature indicators, trajectory-agent relationship feature indicators, and spatiotemporal feature indicators. Data health feature indicators are used to evaluate the quality of trajectory data, including data integrity, duration, and the shape of the represented trajectory. Data integrity reflects the continuity and density of trajectory data points, duration represents the time span of the trajectory, and trajectory shape describes the geometric characteristics of the trajectory. Trajectory-agent relationship feature indicators focus on the interactive behaviors between agents, such as following, overtaking, and avoiding, as well as the positional relationship between the agent and its own trajectory, such as lane keeping or lane changing behaviors. Spatiotemporal feature indicators comprehensively describe the motion characteristics of the agent, including spatiotemporal position features (such as coordinate changes), motion parameter features (such as velocity and acceleration), and trajectory shape features (such as curvature and steering angle).

[0062] In the specific implementation process, firstly, for each trajectory characteristic indicator, the specific judgment conditions for that trajectory indicator are determined. For example, for data integrity, 90% might be set as the threshold; for duration, a minimum of 10 seconds might be required; and for average speed, it might be specified to be between 40 km / h and 120 km / h. The setting of these judgment conditions is based on specific application scenarios and research needs, and requires the support of expert knowledge and data analysis.

[0063] Then, the trajectory feature data corresponding to the trajectory feature indicators extracted for each trajectory are compared one by one with these judgment conditions. For example, the data point density of the trajectory is calculated and compared with the integrity threshold, the time span of the trajectory is calculated and compared with the minimum duration requirement, and the speed distribution of the trajectory is analyzed and checked to see if it is within the specified range. For more complex features, such as interaction behavior, pattern recognition algorithms may be needed to determine whether a specific interaction pattern exists.

[0064] Finally, only those trajectories whose trajectory feature data all meet the judgment conditions corresponding to each trajectory feature index will be identified as the target trajectory.

[0065] For example, suppose we are in a large open-pit mine's unmanned transportation scenario, and we need to select high-quality trajectories that can represent complex interactive behaviors. The set criteria include: data integrity greater than 95%, duration greater than 5 minutes, average speed between 15 km / h and 40 km / h, including at least one turn or gradient change, and at least one interaction with other vehicles or fixed facilities (such as avoiding other vehicles or passing a loading point). For a specific unmanned mining truck trajectory, the data processing procedure is as follows: First, calculate the data point density. Assuming the total trajectory length is 3000 meters and the sampling frequency is 1 Hz, theoretically there should be 3000 data points. In reality, 2940 valid points were recorded, resulting in a data integrity of 98%, meeting the requirement of greater than 95%. Second, calculate the difference between the start and end timestamps of the trajectory, obtaining a duration of 6 minutes and 12 seconds, meeting the condition of greater than 5 minutes. Then, by dividing the total travel distance by the time, we obtain an average speed of 29 km / h, which falls within the required range of 15 km / h to 40 km / h. Next, analysis of the three-dimensional displacement data revealed a turn exceeding 30 degrees in the middle of the trajectory, lasting approximately 20 seconds, and a 7% uphill section, satisfying the requirements for turning and gradient change. Finally, analysis of interactions with the surrounding environment identified a stop at a loading point where ore was loaded, during which the vehicle speed decreased from 25 km / h to 0, remained stationary for approximately 3 minutes, and then restarted. The entire process was clearly recorded, demonstrating the interaction with the fixed infrastructure. Based on these analyses, the trajectory met all the set entry criteria and was therefore selected as the target trajectory. This rigorous data screening process is particularly suitable for mining environments due to their complex terrain, interaction with large equipment, and stringent safety requirements. Selecting high-quality trajectory data helps optimize the performance of autonomous driving systems in critical areas such as steep slopes, narrow turns, and loading / unloading points. This not only improves the efficiency and accuracy of subsequent research but also provides more reliable data support for the decision-making and control of autonomous vehicles in complex mining environments, ultimately enhancing the safety and efficiency of mining operations.

[0066] In one specific embodiment, the method for constructing an intelligent agent interaction scenario may further include the following steps:

[0067] (1) Determine the distribution characteristic indicators corresponding to the merged scene data based on the scene data distribution conditions corresponding to the preset scene;

[0068] (2) Extract scene distribution data features from the merged scene data for each distribution feature index;

[0069] (3) Based on the characteristics of the extracted scene distribution data, determine whether the current merged scene data meets the distribution conditions; if not, perform enhancement processing on the scene data to obtain the target merged scene data.

[0070] Optionally, the distribution characteristic indicators include at least one of the following: business scenario characteristic indicators, time distribution characteristic indicators, and spatial distribution characteristic indicators.

[0071] In one specific embodiment, based on the extracted scene distribution data characteristics, it is determined whether the currently merged scene data meets the distribution conditions; if not, the scene data is enhanced, including the following steps:

[0072] (1) For each distribution feature index, determine whether the extracted scene distribution data features can cover the scene distribution represented by the distribution feature index;

[0073] (2) If it cannot be covered, identify the missing scene data and supplement the missing scene data.

[0074] Optionally, based on the scenario data distribution conditions corresponding to the preset scenarios, distribution characteristic indicators corresponding to the merged scenario data are determined. These distribution characteristic indicators include business scenario characteristic indicators, temporal distribution characteristic indicators, and spatial distribution characteristic indicators. Business scenario characteristic indicators reflect the distribution of different types of interaction scenarios, such as overtaking, following, and meeting oncoming traffic; temporal distribution characteristic indicators indicate the distribution of data in different time periods, such as morning and evening rush hours, daytime, and nighttime; spatial distribution characteristic indicators describe the distribution of data in different geographical locations or road types, such as highways, urban roads, and intersections.

[0075] Furthermore, for each distribution characteristic indicator, scene distribution data features are extracted from the merged scene data. For business scene features, different types of interaction scenarios are identified and statistically analyzed by analyzing the relative motion patterns between intelligent agents. For example, changes in relative distance, relative speed, and acceleration between vehicles are calculated to determine whether overtaking, following, or passing behaviors have occurred. For temporal distribution features, the data is classified and statistically analyzed based on timestamp information to calculate the proportion of data in different time periods. For spatial distribution features, Geographic Information System (GIS) technology is used to perform spatial clustering and distribution analysis on the data to determine the distribution of data on different types of roads.

[0076] Then, based on the extracted scene distribution data features, it is determined whether the currently merged scene data conforms to the preset distribution conditions. For each distribution feature indicator, it is determined whether the extracted scene distribution data features can cover the scene distribution represented by that distribution feature indicator. If certain types of scene data are found to be insufficient or excessive, data augmentation processing is required. Specifically, if certain scene distributions cannot be covered, the missing scene data needs to be identified and supplemented. Data augmentation methods include, but are not limited to: oversampling or data synthesis for scarce scenes, undersampling for excessive scenes, or increasing data diversity through data transformation techniques. For example, for lacking nighttime data, nighttime driving conditions can be simulated by adjusting sensor parameters, or some daytime data can be converted into nighttime data using time mapping techniques. For insufficient specific interaction scenarios, such as lane-changing behavior, new lane-changing scene data can be generated by making minor transformations to existing lane-changing data (such as adjusting the start and end positions of lane changes, changing the duration of lane changes, etc.).

[0077] For example, suppose in an urban autonomous driving scenario, the preset data distribution conditions require that the proportion of data for different interaction types (such as going straight, turning, and changing lanes) should be balanced, the temporal distribution should cover the entire 24 hours, and the spatial distribution should include main roads, secondary roads, and residential streets. After extracting the distribution features, it was found that in the original data, straight-going scenarios accounted for 70%, turning scenarios accounted for 20%, and lane-changing scenarios accounted for only 10%; in terms of temporal distribution, data from nighttime (22:00-6:00) accounted for only 5%; and in terms of spatial distribution, data from residential streets accounted for only 10%.

[0078] To address this uneven distribution, the specific steps of data augmentation are as follows: First, data synthesis and oversampling are performed on lane-changing scenarios. New lane-changing scenario data is generated by making minor transformations to existing data (such as adjusting the start and end positions of lane changes, changing the duration of lane changes, etc.), increasing the proportion of lane-changing scenarios to approximately 25%. Second, nighttime driving conditions are simulated by adjusting sensor parameters, or time mapping techniques are used to convert some daytime data into nighttime data, increasing the proportion of nighttime data to 20%. Finally, data transformation techniques are used to adjust the data characteristics of some main and secondary roads (such as road width, lane type, etc.) to match the characteristics of residential streets, increasing the proportion of residential street data to 25%. Through this data augmentation process, the final target merged scenario data achieves the preset balance requirements in terms of interaction type, temporal distribution, and spatial distribution. This method not only ensures the comprehensiveness and representativeness of the dataset but also expands the originally scarce scenario data through data augmentation techniques, providing richer and more balanced training samples for training multi-agent interaction prediction models.

[0079] In one specific embodiment, before merging the map data and the target trajectory, the following steps may be included:

[0080] (1) Perform stationary point detection processing on the target trajectory to obtain stationary point detection trajectory data;

[0081] (2) Determine the data segment to be filled corresponding to the target trajectory based on the station detection trajectory data, and perform trajectory interpolation processing on the data segment to be filled to obtain the interpolated trajectory data;

[0082] (3) Perform trajectory smoothing processing on the interpolated trajectory data to obtain the enhanced target trajectory;

[0083] (4) Merge the map data and the target trajectory to obtain merged scene data, including: merging the map data and the enhanced target trajectory to obtain merged scene data.

[0084] Optionally, stall detection processing is used to identify locations in the trajectory where the vehicle has stopped or nearly stopped moving. Stall detection refers to identifying time periods in the trajectory data where an agent (such as an autonomous vehicle) remains stationary or moves extremely slowly. These stalls represent waiting at traffic lights, traffic congestion, loading / unloading, or other situations requiring the vehicle to stop. Specific implementations of stall detection are typically based on speed and time thresholds. For example, a stall can be defined as a location where the vehicle's average speed is below a certain threshold (e.g., 1 km / h) over a certain time period (e.g., 10 seconds). The stall detection algorithm iterates through the entire trajectory, calculates the average speed within each time window, and marks points that meet the criteria as stalls. The result of this process is the stall detection trajectory data, which contains the original trajectory points and a label indicating whether each point is a stall.

[0085] Then, based on the stationary point detection trajectory data, the data segments to be filled corresponding to the target trajectory are determined. These data segments typically appear between two stationary points, or in areas where data is missing due to signal loss during data acquisition. Trajectory interpolation is performed on these data segments to obtain interpolated trajectory data. The purpose of trajectory interpolation is to fill in missing trajectory points, making the trajectory more continuous and smooth. Commonly used interpolation methods include linear interpolation, spline interpolation, or Bézier curve interpolation. For example, using cubic spline interpolation can generate a curve that better reflects the actual motion characteristics of the vehicle while maintaining trajectory continuity. After interpolation, the interpolated trajectory data is smoothed to obtain the enhanced target trajectory. The purpose of smoothing is to reduce noise and irregular fluctuations in the trajectory, making it more consistent with the actual motion characteristics of the vehicle. Commonly used smoothing methods include moving average, Gaussian filtering, or Kalman filtering. These methods can effectively remove trajectory jitter caused by GPS or other sensor errors while preserving the main characteristics of the trajectory.

[0086] Finally, the smoothed enhanced target trajectory is merged with the map data to obtain the final merged scene data. This merging process includes not only mapping trajectory points to their corresponding locations on the map, but also associating the trajectory with information such as roads and traffic signs on the map. The resulting merged scene data contains both high-quality vehicle motion trajectories and rich environmental context information.

[0087] For example, suppose we have raw trajectory data recording the movement of an unmanned mining truck in an open-pit mine. The total trajectory length is 10 kilometers, containing 2000 sampling points with a sampling interval of 2 seconds. First, we perform stationary point detection. Considering the characteristics of mining vehicles, we set a speed threshold of 0.5 km / h and a time threshold of 30 seconds. These parameters are chosen to account for the possibility that mining vehicles may need to remain at low speeds or stationary for extended periods at loading / unloading points and on steep slopes. By calculating the average speed over every 15 consecutive points (corresponding to 30 seconds), we find that the average speeds at points 200-230, 1000-1040, and 1600-1640 are all below 0.5 km / h, therefore these points are marked as stationary points. These stationary points may correspond to loading points, unloading points, and steep slope waiting areas, respectively.

[0088] Furthermore, analysis of the trajectory segments between the outposts revealed data gaps between points 600 and 700, likely due to signal obstruction in the mining area. Cubic spline interpolation was performed on this data to generate 100 new trajectory points, making the trajectory continuous in this region. The interpolation considered the characteristics of the mine terrain, ensuring the generated trajectory points corresponded to the actual path direction and slope changes. Then, Gaussian filtering was used to smooth the entire trajectory, with a window size of 7 points and a standard deviation of 2.0. These parameters were chosen to account for significant vibrations and sensor errors that may exist in the mining environment. This processing significantly reduced minor fluctuations in the trajectory caused by the complex terrain of the mining area, resulting in a smoother curve that better reflected the actual driving path. Finally, the processed trajectory was merged with the mine map data, which included mining areas, transportation road networks, loading and unloading point locations, and slope information. By aligning the trajectory points with the centerline of the nearest transportation road and considering the relationship between the outpost locations and known key areas, a complete description of the mine transportation scenario was generated. For example, the standby points 200-230 were identified as waiting to load ore at a loading point; the system recorded the start and end times of loading, as well as the loading amount. The standby points 1000-1040 were identified as unloading ore in the unloading area; the system also recorded detailed unloading information. The standby points 1600-1640 were identified as waiting for the vehicle ahead on a 15-degree steep slope, reflecting traffic control measures in the mine's safety operating procedures. In the area with missing data, points 600-700, the interpolated trajectory shows the vehicle traversing a complex serpentine downhill section. By combining this with terrain data, the average gradient of this section was calculated to be 8%, and several important turning points were marked.

[0089] In one specific embodiment, as shown in Figure 2, a method for training a behavior prediction model is also provided, including:

[0090] S201. Obtain the training trajectory from the merged scene data. For each training trajectory, divide the training trajectory into the first historical trajectory and the second historical trajectory according to the time sequence.

[0091] S202. Input the first historical trajectory into the encoder of the encoder-decoder model for encoding to obtain the encoded vector;

[0092] S203. Input the encoded vector into the multi-level decoder of the encoder-decoder model, and predict the trajectory of multiple consecutive sub-time periods through the multi-level decoder; wherein, the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided.

[0093] S204. The predicted trajectories of multiple consecutive sub-time periods are spliced ​​together and compared with the second historical trajectory.

[0094] S205. Train the encoder-decoder model based on the comparison results to obtain a multi-agent interaction behavior prediction model.

[0095] Optionally, the training trajectories are obtained from the merged scene data. This trajectory data comes from high-quality scene data that has undergone stationary point detection, data interpolation, and trajectory smoothing. For each training trajectory, it is divided into a first historical trajectory and a second historical trajectory in chronological order. This division is usually based on a predefined time point; for example, the first 60% of trajectory points can be used as the first historical trajectory, and the last 40% as the second historical trajectory. This division aims to use the first historical trajectory to predict the second historical trajectory, thereby training the model's predictive ability. Next, the first historical trajectory is input into the encoder of the encoder-decoder model for encoding, resulting in an encoded vector. The encoder is typically a recurrent neural network (RNN) structure, such as a long short-term memory network (LSTM) or a gated recurrent unit (GRU). The encoder processes each time step of the first historical trajectory progressively, compressing the trajectory information into a fixed-length vector, which is the encoded vector, containing the key features and contextual information of the trajectory.

[0096] The resulting encoded vector is then input into the multi-level decoder of the encoder-decoder model. A key innovation of this method is the multi-level decoder, which divides the prediction process into multiple consecutive sub-time periods. These sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided. For example, if the second historical trajectory is 6 seconds long, it can be divided into three 2-second sub-time periods. Each level of the multi-level decoder is responsible for predicting the trajectory for one sub-time period. During the decoding process, the first-level decoder uses the encoded vector as input to predict the trajectory for the first sub-time period. Then, the second-level decoder uses the output of the first-level decoder as input to predict the trajectory for the second sub-time period, and so on. This multi-level decoding approach allows the model to progressively accumulate and update information when predicting long-term trajectories, contributing to improved accuracy in long-term predictions.

[0097] After prediction, the predicted trajectories from multiple consecutive sub-time periods are stitched together to obtain the complete predicted trajectory. This predicted trajectory is then compared with the actual second historical trajectory. The comparison typically uses metrics such as mean squared error (MSE) or mean absolute error (MAE) to measure the difference between the predicted and actual trajectories. Finally, the encoder-decoder model is trained based on the comparison results. This process usually uses the backpropagation algorithm to adjust the model parameters by minimizing the error between the predicted and actual trajectories. The training process is iterative, continuously optimizing the model until a predetermined stopping condition is met, such as the error falling below a certain threshold or reaching the maximum number of iterations.

[0098] For example, suppose we have trajectory data for an unmanned mining truck, totaling 100 seconds in length and containing 500 sampling points (sampling frequency of 5Hz). The first 60 seconds (300 points) are used as the first historical trajectory, and the next 40 seconds (200 points) as the second historical trajectory. The encoder processes the first historical trajectory of 300 points, generating a 256-dimensional encoded vector. This vector contains key features such as the vehicle's motion pattern, speed changes, and steering information within the first 60 seconds. A multi-level decoder divides the 40-second prediction task into four 10-second sub-time periods. The first-level decoder uses the 256-dimensional encoded vector as input to predict the trajectory for the first 10 seconds (50 points). The second-level decoder uses the output of the first level as input to predict the trajectory for the next 10 seconds, and so on.

[0099] Finally, the prediction results from the four sub-time periods are stitched together to form a complete 40-second predicted trajectory. This predicted trajectory is then compared with the actual second historical trajectory. Assuming the mean absolute error (MAE) is used as the evaluation metric, the calculated MAE is 0.5 meters. Based on this error, the model updates its parameters using a backpropagation algorithm. For example, if the model is found to have a large error in predicting turns, the relevant parameters for steering prediction will be further adjusted. This process is repeated, each time using different trajectory data, until the model performance no longer shows a significant improvement. The multi-agent interaction behavior prediction model trained in this way can not only accurately predict short-term trajectories but also improve the accuracy of long-term predictions through multi-level decoding. This is particularly important for autonomous driving systems in mining environments, as they need to make accurate long-term plans in complex terrain and multi-vehicle collaborative environments.

[0100] In one specific embodiment, the number of levels in the multi-level decoder is the same as the number of consecutive sub-time periods; the encoded vector is input into the multi-level decoder of the encoder-decoder model, and the trajectories of the multiple consecutive sub-time periods are predicted by the multi-level decoder, including:

[0101] (1) Input the encoded vector into the first-level decoder and output the predicted trajectory of the first sub-time period in multiple consecutive sub-time periods;

[0102] (2) For each decoder after the first level decoder, the predicted trajectory of the sub-time period corresponding to the previous level decoder is used as the input of the next level decoder, and the predicted trajectory of the sub-time period corresponding to the next level decoder is output; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

[0103] Optionally, the multi-level decoder is designed to improve the accuracy of long-term trajectory prediction. The number of levels in the multi-level decoder is the same as the number of consecutive sub-time periods into which the trajectory to be predicted is divided. This design ensures that each sub-time period is processed by a dedicated decoder. The workflow of the multi-level decoder begins by inputting the encoded vector into the first-level decoder. The encoded vector is generated by the encoder after processing the first historical trajectory and contains the trajectory's key features and contextual information. The first-level decoder receives this encoded vector and outputs the predicted trajectory for the first sub-time period out of multiple consecutive sub-time periods. This process can be understood as the model making predictions about the near future based on historical information.

[0104] Furthermore, a cascaded approach is used for prediction at each decoder level after the first-level decoder. Specifically, the predicted trajectory of the sub-time period corresponding to the previous level's decoder is used as the input to the next level's decoder. The advantage of this design is that each level of decoder not only utilizes the initial encoded information but also the prediction results of the previous time period, thus continuously updating and accumulating information during the prediction process. Each decoder outputs the predicted trajectory of its corresponding sub-time period, and this process continues until the last level of decoder completes its prediction.

[0105] To better understand this process, let's take a specific example of an autonomous driving scenario in a mine. Suppose we need to predict the trajectory of a mining truck over the next 60 seconds. We divide these 60 seconds into three 20-second sub-time periods, and correspondingly, the multi-level decoder also has three levels. First, the encoder processes historical trajectory data from the past 5 minutes, generating a 256-dimensional encoded vector. This vector contains information such as the vehicle's average speed, common steering patterns, and interactions with other vehicles.

[0106] The first-level decoder receives the 256-dimensional encoded vector as input. Through a series of neural network layers (such as LSTM and fully connected layers), it outputs the predicted trajectory for the first 20 seconds. Assuming a trajectory sampling frequency of 1Hz, this output is a 20x3 matrix, where 20 represents the time step number, and 3 represents the x, y coordinates and orientation angle at each time step. The second-level decoder receives not only the original 256-dimensional encoded vector but also the 20x3 matrix output from the first-level decoder as input. This means that when making predictions, the second-level decoder considers not only the original historical information but also the vehicle's possible motion state within the predicted next 20 seconds. The second-level decoder also outputs a 20x3 matrix, representing the predicted trajectory from seconds 21 to 40.

[0107] The third-level decoder repeats a similar process, receiving the original encoded vector, the prediction results from the first and second levels as input, and outputting the predicted trajectory for the last 20 seconds (41-60 seconds). This cascaded prediction method is particularly suitable for long-term trajectory prediction in mining environments. For example, if the first-level decoder predicts that the vehicle will begin climbing a slope within the first 20 seconds, the second-level decoder can use this information to predict the vehicle's possible deceleration behavior in the next 20 seconds. Similarly, if the first two levels of prediction show that the vehicle is approaching a loading point, the third-level decoder can predict that the vehicle may decelerate and eventually stop in the last 20 seconds.

[0108] In this way, multi-level decoders can progressively build a coherent long-term trajectory prediction. Each level adds new information and predictions based on the previous level, which not only improves the accuracy of long-term predictions but also makes the prediction results more consistent with actual vehicle movement patterns and mine operation logic. Finally, the three 20-second predicted trajectories are stitched together to form a complete 60-second predicted trajectory.

[0109] It should be noted that in this disclosure, the decoding results of different time periods are spliced ​​together. The prediction results of the ordinary decoder are shown in Figure 3, and the prediction results of the decoder of this method are shown in Figure 4.

[0110] In Figure 3, segment AB represents the historical trajectory, segment BC represents the true value of the predicted trajectory, and segment BD represents multiple predicted trajectories predicted by the ordinary decoder. As can be seen from Figure 3, the ordinary decoder did not correctly predict the agent's trajectory.

[0111] In Figure 4, segment EF represents the historical trajectory, segment FG represents the true value of the predicted trajectory, and segments FH and FG contain trajectories with arrows that are spliced ​​predicted trajectories from multiple decoders in this disclosure. As can be seen from Figure 4, this method obtains more reasonable prediction results through a step-by-step approach.

[0112] The above describes the method for constructing intelligent agent interaction scenarios and the method for training behavior prediction models in the embodiments of this disclosure. The following describes the apparatus for constructing intelligent agent interaction scenarios in the embodiments of this disclosure. Please refer to Figure 5. One embodiment of the apparatus for constructing intelligent agent interaction scenarios in the embodiments of this disclosure includes:

[0113] The acquisition module 301 is configured to acquire sensor data from multiple intelligent agents and map data of the geographical locations corresponding to the sensor data;

[0114] The detection module 302 is configured to perform trajectory detection on other detected intelligent agents represented by the sensor data of each intelligent agent to obtain a first trajectory; and determine the second trajectory of the intelligent agent based on the sensor data.

[0115] The filtering module 303 is configured to filter the first trajectory and / or the second trajectory according to the scene data admission conditions corresponding to the preset scene to obtain the target trajectory;

[0116] The merging module 304 is configured to merge map data and target trajectory to obtain merged scene data.

[0117] The detection module 302 is specifically configured to: perform semantic segmentation on the sensor data of each agent; perform target detection on the segmented data; and perform target tracking on the detected target agents; determine the target tracking data corresponding to the same target agent based on the target tracking results corresponding to the sensor data of multiple agents; and determine the first trajectory corresponding to each target agent based on the target tracking data corresponding to that target agent.

[0118] Filtering module 303 includes:

[0119] The determining unit is set to determine the trajectory feature indicators corresponding to the driving trajectory based on the scene data admission conditions corresponding to the preset scene.

[0120] The extraction unit is configured to extract data features for each trajectory feature index for each trajectory of the first trajectory and / or the second trajectory, thereby obtaining trajectory feature data for each feature index corresponding to that trajectory.

[0121] The filtering unit is set to filter trajectories based on the admission criteria and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, and obtain the target trajectory.

[0122] The filter unit is specifically set as follows:

[0123] For each trajectory feature indicator, determine the admission criteria and judgment conditions for that trajectory indicator; and compare the trajectory feature data corresponding to the trajectory feature indicator extracted for each trajectory with the corresponding judgment conditions.

[0124] The trajectory whose trajectory feature data all meet the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

[0125] The intelligent agent interaction scenario construction device also includes:

[0126] The determination module is set to determine the distribution characteristic indicators corresponding to the merged scene data based on the scene data distribution conditions corresponding to the preset scene.

[0127] The extraction module is configured to extract scene distribution data features from the merged scene data for each distribution feature index.

[0128] The enhancement module is configured to determine whether the currently merged scene data meets the distribution conditions based on the extracted scene distribution data characteristics; if it does not meet the conditions, the scene data is enhanced to obtain the target merged scene data.

[0129] The enhancement module is specifically configured to: for each distribution feature index, determine whether the extracted scene distribution data features can cover the scene distribution represented by the distribution feature index; if it cannot cover the scene distribution, identify the missing scene data and supplement the missing scene data.

[0130] The intelligent agent interaction scenario construction device also includes:

[0131] The stationary point module is configured to perform stationary point detection processing on the target trajectory to obtain stationary point detection trajectory data.

[0132] The interpolation module is configured to determine the data segment to be filled corresponding to the target trajectory based on the stationary point detection trajectory data, and perform trajectory interpolation processing on the data segment to be filled to obtain interpolated trajectory data.

[0133] The smoothing module is configured to perform trajectory smoothing on the interpolated trajectory data to obtain the enhanced target trajectory.

[0134] The fusion module is configured to merge map data and target trajectory to obtain merged scene data, including: merging map data and enhanced target trajectory to obtain merged scene data.

[0135] The behavior prediction model training device in this disclosure is described below. Referring to Figure 6, one embodiment of the behavior prediction model training device in this disclosure includes:

[0136] The acquisition module 401 is configured to acquire the trajectory to be trained from the merged scene data;

[0137] The segmentation module 402 is configured to divide each training trajectory into a first historical trajectory and a second historical trajectory in chronological order.

[0138] Encoding module 403 is configured to encode the first historical trajectory into the encoder of the encoder-decoder model to obtain an encoded vector;

[0139] The prediction module 404 is configured to input the encoded vector into the multi-level decoder of the encoder-decoder model, and predict the trajectory of multiple consecutive sub-time periods through the multi-level decoder; wherein, the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided.

[0140] The stitching module 405 is configured to stitch together the predicted trajectories of multiple consecutive sub-time periods and compare them with the second historical trajectory.

[0141] Training module 406 is configured to train the encoder-decoder model based on the comparison results to obtain a multi-agent interaction behavior prediction model.

[0142] The prediction module 404 is specifically configured to: input the encoded vector into the multi-level decoder of the encoder-decoder model, and predict the trajectory of multiple consecutive sub-time periods through the multi-level decoder, including: inputting the encoded vector into the first-level decoder and outputting the predicted trajectory of the first sub-time period among multiple consecutive sub-time periods; for each decoder after the first-level decoder, taking the predicted trajectory of the sub-time period corresponding to the previous level decoder as the input of the next level decoder, and outputting the predicted trajectory of the sub-time period corresponding to the next level decoder; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

[0143] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for constructing an intelligent agent interaction scenario, comprising the following steps: Acquire sensor data from multiple intelligent agents and map data showing the geographical locations corresponding to the sensor data; For each agent's sensor data, trajectory detection is performed on other agents detected by the sensor data of that agent to obtain the first trajectory; And determine the agent's second trajectory based on the sensor data; According to the scene data admission conditions corresponding to the preset scene, the first trajectory and / or the second trajectory are filtered to obtain the target trajectory, including: determining the trajectory feature indicators corresponding to the driving trajectory according to the scene data admission conditions corresponding to the preset scene; for each trajectory of the first trajectory and / or the second trajectory, data feature extraction is performed on each trajectory feature indicator to obtain the trajectory feature data corresponding to each feature indicator of the trajectory; based on the admission conditions and the trajectory feature data corresponding to each trajectory feature indicator extracted for each trajectory, the trajectory is filtered to obtain the target trajectory. The map data and target trajectory are merged to obtain merged scene data.

2. The method of claim 1, wherein, For each agent's sensor data, trajectory detection is performed on other agents detected by the sensor data representing that agent, resulting in a first trajectory, including: For each agent's sensor data, semantic segmentation is performed on the agent's sensor data; target detection is performed on the segmented data; and target tracking is performed on the detected target agents. Based on the target tracking results corresponding to the sensor data of multiple agents, determine the target tracking data corresponding to the same target agent; For each target agent, the first trajectory corresponding to the target agent is determined based on the target tracking data corresponding to that target agent.

3. The method of claim 1, wherein, The trajectory feature indicators include at least one of the following: data health feature indicators, trajectory-agent relationship feature indicators, and spatiotemporal feature indicators; The data health feature index is used to characterize at least one of the following: data integrity, duration, and the shape of the trajectory; the trajectory-agent relationship feature index is used to characterize the interaction behavior features of the trajectory-corresponding agent's movement with other agents and / or the positional relationship features with its own trajectory; the spatiotemporal feature index is used to characterize at least one of the following spatiotemporal position features, motion parameter features, and trajectory shape features of the trajectory-corresponding agent. Based on the admission criteria and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, trajectory filtering is performed to obtain the target trajectory, including: For each trajectory feature index, determine the judgment conditions of the admission conditions for that trajectory index; The trajectory feature data corresponding to the trajectory feature index extracted for each trajectory will be compared with the corresponding judgment conditions; The trajectory whose trajectory feature data all meet the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

4. The method of claim 1, wherein, Also includes: Based on the scenario data distribution conditions corresponding to the preset scenario, determine the distribution characteristic indicators corresponding to the merged scenario data; For each distribution characteristic index, scene distribution data features are extracted from the merged scene data; Based on the extracted scene distribution data characteristics, determine whether the currently merged scene data meets the distribution conditions; if not, perform enhancement processing on the scene data to obtain the target merged scene data.

5. The method of claim 4, wherein, The distribution characteristic indicators include at least one of the following: business scenario characteristic indicators, time distribution characteristic indicators, and spatial distribution characteristic indicators; Based on the extracted scene distribution data characteristics, determine whether the currently merged scene data meets the distribution conditions; In cases where the data does not meet the requirements, augmentation processing is performed on the scene data, including: For each distribution feature index, determine whether the extracted scene distribution data features can cover the scene distribution represented by that distribution feature index; If coverage is not possible, identify the missing scene data and supplement it.

6. The method of claim 1, wherein, Before merging the map data and the target trajectory, the process also includes: Stasis detection processing is performed on the target trajectory to obtain stagnant trajectory data; Based on the stationary point detection trajectory data, determine the data segment to be filled corresponding to the target trajectory, and perform trajectory interpolation processing on the data segment to be filled to obtain interpolated trajectory data; The interpolated trajectory data is smoothed to obtain the enhanced target trajectory; The map data and the target trajectory are merged to obtain merged scene data, including: merging the map data and the enhanced target trajectory to obtain merged scene data.

7. A method for training a behavior prediction model, comprising: The training trajectory is obtained from the merged scene data, wherein the merged scene data is obtained by the method of any one of claims 1-6; For each training trajectory, the training trajectory is divided into the first historical trajectory and the second historical trajectory according to the time sequence; The first historical trajectory is input into the encoder of the encoder-decoder model for encoding to obtain the encoded vector; The encoded vector is input into the multi-level decoder of the encoder-decoder model, and the multi-level decoder predicts the trajectory of multiple consecutive sub-time periods respectively; wherein, the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided. The predicted trajectories of multiple consecutive sub-time periods are spliced ​​together and compared with the second historical trajectory; The encoder-decoder model is trained based on the comparison results to obtain a multi-agent interaction behavior prediction model.

8. The method of claim 7, wherein, The number of levels in the multi-level decoder is the same as the number of the plurality of consecutive sub-time periods; The encoded vector is input into the multi-level decoder of the encoder-decoder model, and the multi-level decoder predicts the trajectory for multiple consecutive sub-time periods, including: The encoded vector is input into the first-level decoder, which outputs the predicted trajectory for the first sub-time period in the plurality of consecutive sub-time periods; For each decoder after the first-level decoder, the predicted trajectory of the sub-time period corresponding to the previous level decoder is used as the input of the next level decoder, and the predicted trajectory of the sub-time period corresponding to the next level decoder is output; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

9. A device for constructing an intelligent agent interaction scene, comprising: The acquisition module is configured to acquire sensor data from multiple intelligent agents and map data of the geographical locations corresponding to the sensor data; The detection module is configured to perform trajectory detection on other detected agents represented by the sensor data of each agent, and obtain a first trajectory. And determine the agent's second trajectory based on the sensor data; The filtering module is configured to filter the first trajectory and / or the second trajectory according to the scene data admission conditions corresponding to the preset scene to obtain the target trajectory, including: determining the trajectory feature indicators corresponding to the driving trajectory according to the scene data admission conditions corresponding to the preset scene; extracting data features for each trajectory feature indicator for each trajectory of the first trajectory and / or the second trajectory to obtain the trajectory feature data corresponding to each feature indicator of the trajectory; and filtering the trajectory based on the admission conditions and the trajectory feature data corresponding to each trajectory feature indicator extracted for each trajectory to obtain the target trajectory. The merging module is configured to merge the map data and the target trajectory to obtain merged scene data.

10. A behavior prediction model training device, comprising: The acquisition module is configured to acquire the trajectory to be trained from the merged scene data, wherein the merged scene data is obtained by the method of any one of claims 1-6; The segmentation module is set to divide each training trajectory into a first historical trajectory and a second historical trajectory according to the time sequence. The encoding module is configured to encode the first historical trajectory into the encoder of the encoder-decoder model to obtain an encoded vector; The prediction module is configured to input the encoded vector into the multi-level decoder of the encoder-decoder model, and predict the trajectory of multiple consecutive sub-time periods through the multi-level decoder; wherein, the multiple consecutive sub-time periods correspond to the multiple consecutive sub-time periods into which the second historical trajectory is divided. The stitching module is configured to stitch together the predicted trajectories of multiple consecutive sub-time periods and compare them with the second historical trajectory; The training module is configured to train the encoder-decoder model based on the comparison results to obtain a multi-agent interaction behavior prediction model.

11. A cloud platform comprising: The intelligent agent interaction scene construction device as described in claim 9 and / or the behavior prediction model training device as described in claim 10.

Images