Intelligent agent interaction scene construction method, behavior prediction model training method and device

Abstract

The present application relates to the fields of smart mines, unmanned driving technology, and map technology, and discloses a method for constructing an intelligent agent interaction scene, a method for training a behavior prediction model, and a device. The method includes: obtaining sensor data of multiple intelligent agents and map data of the geographical locations corresponding to the sensor data; for the sensor data of each intelligent agent, performing trajectory detection on other intelligent agents detected by the sensor data of the intelligent agent to obtain a first trajectory; and determining the second trajectory of the intelligent agent based on the sensor data; screening the first trajectory and / or the second trajectory based on the scene data access conditions corresponding to the preset scene to obtain a target trajectory; merging the map data and the target trajectory to obtain merged scene data. The present application can improve the efficiency and accuracy of constructing intelligent agent interaction scenes.

Description

Technical Field

[0001] The present application relates to the fields of smart mines, unmanned driving technology, and map technology, and in particular to a method for constructing an intelligent agent interaction scene, and a method and device for training a behavior prediction model. Background Art

[0002] In the field of trajectory prediction in interactive scenarios, previous technical solutions mainly rely on the motion model of independent agents or rule-based methods. These methods perform well in simple, non-interactive scenarios, but have significant limitations in complex interactive environments, such as urban traffic, mines, docks, etc. Traditional methods mainly include three categories: 1) Single-agent model: This type of method assumes that the movement of each agent is independent of other agents, and learns the prediction model through historical trajectory data. This assumption ignores the interaction between agents, resulting in a significant decrease in prediction accuracy when predicting scenarios with strong interactivity, such as vehicle encounters and pedestrian avoidance. 2) Rule-based method: Some solutions try to simulate the interaction between agents by formulating rules, such as methods based on social force models. Although this method takes into account the mutual influence between agents to a certain extent, the setting of rules is highly dependent on manual work, it is difficult to cover all possible interaction scenarios, and it has poor adaptability to new scenarios. 3) Multi-agent model: All agents in the scene are predicted at the same time, and the prediction model is learned through historical trajectory data.

[0003] However, these existing technical solutions have obvious shortcomings. First, the single-agent model ignores the interaction between agents, resulting in poor prediction accuracy in complex scenarios. Second, although the rule-based method takes the interaction into account, it is difficult to cover all possible interaction situations and lacks flexibility. Finally, although the multi-agent model takes the overall scene into consideration, the interaction patterns are diverse and complex, resulting in many local optimal solutions. The distinction between these solutions is low, and the output results are easily affected by any inaccurate prediction, thereby affecting the accuracy of the entire scene prediction. In addition, these methods generally have problems such as strong data dependence and poor adaptability to new scenarios, and it is difficult to meet the high accuracy and robustness requirements for trajectory prediction in practical applications. Summary of the invention

[0004] The present application provides a method, device and system for constructing an intelligent agent interaction scene, which are used to improve the efficiency and accuracy of the method for constructing an intelligent agent interaction scene.

[0005] In a first aspect, the present application provides a method for constructing an intelligent agent interaction scene, the method comprising: obtaining sensor data of multiple intelligent agents and map data of geographical locations corresponding to the sensor data; for the sensor data of each intelligent agent, performing trajectory detection on other detected intelligent agents represented by the sensor data of the intelligent agent to obtain a first trajectory; and determining a second trajectory of the intelligent agent based on the sensor data; screening the first trajectory and / or the second trajectory based on scene data access conditions corresponding to a preset scene to obtain a target trajectory; and merging the map data and the target trajectory to obtain merged scene data.

[0006] In the second aspect, the present application provides a behavior prediction model training method, which includes: obtaining a trajectory to be trained from the merged scene data; for each trajectory to be trained, dividing the trajectory to be trained into a first historical trajectory and a second historical trajectory in chronological order; inputting the first historical trajectory into the encoder of the encoder-decoder model for encoding to obtain a coding vector; inputting the coding vector into the multi-level decoder of the encoder-decoder model, and predicting the trajectories of multiple continuous sub-time periods respectively through the multi-level decoder; wherein the multiple continuous sub-time periods correspond to the multiple continuous sub-time periods divided by the second historical trajectory; splicing the predicted trajectories of the multiple continuous sub-time periods and comparing them with the second historical trajectory; training the encoder-decoder model according to the comparison results to obtain a multi-agent interactive behavior prediction model.

[0007] In a third aspect, the present application provides an intelligent agent interaction scene construction device, the intelligent agent interaction scene construction device comprising:

[0008] An acquisition module, used to acquire sensor data of multiple agents and map data of geographical locations corresponding to the sensor data;

[0009] A detection module is used to perform trajectory detection on other detected agents represented by the sensor data of each agent to obtain a first trajectory; and determine a second trajectory of the agent based on the sensor data;

[0010] A screening module, used for screening the first trajectory and / or the second trajectory according to the scene data access condition corresponding to the preset scene to obtain a target trajectory;

[0011] The merging module is used to merge the map data and the target trajectory to obtain merged scene data.

[0012] In a fourth aspect, the present application provides a behavior prediction model training device, comprising:

[0013] The acquisition module is used to obtain the trajectory to be trained from the merged scene data;

[0014] A division module, for dividing each to-be-trained trajectory into a first historical trajectory and a second historical trajectory in chronological order;

[0015] an encoding module, configured to input the first historical trajectory into an encoder of an encoder-decoder model for encoding to obtain an encoding vector;

[0016] a prediction module, configured to input the encoding vector into a multi-stage decoder of the encoder-decoder model, and predict the trajectories of a plurality of continuous sub-time periods respectively through the multi-stage decoder; wherein the plurality of continuous sub-time periods correspond to the plurality of continuous sub-time periods into which the second historical trajectory is divided;

[0017] A splicing module, used for splicing the predicted trajectories of multiple consecutive sub-time periods and comparing them with the second historical trajectory;

[0018] The training module is used to train the encoder-decoder model according to the comparison results to obtain a multi-agent interaction behavior prediction model.

[0019] In a fifth aspect, the present application provides a cloud platform, including: an intelligent agent interaction scene construction device and a behavior prediction model training device.

[0020] In the technical solution provided by the present application, for the sensor data of each intelligent agent, trajectory detection is performed on other intelligent agents detected by the sensor data of the intelligent agent to obtain a first trajectory, and the second trajectory of the intelligent agent is determined based on the sensor data. This dual-track processing method effectively improves the comprehensiveness and accuracy of the trajectory data. It can better capture the interactive behaviors and relationships between intelligent agents. Secondly, according to the scene data access conditions corresponding to the preset scene, the first trajectory and / or the second trajectory are screened to obtain the target trajectory, which effectively reduces the impact of data noise and outliers on the final scene construction. Finally, the map data and the target trajectory are merged to obtain the merged scene data, which significantly improves the quality and reliability of the constructed scene. It can not only accurately capture the motion characteristics of individual intelligent agents, but also reflect the complex interactive relationships between intelligent agents. At the same time, combined with the map data, the constructed scene is closer to the actual environment, which enhances the applicability and accuracy of the model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0022] Figure 1 A schematic diagram of an embodiment of a method for constructing an intelligent agent interaction scene in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of an embodiment of a behavior prediction model training method in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the prediction result of a common decoder in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of the prediction result of the decoder of the method in the embodiment of the present application;

[0026] Figure 5 A schematic diagram of an embodiment of a device for constructing an intelligent agent interaction scene in an embodiment of the present application;

[0027] Figure 6 Schematic diagram of an embodiment of a behavior prediction model training device in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The embodiments of the present application provide a method, device and system for constructing an intelligent agent interaction scene. The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described here can be implemented in an order other than that illustrated or described here. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] For ease of understanding, the specific process of the embodiment of the present application is described below. Figure 1 , an embodiment of the method for constructing an intelligent agent interaction scene in the embodiment of the present application includes:

[0030] Step S101, obtaining sensor data of multiple agents and map data of geographical locations corresponding to the sensor data;

[0031] Step S102: for each agent's sensor data, perform trajectory detection on other agents detected by the agent's sensor data to obtain a first trajectory; and determine a second trajectory of the agent based on the sensor data;

[0032] Step S103: screening the first trajectory and / or the second trajectory according to the scene data access condition corresponding to the preset scene to obtain the target trajectory;

[0033] Step S104: merge the map data and the target trajectory to obtain merged scene data.

[0034] It is understandable that the execution subject of the present application can be an intelligent agent interaction scene construction device, or a terminal, an unmanned vehicle or a server, which is not limited here. The present application embodiment takes the server as the execution subject as an example for explanation.

[0035] Optionally, the sensor data of multiple agents and the map data of the corresponding geographic locations of the sensor data are obtained through the server. These data come from devices such as vehicle-mounted cameras, laser radars, and GPS, and include the location and speed information of the agent itself and the perception data of the surrounding environment. For example, an autonomous vehicle may collect its own GPS coordinates, driving speed, and the relative position and motion information of surrounding vehicles and pedestrians. For the sensor data of each agent, the server performs trajectory detection on other agents detected by the agent to obtain a first trajectory. This process involves processing the original data, including semantic segmentation, target detection, and target tracking. Semantic segmentation divides the image data into different semantic areas, such as roads, vehicles, pedestrians, etc. Target detection identifies specific target objects, such as other vehicles or pedestrians. Target tracking tracks the motion trajectories of these target objects through continuous frame data. At the same time, the second trajectory of the agent is determined based on the sensor data of the agent itself. The combination of these two steps takes into account both the movement of the agent itself and the movement of other surrounding agents, laying the foundation for subsequent interaction prediction.

[0036] Then, the server screens the first trajectory and / or the second trajectory according to the scene data access conditions corresponding to the preset scene to obtain the target trajectory. The access conditions are set according to the constraints of the business on data features, including data health features, trajectory and agent relationship features, and spatiotemporal features. Data health features include data integrity, duration, trajectory shape, etc.; trajectory and agent relationship features include whether they are adjacent, whether there is interaction, and specific interactive behaviors such as following, overtaking, crossing, etc.; spatiotemporal features include average driving speed, acceleration, trajectory curvature, etc. For example, the access conditions may require that the data is complete, the duration is greater than 9 seconds, the trajectory curvature meets the requirements of the curve, and the overtaking trajectory with a speed greater than 30km / h. These conditions ensure that the data entering the subsequent processing has sufficient quality and representativeness. Finally, the server merges the map data and the target trajectory to obtain the merged scene data. This step combines the filtered trajectory data with the geographic environment information to form a complete scene description. The merged scene data not only contains the movement trajectory of the agent, but also contains environmental information such as road structure and traffic signs, providing rich context for subsequent interaction prediction.

[0037] For example, suppose that in an intersection scenario, the server receives sensor data from multiple vehicles. One vehicle, A, is going straight through the intersection, and another vehicle, B, is preparing to turn left. The sensor data of vehicle A shows that it is going straight at a speed of 50km / h, and it detects that vehicle B is slowing down and turning on the left turn signal. The sensor data of vehicle B shows that its speed has dropped from 40km / h to 20km / h, and the steering wheel angle has gradually turned to the left. The server processes these raw data to obtain the precise trajectories of the two vehicles. Then, according to the access conditions (such as data integrity > 95%, duration > 10 seconds, involving intersection turning operations, etc.), these trajectories are judged as valid data. Finally, these trajectory data are merged with the map data of the intersection (including road width, traffic light location, etc.) to form a complete interactive scene description.

[0038] By executing the above steps, for the sensor data of each agent, the trajectory detection of other agents detected by the sensor data of the agent is performed to obtain the first trajectory, and the second trajectory of the agent is determined according to the sensor data. This dual-track processing method effectively improves the comprehensiveness and accuracy of the trajectory data. It can better capture the interactive behaviors and relationships between agents. Secondly, according to the scene data access conditions corresponding to the preset scene, the first trajectory and / or the second trajectory are screened to obtain the target trajectory, which effectively reduces the impact of data noise and outliers on the final scene construction. Finally, the map data and the target trajectory are merged to obtain the merged scene data, which significantly improves the quality and reliability of the constructed scene. It can not only accurately capture the motion characteristics of individual agents, but also reflect the complex interactive relationships between agents. At the same time, combined with the map data, the constructed scene is closer to the actual environment, which enhances the applicability and accuracy of the model in practical applications.

[0039] In a specific embodiment, performing trajectory detection on other agents detected by the sensor data of the agent to obtain a first trajectory specifically includes the following steps:

[0040] (1) For each agent’s sensor data, perform semantic segmentation on the agent’s sensor data; perform target detection on the segmented data; and perform target tracking on the detected target agent;

[0041] (2) Determine the target tracking data corresponding to the same target agent based on the target tracking results corresponding to the sensor data of multiple agents;

[0042] (3) For each target intelligent agent, determine a first trajectory corresponding to the target intelligent agent according to the target tracking data corresponding to the target intelligent agent.

[0043] Optionally, in the agent interaction scene construction method, the agent specifically refers to an unmanned vehicle, and a semantic segmentation operation is performed on the sensor data of each unmanned vehicle. Semantic segmentation technology classifies each pixel in the original image into a specific category, such as roads, other vehicles, traffic signs, etc. In the unmanned driving scene, a deep learning model specially trained for the road environment, such as DeepLab or PSPNet, is used to convert the original image into a semantic labeling map, where each color represents a specific element in the road scene. Then, target detection is performed on the segmented data. In the unmanned driving scene, target detection algorithms such as YOLOv5 or Faster R-CNN are optimized for identifying and locating other vehicles. These algorithms output a list containing vehicle types and precise bounding box coordinates. Target detection pays special attention to other unmanned vehicles for subsequent interaction predictions.

[0044] Subsequently, the detected target unmanned vehicle is tracked. In unmanned driving systems, commonly used target tracking algorithms include improved Kalman filters or SORT (Simple Online and Realtime Tracking) algorithms. These algorithms are specifically optimized for vehicle motion characteristics and can maintain stable tracking in high-speed motion and partial occlusion. Target tracking assigns a unique ID to each detected unmanned vehicle and predicts its position, velocity, and acceleration in consecutive frames.

[0045] After obtaining the target tracking results corresponding to the sensor data of multiple unmanned vehicles, the target tracking data corresponding to the same target unmanned vehicle is determined. Data association uses the improved Hungarian algorithm or the joint probability data association (JPDA) method to associate the observation results of different sensors (such as cameras, lidars, and millimeter-wave radars) to the same target unmanned vehicle. Data fusion uses methods such as extended Kalman filtering or particle filtering to comprehensively consider the information of multiple sensors to obtain a more accurate state estimate of the target unmanned vehicle. Finally, for each target unmanned vehicle, the first trajectory corresponding to the target unmanned vehicle is determined according to its corresponding target tracking data. In unmanned driving scenarios, trajectory generation usually uses vehicle kinematic models combined with curve fitting technology. For example, the constant acceleration model (CA) or constant steering angular rate and acceleration model (CTRV) commonly used in adaptive cruise control (ACC) systems is used, combined with Bessel curve fitting, to convert discrete tracking points into continuous trajectory curves that conform to the vehicle's motion characteristics. This method can not only fill in the gaps in the observation data, but also reduce the impact of measurement noise and generate smoother trajectories that conform to the actual vehicle motion laws.

[0046] For example: an unmanned vehicle (unmanned vehicle A) is collecting data. Its lidar and camera systems generate 10 frames of point cloud and image data per second. First, the semantic segmentation algorithm processes this data to distinguish between the road, other vehicles, guardrails, etc. In one frame, the algorithm identifies 3 other unmanned vehicles. The object detection algorithm accurately locates these unmanned vehicles. For example, for one of the unmanned vehicles (target B), the algorithm outputs its bounding box coordinates as (x=50m, y=3.5m, length=4.5m, width=1.8m) with a confidence level of 0.98. The object tracking algorithm then tracks target B in consecutive frames and records its position changes. Assume that in the next 1 second (10 frames), the coordinates of the center point of target B are: (50m,3.5m), (52m,3.5m), ..., (70m,3.5m). The data association step associates these observations from the lidar and camera to the same target B. The data fusion step combines the data from the two sensors to obtain a more accurate position and velocity estimate. Finally, the vehicle kinematic model and curve fitting technology are used to generate the continuous trajectory of target B. For example, if the CTRV model is used, a smooth trajectory function can be obtained, which not only accurately expresses the observed position, but also reasonably estimates the unobserved position in the middle, while ensuring that the trajectory conforms to the vehicle's motion characteristics. It can accurately capture the motion trajectory of each unmanned vehicle and provide rich interactive information.

[0047] In a specific embodiment, the process of performing the step of screening the first track and / or the second track according to the scene data access condition corresponding to the preset scene may specifically include the following steps:

[0048] (1) Determine the trajectory characteristic index corresponding to the driving trajectory according to the scene data access conditions corresponding to the preset scene;

[0049] (2) for each trajectory of the first trajectory and / or the second trajectory, extracting data features for each trajectory feature index, and obtaining trajectory feature data corresponding to each feature index of the trajectory;

[0050] (3) Based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, trajectory screening is performed to obtain the target trajectory.

[0051] Optionally, according to the scene data access conditions corresponding to the preset scene, the trajectory characteristic indicators corresponding to the driving trajectory are determined. These characteristic indicators include data health characteristic indicators, trajectory and agent relationship characteristic indicators, and spatiotemporal characteristic indicators. The data health characteristic indicators are used to characterize the integrity, duration and shape of the trajectory data. The trajectory and agent relationship characteristic indicators are used to characterize the interactive behavior characteristics of the trajectory corresponding to the agent movement with other agents and the position relationship characteristics with its own trajectory. The spatiotemporal characteristic indicators are used to characterize the spatiotemporal position characteristics, motion parameter characteristics and trajectory shape characteristics of the agent corresponding to the trajectory.

[0052] For each trajectory of the first trajectory and / or the second trajectory, data feature extraction is performed for each trajectory feature indicator to obtain trajectory feature data corresponding to each feature indicator of the trajectory. This process involves a variety of data processing technologies. For data health features, integrity is evaluated by calculating the density and continuity of data points, duration is calculated using timestamp differences, and trajectory shape is evaluated by curvature analysis. The extraction of trajectory and agent relationship features involves calculating the relative distance, relative speed and acceleration between agents, and identifying specific interaction modes such as following, overtaking or avoidance behaviors. The extraction of spatiotemporal features includes calculating motion parameters such as average speed, acceleration, steering angular rate, and spatial distribution characteristics of the trajectory.

[0053] Finally, based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, the trajectory is screened to obtain the target trajectory. This involves setting the threshold or range for each feature index and evaluating the feature data of each trajectory. Only trajectories that meet all feature index requirements will be selected as target trajectories.

[0054] For example, in a highway scenario, the preset scenario data access conditions require that trajectory data meet the following conditions: data integrity greater than 95%, duration greater than 10 seconds, average speed between 60km / h and 120km / h, and at least one lane change behavior. Based on these conditions, the trajectory feature indicators that need to be extracted include: data point density, trajectory duration, average speed, and lateral displacement.

[0055] For a specific trajectory, the data processing process is as follows: First, calculate the data point density. Assuming that the total length of the trajectory is 1000 meters and there are 980 valid data points, the data integrity is 98%, which meets the requirements. Secondly, calculate the difference between the start and end timestamps of the trajectory, and the duration is 15 seconds, which also meets the conditions. Then, by calculating the total driving distance divided by the time, the average speed is 90km / h, which is within the required range. Finally, analyze the lateral displacement data and find that there is a lateral movement of more than 3.5 meters in the middle of the trajectory, which is determined to be a lane change behavior.

[0056] Optionally, the trajectory characteristic index includes at least one of the following: a data health characteristic index, a trajectory and agent relationship characteristic index, and a spatiotemporal characteristic index;

[0057] Data health characteristic indicators, used to characterize at least one of the following trajectory data: data integrity, duration, and the shape of the trajectory being characterized; trajectory-agent relationship characteristic indicators, used to characterize the interactive behavior characteristics of the trajectory-corresponding agent movement with other agents and / or the position relationship characteristics with its own trajectory; spatiotemporal characteristic indicators, used to characterize at least one of the following spatiotemporal position characteristics, motion parameter characteristics, and trajectory shape characteristics of the trajectory-corresponding agent;

[0058] In a specific embodiment, the process of performing trajectory screening based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory to obtain the target trajectory step may specifically include the following steps:

[0059] (1) For each trajectory characteristic indicator, determine the judgment condition of the access condition for the trajectory indicator; and compare the trajectory characteristic data corresponding to the trajectory characteristic indicator extracted for each trajectory with the corresponding judgment condition;

[0060] (2) The trajectory whose trajectory feature data satisfies the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

[0061] Optionally, the trajectory feature indicators include data health feature indicators, trajectory and agent relationship feature indicators, and spatiotemporal feature indicators. Data health feature indicators are used to evaluate the quality of trajectory data, including data integrity, duration, and the shape of the trajectory represented. Data integrity reflects the continuity and density of trajectory data points, duration represents the time span of the trajectory, and trajectory shape describes the geometric characteristics of the trajectory. Trajectory and agent relationship feature indicators focus on the interactive behaviors between agents, such as following, overtaking, and avoiding, as well as the positional relationship between the agent and its own trajectory, such as lane keeping or lane changing behaviors. Spatiotemporal feature indicators comprehensively describe the motion characteristics of the agent, including spatiotemporal position characteristics (such as coordinate changes), motion parameter characteristics (such as speed, acceleration), and trajectory shape characteristics (such as curvature, steering angle).

[0062] In the specific implementation process, first, for each trajectory characteristic indicator, determine the specific judgment conditions for the access conditions for the trajectory indicator. For example, for data integrity, 90% may be set as the threshold; for duration, at least 10 seconds may be required; for average speed, it may be stipulated to be between 40km / h and 120km / h. The setting of these judgment conditions is based on specific application scenarios and research needs, and requires expert knowledge and data analysis support.

[0063] Then, the trajectory feature data corresponding to the trajectory feature indicators extracted for each trajectory are compared with these judgment conditions one by one. For example, the data point density of the trajectory is calculated and compared with the integrity threshold, the time span of the trajectory is calculated and compared with the minimum duration requirement, and the speed distribution of the trajectory is analyzed and checked whether it is within the specified range. For more complex features, such as interactive behavior, pattern recognition algorithms may be needed to determine whether there is a specific interaction pattern.

[0064] Finally, only those trajectories whose trajectory feature data satisfy the judgment conditions corresponding to each trajectory feature index will be determined as the target trajectory.

[0065] For example, suppose that in an unmanned transportation scenario in a large open-pit mine, it is necessary to screen out high-quality trajectories that can represent complex interactive behaviors. The set entry conditions include: data integrity greater than 95%, duration greater than 5 minutes, average speed between 15km / h and 40km / h, at least one turn or slope change, and at least one interaction with other vehicles or fixed facilities (such as avoiding other vehicles or passing through loading points). For a specific unmanned mining truck trajectory, the data processing process is as follows: First, calculate the data point density. Assuming that the total length of the trajectory is 3000 meters and the sampling frequency is 1Hz, there should be 3000 data points in theory, and 2940 valid points are actually recorded, then the data integrity is 98%, which meets the requirement of greater than 95%. Secondly, calculate the difference between the start and end timestamps of the trajectory, and the duration is 6 minutes and 12 seconds, which meets the condition of greater than 5 minutes. Then, by calculating the total travel distance divided by the time, the average speed is 29km / h, which is within the required range of 15km / h to 40km / h. Next, by analyzing the three-dimensional displacement data, it was found that there was a turn of more than 30 degrees in the middle of the trajectory, which lasted for about 20 seconds, and a 7% uphill section, which was determined to meet the requirements of turning and slope changes. Finally, by analyzing the interaction with the surrounding environment, a behavior of stopping and loading ore at the loading point was identified, in which the vehicle speed dropped from 25km / h to 0, stayed for about 3 minutes and then restarted. The whole process clearly recorded the interaction with fixed facilities. Through the above analysis, the trajectory met all the set access conditions and was therefore selected as the target trajectory. This rigorous data screening process is particularly suitable for mining environments, because mining scenes have complex terrain, interaction with large equipment, and strict safety requirements. By screening out high-quality trajectory data, it can help optimize the performance of unmanned driving systems in key areas such as steep slopes, narrow turns, and loading and unloading points. This not only improves the efficiency and accuracy of subsequent research, but also provides more reliable data support for the decision-making and control of mine unmanned vehicles in complex environments, ultimately improving the safety and efficiency of mining operations.

[0066] In a specific embodiment, the method for constructing an intelligent agent interaction scene may further specifically include the following steps:

[0067] (1) Determine the distribution characteristic index corresponding to the merged scene data according to the scene data distribution conditions corresponding to the preset scene;

[0068] (2) Extract scene distribution data features from the merged scene data for each distribution feature index;

[0069] (3) Determine whether the currently merged scene data meets the distribution conditions based on the extracted scene distribution data features; if not, enhance the scene data to obtain the target merged scene data.

[0070] Optionally, the distribution characteristic index includes at least one of the following: a business scenario characteristic index, a time distribution characteristic index, and a space distribution characteristic index;

[0071] In a specific embodiment, according to the extracted scene distribution data features, determining whether the currently merged scene data meets the distribution condition; if not, performing enhancement processing on the scene data includes the following steps:

[0072] (1) For each distribution feature indicator, determine whether the extracted scene distribution data features can cover the scene distribution represented by the distribution feature indicator;

[0073] (2) If coverage is not possible, determine the missing scene data and supplement the missing scene data.

[0074] Optionally, according to the scene data distribution conditions corresponding to the preset scene, the distribution characteristic indicators corresponding to the merged scene data are determined. These distribution characteristic indicators include business scene characteristic indicators, time distribution characteristic indicators and space distribution characteristic indicators. Business scene characteristic indicators reflect the distribution of different types of interaction scenes, such as overtaking, following, meeting, etc.; time distribution characteristic indicators represent the distribution of data in different time periods, such as morning and evening peaks, daytime, nighttime, etc.; space distribution characteristic indicators describe the distribution of data in different geographical locations or road types, such as highways, urban roads, intersections, etc.

[0075] Then, for each distribution characteristic index, the scene distribution data characteristics of the merged scene data are extracted. For business scene characteristics, different types of interaction scenes are identified and counted by analyzing the relative motion patterns between intelligent bodies. For example, the changes in the relative distance, relative speed and acceleration between vehicles are calculated to determine whether overtaking, following or meeting has occurred. For time distribution characteristics, classified statistics are performed based on the timestamp information of the data, and the proportion of data in different time periods is calculated. For spatial distribution characteristics, geographic information system (GIS) technology is used to perform spatial clustering and distribution analysis on the data to determine the distribution of data on different types of roads.

[0076] Then, according to the extracted scene distribution data features, determine whether the currently merged scene data meets the preset distribution conditions. For each distribution feature index, determine whether the extracted scene distribution data features can cover the scene distribution represented by the distribution feature index. If it is found that some types of scene data are insufficient or excessive, data enhancement processing is required. Specifically, if some scene distributions cannot be covered, it is necessary to determine the lack of scene data and supplement the lack of scene data. The methods of data enhancement processing include but are not limited to: oversampling or data synthesis for scarce scenes, undersampling for excessive scenes, or increasing data diversity through data transformation technology. For example, for the lack of night data, adjust the sensor parameters to simulate night driving conditions, or use time mapping technology to convert part of the daytime data into night data. For insufficient specific interaction scenarios, such as lane change behavior, new lane change scene data is generated by making slight changes to the existing lane change data (such as adjusting the start and end positions of the lane change, changing the duration of the lane change, etc.).

[0077] For example, in an urban driverless driving scenario, the preset scenario data distribution conditions require that the data ratio of different interaction types (such as straight driving, turning, and lane changing) should be balanced, the time distribution should cover 24 hours a day, and the spatial distribution should include main roads, secondary roads, and residential streets. After extracting the distribution features, it was found that in the original data, straight driving scenarios accounted for 70%, turning scenarios accounted for 20%, and lane changing scenarios accounted for only 10%; in terms of time distribution, the data at night (22:00-6:00) accounted for only 5%; in terms of spatial distribution, the data on residential streets accounted for only 10%.

[0078] In view of this unbalanced distribution, the specific steps of data enhancement processing are as follows: First, data synthesis and oversampling are performed on the lane change scene. By making slight changes to the existing lane change data (such as adjusting the start and end positions of the lane change, changing the duration of the lane change, etc.), new lane change scene data is generated, and the proportion of lane change scenes is increased to about 25%. Secondly, by adjusting the sensor parameters to simulate night driving conditions, or using time mapping technology to convert part of the daytime data into nighttime data, the proportion of nighttime data is increased to 20%. Finally, through data conversion technology, the data features of some main roads and secondary roads (such as road width, lane line type, etc.) are adjusted to data that conforms to the characteristics of residential streets, and the proportion of residential street data is increased to 25%. Through this data enhancement processing, the target merged scene data finally obtained meets the preset balance requirements in terms of interaction type, time distribution and spatial distribution. This method not only ensures the comprehensiveness and representativeness of the data set, but also expands the originally scarce scene data through data enhancement technology, providing richer and more balanced training samples for the training of multi-agent interaction prediction models.

[0079] In a specific embodiment, before merging the map data and the target track, the following steps may be specifically included:

[0080] (1) Perform stationary point detection on the target trajectory to obtain stationary point detection trajectory data;

[0081] (2) Determine the data segment to be filled corresponding to the target trajectory according to the stationary point detection trajectory data, and perform trajectory interpolation processing on the data segment to be filled to obtain interpolated trajectory data;

[0082] (3) Perform trajectory smoothing on the interpolated trajectory data to obtain the enhanced target trajectory;

[0083] (4) Merging the map data and the target trajectory to obtain merged scene data, including: merging the map data and the enhanced target trajectory to obtain merged scene data.

[0084] Optionally, a stationary point detection process is used to identify locations in the trajectory where the vehicle stops or almost stops moving. Stationary point detection refers to the identification of time periods in the trajectory data where an agent (such as an unmanned vehicle) stops at a certain location or moves very slowly. These stationary points represent waiting at traffic lights, traffic congestion, loading and unloading of goods, or other situations that require the vehicle to stop. The specific implementation of stationary point detection is usually based on speed thresholds and time thresholds. For example, it can be defined that if the average speed of the vehicle within a certain period of time (such as 10 seconds) is lower than a certain threshold (such as 1km / h), then the location points within this period of time are considered to constitute a stationary point. The stationary point detection algorithm traverses the entire trajectory, calculates the average speed in each time window, and marks the points that meet the conditions as stationary points. The result of this process is the stationary point detection trajectory data, which contains the original trajectory points and a mark for each point whether it is a stationary point.

[0085] Then, the data segments to be filled corresponding to the target trajectory are determined according to the stationary point detection trajectory data. The data segments to be filled usually appear between two stationary points, or in the data missing area caused by signal loss during the data acquisition process. Trajectory interpolation processing is performed on these data segments to obtain interpolated trajectory data. The purpose of trajectory interpolation is to supplement the missing trajectory points to make the trajectory more continuous and smooth. Common interpolation methods include linear interpolation, spline interpolation or Bezier curve interpolation. For example, using cubic spline interpolation can generate a curve that is more in line with the actual motion characteristics of the vehicle while maintaining the continuity of the trajectory. After the interpolation is completed, the interpolated trajectory data is subjected to trajectory smoothing processing to obtain the enhanced target trajectory. The purpose of smoothing is to reduce the noise and irregular fluctuations in the trajectory so that the trajectory is more in line with the actual vehicle motion characteristics. Common smoothing methods include moving average, Gaussian filtering or Kalman filtering. These methods can effectively remove the trajectory jitter caused by GPS or other sensor errors while retaining the main features of the trajectory.

[0086] Finally, the enhanced target trajectory after smoothing is merged with the map data to obtain the final merged scene data. This merging process not only includes mapping the trajectory points to the corresponding positions on the map, but also includes associating the trajectory with information such as roads and traffic signs on the map. The merged scene data obtained in this way contains both high-quality vehicle motion trajectories and rich environmental context information.

[0087] For example: suppose there is an original trajectory data that records the driving of an unmanned mining truck in an open-pit mine. The total length of the trajectory is 10 kilometers, including 2,000 sampling points with a sampling interval of 2 seconds. First, stationary point detection is performed. Considering the characteristics of mining vehicles, the speed threshold is set to 0.5 km / h and the time threshold is set to 30 seconds. The selection of these parameters takes into account that mining vehicles may need to be in a low-speed or stopped state for a long time at loading and unloading points and steep slope sections. By calculating the average speed of every 15 consecutive points (corresponding to 30 seconds), it is found that the average speed at points 200-230, 1000-1040, and 1600-1640 is less than 0.5 km / h, so these points are marked as stationary points. These stationary points may correspond to loading points, unloading points, and steep slope waiting areas, respectively.

[0088] Then, the trajectory segment between the stationary points was analyzed, and it was found that there was data missing between the 600th and 700th points, which may be caused by signal occlusion in the mining area. Cubic spline interpolation was performed on this data to generate 100 new trajectory points to make the trajectory continuous in this area. The characteristics of the mine terrain were taken into account during interpolation to ensure that the generated trajectory points were consistent with the actual path direction and slope changes. Then, the entire trajectory was smoothed using a Gaussian filter with a window size of 7 points and a standard deviation of 2.0. The selection of these parameters took into account the large vibrations and sensor errors that may exist in the mining environment. After this treatment, the small jitter in the trajectory caused by the complex terrain of the mining area was significantly reduced, and the curve became smoother, better reflecting the actual driving path. Finally, the processed trajectory was merged with the mine map data. The mine map data contains mining areas, transportation road networks, loading and unloading point locations, slope information, etc. By aligning the trajectory points with the centerline of the nearest transportation road and considering the relationship between the stationary point locations and known key areas, a complete mine transportation scene description was generated. For example, the stationary point at points 200-230 was identified as waiting to load ore at a loading point, and the system recorded the start and end time of loading, as well as the loading volume information. The stationary point at points 1000-1040 was identified as unloading ore in the unloading area, and the system also recorded detailed information about the unloading. The stationary point at points 1600-1640 was identified as waiting for the vehicle in front to pass on a 15-degree steep slope, which reflects the traffic control measures in the mine safety operating procedures. In the area of ​​points 600-700 where data is missing, the interpolated trajectory shows that the vehicle has passed through a complex serpentine downhill section. By combining with the terrain data, the average slope of this section is calculated to be 8%, and several important turning points are marked.

[0089] In a specific embodiment, if Figure 2 As shown, a behavior prediction model training method is also provided, including:

[0090] S201, obtaining a to-be-trained trajectory from the merged scene data, and for each to-be-trained trajectory, dividing the to-be-trained trajectory into a first historical trajectory and a second historical trajectory in chronological order;

[0091] S202, inputting the first historical trajectory into the encoder of the encoder-decoder model for encoding to obtain an encoding vector;

[0092] S203, inputting the encoding vector into a multi-stage decoder of an encoder-decoder model, and predicting the trajectories of a plurality of continuous sub-time periods respectively through the multi-stage decoder; wherein the plurality of continuous sub-time periods correspond to the plurality of continuous sub-time periods into which the second historical trajectory is divided;

[0093] S204, splicing the predicted trajectories of multiple consecutive sub-time periods and comparing them with the second historical trajectory;

[0094] S205. Train the encoder-decoder model according to the comparison results to obtain a multi-agent interaction behavior prediction model.

[0095] Optionally, the trajectory to be trained is obtained from the merged scene data. These trajectory data come from high-quality scene data that have been previously processed by stationary point detection, data interpolation, and trajectory smoothing. For each trajectory to be trained, it is divided into a first historical trajectory and a second historical trajectory in chronological order. This division is usually based on a predefined time point. For example, the first 60% of the trajectory points can be used as the first historical trajectory and the last 40% as the second historical trajectory. This division method aims to use the first historical trajectory to predict the second historical trajectory, thereby training the prediction ability of the model. Next, the first historical trajectory is input into the encoder of the encoder-decoder model for encoding to obtain an encoding vector. The encoder is usually a recurrent neural network (RNN) structure, such as a long short-term memory network (LSTM) or a gated recurrent unit (GRU). The encoder processes each time step of the first historical trajectory step by step and compresses the trajectory information into a fixed-length vector, which is the encoding vector, which contains the key features and context information of the trajectory.

[0096] The resulting encoding vector is then input into the multi-stage decoder of the encoder-decoder model. The multi-stage decoder is an innovation of the present method, which divides the prediction process into multiple continuous sub-time periods. These sub-time periods correspond to the multiple continuous sub-time periods into which the second historical trajectory is divided. For example, if the second historical trajectory is 6 seconds long, it can be divided into 3 sub-time periods of 2 seconds. Each level of the multi-stage decoder is responsible for predicting the trajectory of a sub-time period. During the decoding process, the first-stage decoder uses the encoding vector as input to predict the trajectory of the first sub-time period. Then, the second-stage decoder uses the output of the first-stage decoder as input to predict the trajectory of the second sub-time period, and so on. This multi-stage decoding approach allows the model to gradually accumulate and update information when predicting long-term trajectories, which helps to improve the accuracy of long-term predictions.

[0097] After the prediction is completed, the predicted trajectories of multiple consecutive sub-time periods are spliced ​​together to obtain a complete predicted trajectory. Then, this predicted trajectory is compared with the actual second historical trajectory. The comparison usually uses indicators such as mean square error (MSE) or mean absolute error (MAE) to measure the gap between the predicted trajectory and the actual trajectory. Finally, the encoder-decoder model is trained based on the comparison results. This process usually uses the back propagation algorithm to adjust the parameters of the model by minimizing the error between the predicted trajectory and the actual trajectory. The training process is iterative, continuously optimizing the model until a predetermined stopping condition is reached, such as the error is lower than a certain threshold or the maximum number of iterations is reached.

[0098] For example: suppose there is a trajectory data of an unmanned truck in a mine, with a total length of 100 seconds and 500 sampling points (sampling frequency is 5Hz). The first 60 seconds (300 points) are taken as the first historical trajectory, and the last 40 seconds (200 points) are taken as the second historical trajectory. The encoder processes the first historical trajectory of 300 points and generates a 256-dimensional encoding vector. This vector contains key features such as the vehicle's motion pattern, speed change, and steering information in the first 60 seconds. The multi-level decoder divides the 40-second prediction task into four 10-second sub-time periods. The first-level decoder uses the 256-dimensional encoding vector as input to predict the trajectory of the first 10 seconds (50 points). The second-level decoder then uses the output of the first level as input to predict the next 10 seconds, and so on.

[0099] Finally, the prediction results of the four sub-time periods are spliced ​​into a complete 40-second prediction trajectory. This predicted trajectory is compared with the actual second historical trajectory. Assuming that the mean absolute error (MAE) is used as the evaluation metric, the calculated MAE is 0.5 meters. Based on this error, the model updates the parameters through the back-propagation algorithm. For example, if it is found that the model has a large error in predicting turns, then the relevant parameters for turning prediction will be adjusted more. This process is repeated, using different trajectory data each time, until the model performance is no longer significantly improved. The multi-agent interactive behavior prediction model trained in this way can not only accurately predict short-term trajectories, but also improve the accuracy of long-term predictions through multi-level decoding. This is particularly important for unmanned driving systems in mining environments, because it requires accurate long-term planning in complex terrain and multi-vehicle collaborative environments.

[0100] In a specific embodiment, the number of stages of the multi-stage decoder is the same as the number of the plurality of continuous sub-time periods; the encoding vector is input into the multi-stage decoder of the encoder-decoder model, and the trajectories of the plurality of continuous sub-time periods are predicted respectively by the multi-stage decoder, including:

[0101] (1) Input the encoding vector into the first-level decoder, and output the predicted trajectory of the first sub-time period among multiple consecutive sub-time periods;

[0102] (2) For each level of decoder after the first level decoder, the predicted trajectory of the sub-time period corresponding to the previous level decoder is used as the input of the next level decoder, and the predicted trajectory of the sub-time period corresponding to the next level decoder is output; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

[0103] Optionally, a multi-stage decoder is designed to improve the accuracy of long-term trajectory prediction. The number of stages of the multi-stage decoder is the same as the number of continuous sub-time periods into which the trajectory to be predicted is divided. This design ensures that each sub-time period is processed by a dedicated decoder. The workflow of the multi-stage decoder begins by inputting the encoding vector into the first-stage decoder. The encoding vector is generated by the encoder after processing the first historical trajectory, and contains the key features and contextual information of the trajectory. The first-stage decoder receives this encoding vector and outputs the predicted trajectory for the first sub-time period among multiple continuous sub-time periods. This process can be understood as the model's prediction of the near future based on historical information.

[0104] Furthermore, for each level of decoder after the first level decoder, a cascaded approach is used for prediction. Specifically, the predicted trajectory of the sub-time period corresponding to the previous level decoder predicted by the previous level decoder is used as the input of the next level decoder. The advantage of this design is that the decoder at each level not only uses the initial coding information, but also can use the prediction results of the previous time period, so as to continuously update and accumulate information during the prediction process. Each decoder outputs the predicted trajectory of the sub-time period corresponding to the decoder, and this process continues until the last level decoder completes the prediction.

[0105] To better understand this process, let's take a specific unmanned driving scenario in a mine as an example. Suppose you need to predict the trajectory of a mining truck in the next 60 seconds. The 60 seconds are divided into three 20-second sub-time periods. Correspondingly, the multi-level decoder also has three levels. First, the encoder processes the historical trajectory data for the past 5 minutes and generates a 256-dimensional encoding vector. This vector contains information such as the average speed of the vehicle, common steering patterns, and interactions with other vehicles.

[0106] The first-level decoder receives this 256-dimensional encoding vector as input. Through a series of neural network layers (such as LSTM layers and fully connected layers), it outputs the predicted trajectory for the previous 20 seconds. Assuming that our trajectory sampling frequency is 1Hz, then this output is a 20x3 matrix, where 20 is the number of time steps and 3 represents the x, y coordinates and direction angles of each time step. The second-level decoder not only receives the original 256-dimensional encoding vector, but also receives the 20x3 matrix output by the first-level decoder as input. This means that when making predictions, the second-level decoder not only considers the original historical information, but also considers the possible motion state of the vehicle in the next 20 seconds that has just been predicted. The second-level decoder also outputs a 20x3 matrix, representing the predicted trajectory for 21-40 seconds.

[0107] The third-level decoder repeats a similar process, receiving the original encoding vector, the prediction results of the first and second levels as input, and outputting the predicted trajectory for the last 20 seconds (41-60 seconds). This cascaded prediction method is particularly suitable for long-term trajectory prediction in mining environments. For example, if the first-level decoder predicts that the vehicle starts to climb a slope within the first 20 seconds, the second-level decoder can predict the possible deceleration behavior of the vehicle in the next 20 seconds based on this information. Similarly, if the first two-level predictions show that the vehicle is approaching a loading point, the third-level decoder can predict that the vehicle may slow down and eventually stop in the last 20 seconds.

[0108] In this way, the multi-level decoder can gradually build a coherent long-term trajectory prediction, with each level adding new information and predictions based on the previous level. This not only improves the accuracy of the long-term prediction, but also makes the prediction results more consistent with the actual vehicle movement laws and mine operation logic. Finally, the three 20-second prediction trajectories are spliced ​​together to form a complete 60-second prediction trajectory.

[0109] It should be noted that in this application, the decoding results of different time periods are spliced ​​together to compare the prediction results of ordinary decoders. Figure 3 As shown, the prediction result of the decoder of this method (such as Figure 4 ),

[0110] Among them, Figure 3 In the example, segment AB is the historical trajectory, segment BC is the true value of the predicted trajectory, and segment BD is the multiple predicted trajectories predicted by the common decoder. Figure 3 As can be seen from the figure, the vanilla decoder does not correctly predict the agent’s trajectory.

[0111] exist Figure 4 In the example, the EF segment is the historical trajectory, the FG segment is the true value of the predicted trajectory, and the trajectories with arrows in the FH and FG segments are the spliced ​​predicted trajectories predicted by multiple decoders in this application. Figure 4 It can be seen that in this method, a more reasonable prediction result is obtained by a step-by-step approach.

[0112] The above describes the method for constructing an intelligent agent interaction scene and the method for training a behavior prediction model in the embodiment of the present application. The following describes the device for constructing an intelligent agent interaction scene in the embodiment of the present application. Figure 5 , an embodiment of the intelligent agent interaction scene construction device in the embodiment of the present application includes:

[0113] An acquisition module 301 is used to acquire sensor data of multiple agents and map data of geographic locations corresponding to the sensor data;

[0114] A detection module 302 is used to perform trajectory detection on other agents detected by the sensor data of each agent to obtain a first trajectory; and determine a second trajectory of the agent based on the sensor data;

[0115] A screening module 303, configured to screen the first trajectory and / or the second trajectory according to a scene data access condition corresponding to a preset scene to obtain a target trajectory;

[0116] The merging module 304 is used to merge the map data and the target trajectory to obtain merged scene data.

[0117] The detection module 302 is specifically used to: perform semantic segmentation on the sensor data of each intelligent agent; perform target detection on the segmented data; and track the detected target intelligent agent; determine the target tracking data corresponding to the same target intelligent agent based on the target tracking results corresponding to the sensor data of multiple intelligent agents; and determine the first trajectory corresponding to each target intelligent agent based on the target tracking data corresponding to the target intelligent agent.

[0118] The screening module 303 includes:

[0119] A determination unit, used to determine a trajectory characteristic index corresponding to the driving trajectory according to a scene data access condition corresponding to a preset scene;

[0120] an extraction unit, configured to extract data features for each trajectory feature index of each trajectory of the first trajectory and / or the second trajectory, and obtain trajectory feature data corresponding to each feature index of the trajectory;

[0121] The screening unit is used to screen the trajectories based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory to obtain a target trajectory.

[0122] The screening unit is specifically used for:

[0123] For each trajectory characteristic indicator, determining the judgment condition of the access condition for the trajectory indicator; and comparing the trajectory characteristic data corresponding to the trajectory characteristic indicator extracted for each trajectory with the corresponding judgment condition;

[0124] The trajectory whose trajectory feature data satisfies the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

[0125] The intelligent agent interaction scene construction device also includes:

[0126] A determination module, used to determine the distribution characteristic index corresponding to the merged scene data according to the scene data distribution condition corresponding to the preset scene;

[0127] An extraction module, used for extracting scene distribution data features from the merged scene data for each distribution feature index respectively;

[0128] The enhancement module is used to determine whether the currently merged scene data meets the distribution condition based on the extracted scene distribution data features; if not, the scene data is enhanced to obtain the target merged scene data.

[0129] The enhancement module is specifically used to: for each distribution feature indicator, determine whether the extracted scene distribution data feature can cover the scene distribution represented by the distribution feature indicator; if it cannot be covered, determine the missing scene data and supplement the missing scene data.

[0130] The intelligent agent interaction scene construction device also includes:

[0131] The stationary point module is used to perform stationary point detection processing on the target trajectory to obtain stationary point detection trajectory data;

[0132] An interpolation module, used to determine a data segment to be filled corresponding to a target trajectory according to the stationary point detection trajectory data, and perform trajectory interpolation processing on the data segment to be filled to obtain interpolated trajectory data;

[0133] A smoothing module, used for performing trajectory smoothing processing on the interpolation trajectory data to obtain an enhanced target trajectory;

[0134] The fusion module is used to merge the map data and the target trajectory to obtain merged scene data, including: merging the map data and the enhanced target trajectory to obtain merged scene data.

[0135] The following describes the behavior prediction model training device in the embodiment of the present application. Figure 6 , an embodiment of the behavior prediction model training device in the embodiment of the present application includes:

[0136] The acquisition module 401 is used to obtain the trajectory to be trained from the merged scene data;

[0137] A division module 402 is used to divide each to-be-trained trajectory into a first historical trajectory and a second historical trajectory in chronological order;

[0138] An encoding module 403 is used to input the first historical trajectory into an encoder of an encoder-decoder model for encoding to obtain an encoding vector;

[0139] A prediction module 404, configured to input the encoding vector into a multi-stage decoder of the encoder-decoder model, and predict the trajectories of a plurality of continuous sub-time periods respectively through the multi-stage decoder; wherein the plurality of continuous sub-time periods correspond to the plurality of continuous sub-time periods into which the second historical trajectory is divided;

[0140] A splicing module 405 is used to splice the predicted trajectories of multiple consecutive sub-time periods and compare them with the second historical trajectory;

[0141] The training module 406 is used to train the encoder-decoder model according to the comparison results to obtain a multi-agent interaction behavior prediction model.

[0142] The prediction module 404 is specifically used to: input the coding vector into the multi-stage decoder of the encoder-decoder model, and predict the trajectories of multiple continuous sub-time periods respectively through the multi-stage decoder, including: inputting the coding vector into the first-stage decoder, and outputting the predicted trajectory of the first sub-time period among the multiple continuous sub-time periods; for each level of decoder after the first-stage decoder, using the predicted trajectory of the sub-time period corresponding to the previous level decoder predicted by the previous level decoder as the input of the next level decoder, and outputting the predicted trajectory of the sub-time period corresponding to the next level decoder; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

[0143] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, systems and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0144] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for constructing an intelligent agent interaction scene, characterized in that: The following steps are involved: Acquire sensor data of multiple agents and map data of geographic locations corresponding to the sensor data; For each agent's sensor data, performing trajectory detection on other agents detected and represented by the agent's sensor data to obtain a first trajectory; and determining a second trajectory of the agent based on the sensor data; According to the scene data access conditions corresponding to the preset scene, the first trajectory and / or the second trajectory are screened to obtain a target trajectory, including: determining a trajectory feature index corresponding to the driving trajectory according to the scene data access conditions corresponding to the preset scene; for each trajectory of the first trajectory and / or the second trajectory, respectively extracting data features for each trajectory feature index to obtain trajectory feature data corresponding to each feature index of the trajectory; based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, respectively, screening the trajectory to obtain the target trajectory, wherein the scene data access conditions include trajectory-agent relationship features, and the trajectory-agent relationship features are used to characterize the interactive behavior features of the trajectory corresponding to the intelligent body movement and other intelligent bodies; The map data and the target track are merged to obtain merged scene data.

2. The method according to claim 1, characterized in that For each agent's sensor data, trajectory detection is performed on other agents detected by the agent's sensor data to obtain a first trajectory, including: For each agent's sensor data, semantic segmentation is performed on the agent's sensor data; target detection is performed on the segmented data; and target tracking is performed on the detected target agent; Determine target tracking data corresponding to the same target agent based on target tracking results corresponding to sensor data of multiple agents; For each target intelligent agent, a first trajectory corresponding to the target intelligent agent is determined according to the target tracking data corresponding to the target intelligent agent.

3. The method according to claim 1, characterized in that The trajectory characteristic index includes at least one of the following: a data health characteristic index, a trajectory and agent relationship characteristic index, and a spatiotemporal characteristic index; The data health characteristic index is used to characterize at least one of the following trajectory data: data integrity, duration, and the shape of the trajectory being characterized; the trajectory-agent relationship characteristic index is used to characterize the interactive behavior characteristics of the trajectory-corresponding agent movement with other agents and / or the position relationship characteristics with its own trajectory; the spatiotemporal characteristic index is used to characterize at least one of the following spatiotemporal position characteristics, motion parameter characteristics, and trajectory shape characteristics of the trajectory-corresponding agent; Based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, trajectory screening is performed to obtain a target trajectory, including: For each trajectory characteristic indicator, determining the judgment condition of the access condition for the trajectory characteristic indicator; and comparing the trajectory characteristic data corresponding to the trajectory characteristic indicator extracted for each trajectory with the corresponding judgment condition; The trajectory whose trajectory feature data satisfies the judgment conditions corresponding to each trajectory feature index is determined as the target trajectory.

4. The method according to claim 1, characterized in that Also includes: Determine the distribution characteristic index corresponding to the merged scene data according to the scene data distribution condition corresponding to the preset scene; According to each distribution characteristic index, scene distribution data characteristics are extracted from the merged scene data; According to the extracted scene distribution data features, it is determined whether the currently merged scene data meets the distribution conditions; if not, the scene data is enhanced to obtain the target merged scene data.

5. The method according to claim 4, characterized in that The distribution characteristic index includes at least one of the following: a business scenario characteristic index, a time distribution characteristic index, and a space distribution characteristic index; Determining whether the currently merged scene data meets the distribution condition according to the extracted scene distribution data characteristics; In the case of non-compliance, the scene data is enhanced, including: For each distribution feature indicator, determining whether the extracted scene distribution data feature can cover the scene distribution represented by the distribution feature indicator; If coverage is not possible, determine the missing scene data and supplement the missing scene data.

6. The method according to claim 1, characterized in that Before merging the map data and the target track, the method further includes: Perform stationary point detection processing on the target trajectory to obtain stationary point detection trajectory data; Determine a data segment to be filled corresponding to the target trajectory according to the stationary point detection trajectory data, and perform trajectory interpolation processing on the data segment to be filled to obtain interpolated trajectory data; Performing trajectory smoothing processing on the interpolation trajectory data to obtain an enhanced target trajectory; The map data and the target trajectory are merged to obtain merged scene data, including: the map data and the enhanced target trajectory are merged to obtain merged scene data.

7. A behavior prediction model training method, characterized in that: include: Acquire the trajectory to be trained from the merged scene data, wherein the merged scene data is obtained by the method according to any one of claims 1 to 6; For each to-be-trained trajectory, the to-be-trained trajectory is divided into a first historical trajectory and a second historical trajectory in chronological order; Inputting the first historical trajectory into an encoder of an encoder-decoder model for encoding to obtain an encoding vector; Inputting the encoding vector into a multi-stage decoder of the encoder-decoder model, and predicting the trajectories of a plurality of continuous sub-time periods respectively through the multi-stage decoder; wherein the plurality of continuous sub-time periods correspond to the plurality of continuous sub-time periods into which the second historical trajectory is divided; splicing the predicted trajectories of multiple consecutive sub-time periods and comparing them with the second historical trajectory; The encoder-decoder model is trained according to the comparison results to obtain a multi-agent interaction behavior prediction model.

8. The method according to claim 7, characterized in that The number of stages of the multi-stage decoder is the same as the number of the plurality of consecutive sub-time periods; Inputting the encoding vector into a multi-stage decoder of the encoder-decoder model, and predicting the trajectories of a plurality of consecutive sub-time periods respectively through the multi-stage decoder, comprising: Inputting the encoding vector into a first-stage decoder, and outputting a predicted trajectory for a first sub-time period in the plurality of consecutive sub-time periods; For each level of decoder after the first level decoder, the predicted trajectory of the sub-time period corresponding to the previous level decoder predicted by the previous level decoder is used as the input of the next level decoder, and the predicted trajectory of the sub-time period corresponding to the next level decoder is output; until the last level decoder outputs the predicted trajectory of the sub-time period corresponding to the last level decoder.

9. An intelligent agent interactive scene construction device, characterized in that: include: An acquisition module, used to acquire sensor data of multiple agents and map data of geographical locations corresponding to the sensor data; A detection module, configured to perform trajectory detection on other detected intelligent agents represented by the sensor data of each intelligent agent, to obtain a first trajectory; and determining a second trajectory of the agent based on the sensor data; A screening module, for screening the first trajectory and / or the second trajectory according to the scene data access conditions corresponding to the preset scene to obtain a target trajectory, including: determining the trajectory feature index corresponding to the driving trajectory according to the scene data access conditions corresponding to the preset scene; for each trajectory of the first trajectory and / or the second trajectory, respectively extracting data features for each trajectory feature index to obtain trajectory feature data corresponding to each feature index of the trajectory; based on the access conditions and the trajectory feature data corresponding to each trajectory feature index extracted for each trajectory, screening the trajectory to obtain the target trajectory, wherein the scene data access conditions include trajectory and agent relationship features, and the trajectory and agent relationship features are used to characterize the interactive behavior features of the trajectory corresponding to the agent movement and other agents; The merging module is used to merge the map data and the target trajectory to obtain merged scene data.

10. A behavior prediction model training device, characterized in that: include: An acquisition module, used to obtain the trajectory to be trained from the merged scene data, wherein the merged scene data is obtained by the method according to any one of claims 1 to 6; A division module, for dividing each to-be-trained trajectory into a first historical trajectory and a second historical trajectory in chronological order; an encoding module, configured to input the first historical trajectory into an encoder of an encoder-decoder model for encoding to obtain an encoding vector; a prediction module, configured to input the encoding vector into a multi-stage decoder of the encoder-decoder model, and predict the trajectories of a plurality of continuous sub-time periods respectively through the multi-stage decoder; wherein the plurality of continuous sub-time periods correspond to the plurality of continuous sub-time periods into which the second historical trajectory is divided; A splicing module, used for splicing the predicted trajectories of multiple consecutive sub-time periods and comparing them with the second historical trajectory; The training module is used to train the encoder-decoder model according to the comparison results to obtain a multi-agent interaction behavior prediction model.

11. A cloud platform, characterized in that: include: An intelligent agent interaction scene construction device as described in claim 9 and / or a behavior prediction model training device as described in claim 10.

Images