An agent trajectory prediction method, device, equipment and medium
By using an equivariant prediction model, which utilizes an equivariant recurrent neural network and equivariant constant differential equations to process the agent's historical data, the generalization problem of existing trajectory prediction methods under geometric transformations is solved. This achieves accurate trajectory prediction in rotation and translation scenarios, improving the model's robustness and generalization ability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN MSU-BIT UNIVERSITY
- Filing Date
- 2026-03-16
- Publication Date
- 2026-08-04
AI Technical Summary
Existing trajectory prediction methods do not have strong generalization ability to geometric transformations and cannot output the corresponding motion trajectory after geometric transformation when the agent's historical observation data undergoes geometric transformation.
An equivariant prediction model is adopted, which uses an equivariant recurrent neural network (E-RNN) composed of an encoder and a decoder and equivariant neural ODEs to process the agent's historical position, velocity and acceleration, ensuring that the predicted velocity and historical data maintain the same geometric transformation characteristics, thereby predicting the agent's future trajectory.
It can accurately predict the trajectory of an agent even when geometric transformations such as rotation and translation occur in the agent's motion scene, improving the robustness and generalization ability of the model and reducing the amount of training data required.
Smart Images
Figure CN121858926B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, specifically to a method, apparatus, device, and medium for predicting the trajectory of an intelligent agent. Background Technology
[0002] Trajectory prediction is a core technology in autonomous driving, robotics, and human-computer interaction. It involves predicting the future movement of a dynamic agent (such as a pedestrian or vehicle) based on historical observation data. Existing technologies use predictive models to predict the future trajectory of an agent based on its historical observation data. However, because existing predictive models lack equivariance, they cannot output the trajectory after geometric transformation if the historical observation data undergoes a geometric change.
[0003] In summary, existing trajectory prediction methods do not have strong generalization ability to geometric transformations.
[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention
[0005] To address the aforementioned technical problems, this invention provides a method, apparatus, device, and medium for predicting the trajectory of an intelligent agent, which solves the problem that existing trajectory prediction methods do not have strong generalization ability for geometric transformations.
[0006] To achieve the above objectives, the present invention adopts the following technical solution:
[0007] In a first aspect, the present invention provides a method for predicting the trajectory of an intelligent agent, comprising:
[0008] Acquire historical movement data of the intelligent agent;
[0009] An equivariant prediction model is applied to the historical movement data to predict the speed of the agent, resulting in a predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data.
[0010] Based on the predicted speed, the predicted trajectory of the agent is obtained.
[0011] In one implementation, an equivariant prediction model is applied to the historical mobility data to predict the agent's speed, resulting in a predicted speed, including:
[0012] Determine the historical position, historical velocity, and historical acceleration from the historical movement data;
[0013] An equivariant prediction model is applied to the historical position, historical velocity, and historical acceleration to predict the agent's velocity, thus obtaining the predicted velocity.
[0014] In one implementation, the equivariant prediction model consists of an encoder and a decoder. The equivariant prediction model is applied to the historical position, the historical velocity, and the historical acceleration to predict the agent's velocity, resulting in a predicted velocity, including:
[0015] The encoder is applied to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration.
[0016] A decoder is applied to the hidden state to predict the agent's speed, resulting in a predicted speed. The decoder has the property of ensuring that the predicted speed maintains the same geometric transformation as the hidden state.
[0017] In one implementation, the encoder is an equivariant recurrent neural network, and the decoder is a decoder constructed based on equivariant recurrent differential equations.
[0018] In one implementation, a decoder is applied to the hidden state to predict the agent's speed, resulting in a predicted speed, including:
[0019] A decoder is applied to the hidden state to decode the hidden state through an equivariant linear layer of the decoder, thereby obtaining decoded features;
[0020] The decoded features are integrated with respect to time to predict the speed of the agent, thus obtaining the predicted speed.
[0021] In one implementation, the training of the isovariant prediction model includes:
[0022] Obtain the sample movement data of the agent at the sample time and the sample velocity of the agent after the sample time;
[0023] The equivariant prediction model is applied to the sample movement data, and the equivariant prediction model predicts the training speed of the agent;
[0024] Based on the sample velocity and the training velocity, determine the loss function of the isovariant prediction model;
[0025] The training of the isovariant prediction model is completed based on the loss function.
[0026] In one implementation, the geometric transformation includes rotation and translation.
[0027] Secondly, embodiments of the present invention also provide an intelligent agent trajectory prediction device, wherein the device comprises the following components:
[0028] The data acquisition module is used to acquire the agent's historical movement data;
[0029] The speed prediction module is used to apply an equivariant prediction model to the historical movement data to predict the speed of the agent and obtain the predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data.
[0030] A trajectory prediction module is used to obtain the predicted trajectory of the agent based on the predicted speed;
[0031] In one implementation, the velocity prediction module includes the following components:
[0032] A data decomposition unit is used to determine the historical position, historical velocity, and historical acceleration in the historical movement data;
[0033] The prediction unit is used to apply an equivariant prediction model to the historical position, the historical velocity, and the historical acceleration to predict the velocity of the agent and obtain the predicted velocity.
[0034] In one implementation, the isovariant prediction model consists of an encoder and a decoder, and the prediction unit includes the following components:
[0035] A hidden state calculation component is used to apply the encoder to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration.
[0036] A velocity prediction component is used to apply a decoder to the hidden state to predict the velocity of the agent, thereby obtaining a predicted velocity. The decoder has the property of ensuring that the predicted velocity maintains the same geometric transformation as the hidden state.
[0037] In one implementation, the velocity prediction component includes the following components:
[0038] A decoding component applies a decoder to the hidden state to decode the hidden state through an equivariant linear layer of the decoder, thereby obtaining decoded features;
[0039] An integral component is used to integrate the decoded features with respect to time to predict the speed of the agent, thereby obtaining the predicted speed.
[0040] In one implementation, the device further includes the following components:
[0041] The sample acquisition module is used to acquire the sample movement data of the agent at the sample time and the sample velocity of the agent after the sample time.
[0042] The training prediction module is used to apply the equivariant prediction model to the sample movement data, and the equivariant prediction model predicts the training speed of the agent.
[0043] The loss function calculation module is used to determine the loss function of the equivariant prediction model based on the sample speed and the training speed.
[0044] The training module is used to train the equal variation prediction model based on the loss function.
[0045] Thirdly, embodiments of the present invention also provide a terminal device, wherein the terminal device includes a memory, a processor, and an intelligent agent trajectory prediction program stored in the memory and executable on the processor, wherein when the processor executes the intelligent agent trajectory prediction program, it implements the steps of the intelligent agent trajectory prediction method described above.
[0046] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing an agent trajectory prediction program, wherein when the agent trajectory prediction program is executed by a processor, it implements the steps of the agent trajectory prediction method described above.
[0047] Beneficial Effects: This invention applies an equivariant prediction model to the historical movement data of an intelligent agent. The equivariant prediction model predicts the agent's future speed based on this historical data, and finally constructs the agent's predicted trajectory based on the predicted speed. The equivariant prediction model of this invention possesses equivariance, meaning that when the movement data input to the equivariant prediction model undergoes a geometric transformation, the model will also perform a corresponding geometric transformation on the output predicted speed, thus obtaining the predicted trajectory after the geometric transformation. From the above analysis, it can be seen that as long as the equivariant prediction model has been trained, even if the movement data input to the equivariant prediction model during the inference stage undergoes a geometric transformation relative to the movement data used in the training stage, the equivariant prediction model during the inference stage can still output the corresponding predicted speed based on the geometrically transformed movement data. Therefore, the trajectory prediction method of this invention is applicable to scenarios involving geometric transformations. Attached Figure Description
[0048] Figure 1 This is an overall flowchart of the present invention;
[0049] Figure 2 This is a schematic diagram of scene rotation in an embodiment of the present invention;
[0050] Figure 3 This is a schematic diagram of the codec in an embodiment of the present invention;
[0051] Figure 4A structural diagram of the intelligent agent trajectory prediction device provided by the present invention;
[0052] Figure 5 This is a block diagram illustrating the internal structure of a terminal device provided in an embodiment of the present invention. Detailed Implementation
[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0054] Research has shown that trajectory prediction is a core technology in autonomous driving, robotics, and human-computer interaction. Trajectory prediction involves predicting the future movement of a dynamic agent (such as a pedestrian or vehicle) based on historical observation data. Existing technologies use predictive models to predict the future trajectory of an agent based on its historical observation data. However, because existing predictive models lack equivariance, they cannot output the corresponding geometrically transformed trajectory if the historical observation data undergoes a geometric transformation.
[0055] In other words, existing models often struggle to effectively handle the inherent geometric and physical symmetries in trajectory data. For example, rotating the scene before predicting the trajectory should be equivalent to predicting the trajectory before rotating. However, standard recurrent neural networks do not have an explicit mechanism to maintain this invariance or equivariance.
[0056] To address the aforementioned technical problems, this invention provides a method, apparatus, device, and medium for predicting the trajectory of an intelligent agent. It solves the problem that existing trajectory prediction methods lack strong generalization ability to geometric transformations. First, historical movement data of the intelligent agent is acquired; then, an isovariant prediction model is applied to the historical movement data to predict the speed of the intelligent agent, thus obtaining the predicted speed; finally, based on the predicted speed, the predicted trajectory of the intelligent agent is obtained, and the predicted trajectory is used to characterize the future trajectory of the intelligent agent.
[0057] For example, the intelligent agent could be a self-driving car using an autonomous driving mode. The self-driving car uses an equal-variable prediction model to predict the trajectory of other cars and controls its future movement trajectory based on the predicted trajectory of other cars.
[0058] like Figure 2 As shown, Figure 2 The green trajectory represents the historical movement data of his vehicle, and the red trajectory represents the predicted trajectory of his vehicle. Figure 2 The squares in the diagram represent your own car, and the circles represent other cars.
[0059] Figure 2The upper part represents the driving scenario of another vehicle traveling from south to north to a T-junction. The own vehicle is located at the side intersection of the T-junction, meaning it is behind the other vehicle in the north-south direction. The own vehicle uses its built-in isovariant prediction model to predict the speed of the other vehicle based on its historical movement data. Then, the own vehicle obtains the predicted trajectory of the other vehicle based on the predicted speed. Finally, the own vehicle determines whether the other vehicle should turn west or east based on the predicted trajectory, and controls its future movement trajectory based on the predicted turning direction of the other vehicle.
[0060] By rotating the driving scene of his car from south to north by 90 degrees, we get the following: Figure 2 The lower half of the driving scenario depicts a car traveling from east to west towards a T-junction, while the driver's car is positioned at the main intersection. The driver uses its built-in isotropic prediction model to predict the speed of the other car traveling east-west based on its historical movement data in this scenario. Then, the driver uses this predicted speed to determine the other car's predicted trajectory. Finally, based on the predicted trajectory, the driver determines whether traveling straight north-south will result in a collision with the other car, and controls its future trajectory based on this determination.
[0061] Because the isotropic prediction model of this invention is isotropic, even if the driving scene changes, the vehicle can still predict the speed of other vehicles through its built-in isotropic prediction model.
[0062] Example 1 provides a training method for an equivariant prediction model. This equivariant prediction model consists of an encoder and a decoder. The encoder is an equivariant recurrent neural network (E-RNN), which includes repeatedly used equivariant neural networks. An equivariant neural network is a model based on a recurrent neural network architecture, and its linear layers exhibit equivariance. The decoder is constructed based on equivariant neural ordinary differential equations (ODEs).
[0063] Equivariant recurrent neural networks are networks that combine recurrent neural networks with equivariant deep learning (equivariant linear layers).
[0064] E-RNNs exhibit equivariance, meaning that a geometric transformation of the input will result in a corresponding geometric transformation of the output. For example, in keypoint detection, the input is a face image, and the output is the coordinates of keypoints such as the eyes and the tip of the nose. If the image is rotated 90 degrees, the predicted keypoint coordinates will also rotate 90 degrees accordingly, meaning the output undergoes a synchronous rotation transformation with the input.
[0065] Equivariance allows the equivariant prediction model to perform inference and prediction at rotation angles that are not used during the training phase. In other words, equivariance is encoded in the network structure of the equivariant prediction model. If the equivariant prediction model is trained using data from one direction, the trained equivariant prediction model can be applied to all directions. That is, the trained equivariant prediction model can perform inference and prediction based on data from all directions.
[0066] The equivariance of equivariant recurrent neural networks strictly guarantees that the model predictions conform to physical symmetry, avoiding the learning of false non-physical patterns.
[0067] Equivariant deep learning shows great potential in embedding symmetry awareness into neural networks, enabling models to achieve better generalization with fewer training samples. By introducing equivariant linear layers, the model gains transformability through rotation and translation, thereby improving its robustness and operability.
[0068] The algorithm acquires sample movement data of the agent at sample time points and inputs the sample movement data into the encoder. The encoder extracts the trajectory features of the agent from the input sample movement data. These trajectory features are the hidden states. The hidden states are then input into the decoder. The decoder predicts the training speed of the agent after the sample time points based on the hidden states.
[0069] The actual sample velocity of the agent after the sample time is collected, and the difference between the actual sample velocity and the training velocity is calculated. In this embodiment, the mean squared error or negative log-likelihood loss function is used to represent this difference. The parameters of the equivariant prediction model are updated based on this difference to minimize the difference between the training velocity and the sample velocity after geometric transformation. The training method in this embodiment is end-to-end training.
[0070] The above training method can be used to train the equivariant prediction model for any of the SO(2) group, O(2) group, SO(3) group and O(3) group. That is, the training method of this embodiment can be extended to SO(2) group, O(2) group, SO(3) group and O(3) group.
[0071] SO(2) groups are used to describe rotations in two-dimensional space. SO(2) group is the Special Orthogonal Group of degree 2. For example, the change in heading angle of a two-dimensional mobile robot constitutes an SO(2) trajectory.
[0072] The SO(3) group represents the rotation matrix in three-dimensional space. The SO(3) group is the group of all rotations about the origin in three-dimensional Euclidean space. For example, the three-dimensional pose of drones and robotic arms belongs to the SO(3) group.
[0073] The O(2) group is used to describe two-dimensional rotational and reflection symmetry. The O(2) group is the Orthogonal Group of degree2.
[0074] The O(3) group represents a three-dimensional orthogonal transformation. The O(3) group is the Orthogonal Group of degree 3.
[0075] Example 2: This example provides a method for predicting the trajectory of an intelligent agent. This method can be applied to a terminal device, which can be a terminal product with data processing capabilities, such as an intelligent agent controller. In this example, as... Figure 1 As shown, the agent trajectory prediction method specifically includes the following steps:
[0076] S100, acquires the agent's historical movement data;
[0077] S200, Apply an equivariant prediction model to the historical movement data to predict the speed of the agent and obtain the predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data.
[0078] S300, based on the predicted speed, the predicted trajectory of the agent is obtained.
[0079] The predicted trajectory is used to characterize the future trajectory of the agent, and obtaining the predicted trajectory based on the predicted speed is a prior art technique.
[0080] The intelligent agent in step S100 can be either a robot or an autonomous vehicle. When the intelligent agent is a robot, it uses... This represents the historical movement data in step S100.
[0081] ;
[0082] In the formula, 1 represents the start time. Representing the current moment, This represents the set of positions of the robot gripper from the initial time to the current time. This represents the set of velocities of the robot gripper from the initial moment to the current moment. This represents the set of accelerations of the robot gripper from the initial moment to the current moment. In this embodiment, the initial moment to the current moment includes three consecutive moments, meaning that this embodiment predicts the robot gripper's velocity at future moments based on the robot's historical movement data at these three consecutive historical moments.
[0083] Step S200 includes the following specific steps S201, S202, S203, and S204:
[0084] S201, determine the historical position, historical speed and historical acceleration in the historical movement data.
[0085] The historical position in this embodiment includes the position of the agent at three time points, the historical velocity includes the velocity of the agent at three time points, and the historical acceleration includes the acceleration of the agent at three time points.
[0086] S202, the encoder is applied to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration.
[0087] Historical location, historical velocity, and historical acceleration constitute the aforementioned historical movement data. The encoder is the equivalent recurrent neural network E-RNN.
[0088] The encoder is applied to the historical position, historical velocity, and historical acceleration; that is, the historical position, historical velocity, and historical acceleration are input into the E-RNN, and the E-RNN outputs the hidden state. :
[0089] ;
[0090] E-RNN outputs the hidden state based on the following formula. One of them :
[0091] ;
[0092] The first one predicted by E-RNN The hidden state at all times Representing the Historical movement data at any given moment Representing the The hidden state of history at any given moment This represents a linear layer (i.e., the input layer) within an E-RNN, which exhibits equivariance. This represents the hidden layer within an E-RNN, which exhibits equivariance. An RNN containing linear and hidden layers is transformed into an E-RNN by training these layers to exhibit equivariance.
[0093] This embodiment The geometric transformation under the action of the SO(3) group is as follows:
[0094] ;
[0095] In the formula, Represents any geometric transformation in the SO(3) group, represent The first-order irreducible representation, Represents the result after geometric transformation , This represents the set of positions of the agent at all times. This represents the set of velocities of the agent at all times. It represents the set of accelerations of the agent at all times.
[0096] when When the above geometric transformation occurs, A corresponding geometric transformation will also occur, namely:
[0097] ;
[0098] represent pass Effect on , The representation of the group in the hidden state space. Symbol representing function, Represents the state after geometric transformation , It represents the meaning of mapping.
[0099] S203, regarding the hidden state A decoder is applied to decode the hidden state through an equivariant linear layer of the decoder to obtain decoded features. .
[0100] This isovariant linear layer is a fully connected layer with isovariance, which means that the symmetry remains unchanged under geometric transformations such as rotation and translation.
[0101] S204, Integrate the decoded features with respect to time to predict the speed of the agent, and obtain the predicted speed.
[0102] The future velocity is modeled using the isovariant neural ordinary differential equations (ODEs) of the decoder to construct the following ordinary differential equation:
[0103] ;
[0104] represent Differential with respect to time, where After passing through a linear layer, it becomes velocity. Representing a future moment, the starting moment of the future moment is... time, This is the final historical moment, which is also the present moment. The dynamic function representing the agent is equivariant.
[0105] right Perform the following integration:
[0106] ;
[0107] This represents the speed of the agent at a future moment; the speed at a future moment is the predicted speed.
[0108] when When a geometric transformation occurs, Corresponding geometric transformations will also occur, leading to Corresponding geometric transformations will also occur, namely .
[0109] represent pass Effect on , The representation of the group in the output space. Symbol representing function, Represents the state after geometric transformation .
[0110] This can be proven using the following formula. It has equivariance:
[0111] ;
[0112] This embodiment can also predict the agent's future [number]th [time] using the following formula. Speed of time :
[0113] ;
[0114] Represents a non-linear activation function. Representative used for processing A linear transformation layer that is equivariant.
[0115] by Figure 3 For example, the encoder and decoder in this embodiment are explained:
[0116] The agent's movement data is collected at three consecutive historical moments. The historical movement data at these three moments is input into the encoder. The encoder outputs the hidden state. The hidden state is then passed through a linear layer and input into the decoder. The decoder predicts the agent's speed at subsequent consecutive moments.
[0117] Example 3: Based on Example 2, the intelligent agent trajectory prediction method is applied to autonomous vehicles to predict their trajectories, including:
[0118] Autonomous vehicles use their own sensors to collect sample movement data of surrounding vehicles (i.e., the predicted intelligent agents) before a set time point and sample speed data of surrounding vehicles after the set time point.
[0119] The sample movement data of surrounding vehicles are input into the isotropic prediction model, which outputs the training speed of surrounding vehicles after a set time point.
[0120] Based on the mean squared error loss of sample velocity and training velocity, the parameters of the isovariant prediction model are updated to complete the training of the isovariant prediction model.
[0121] After the training of the isotropic prediction model is completed, even if the movement data of surrounding vehicles undergoes geometric transformation, the isotropic prediction model can still predict the speed of surrounding vehicles based on the geometrically transformed movement data because of its isotropic nature.
[0122] Based on the predicted speeds and trajectories of surrounding vehicles, the autonomous vehicle is controlled to implement obstacle avoidance measures in response to the predicted trajectories of surrounding vehicles.
[0123] Example 4, based on Example 2, the process of obtaining the predicted trajectory of the agent based on the predicted speed in this example includes the following specific steps:
[0124] Based on the predicted speed, the distance that the agent can move within the predicted time period is estimated, and the estimated distance is obtained.
[0125] The agent's current position and historical movement trajectory are obtained. Based on the current position, estimated distance, and historical movement trajectory, the agent's predicted trajectory is determined.
[0126] For example, if the intelligent agent is a drone, the first step is to predict the drone's speed at various future moments. Based on these speeds, the average speed is calculated, and the duration of each moment is calculated. Multiplying the speed by this duration gives the estimated distance. If the drone's current location is position A, and its historical movement trajectory includes the trajectory from position A to position B, from position A to position C, and from position A to position D, the distance from position A to position B best matches the estimated distance. Therefore, it can be predicted that the drone's subsequent movement trajectory will be the historical trajectory from position A to position B; that is, the historical trajectory from position A to position B is used as the predicted trajectory.
[0127] The aforementioned drone is located in a drone swarm. The control drone uses the aforementioned trajectory method to predict the movement trajectories of other drones in the drone swarm. The current drone then plans its own trajectory based on the predicted movement trajectories of the other drones.
[0128] As the drone moves along the planned trajectory, obstacles are identified using the lidar on the drone, and the planned trajectory is adjusted in real time based on the identified obstacles to optimize the trajectory.
[0129] In this embodiment, adjusting the planned trajectory based on obstacles to optimize the planned trajectory includes the following specific steps:
[0130] The lidar emits a laser in the direction of the planned trajectory. Based on the time from emission to reception, it calculates the distance between the object reflecting the laser and the current drone. It determines whether this distance is within the planned trajectory range. If it is, it activates the vertical scanning lidar to scan the object vertically. By scanning the object vertically, it determines the object's height and compares it with the drone's obstacle clearance height. If the object's height is lower than or equal to the obstacle clearance height, it is not considered an obstacle, and the drone moves along the planned trajectory. If the object's height is greater than the obstacle clearance height, it is considered an obstacle, and it activates the horizontal scanning lidar to scan the obstacle horizontally. By scanning the obstacle horizontally, it determines the obstacle's width and shifts the predicted trajectory away from the obstacle at the point where it passes it, thus optimizing the planned trajectory.
[0131] The above method is used to optimize the planned trajectory, and then the current UAV is controlled to move along the optimized planned trajectory, including the following control methods:
[0132] Images of various objects along the optimized planned trajectory are collected by the camera carried by the current drone, and obstacle recognition technology is applied to the images to identify each obstacle.
[0133] The distance between the drone's fuselage and the obstacle is measured using the ranging module of the lidar carried by the drone.
[0134] Compare this distance with the current deceleration distance of the drone to determine whether the drone needs to decelerate. If the distance is less than the current deceleration distance, then the drone will decelerate gradually to reduce its speed and prevent it from crashing into an obstacle at high speed and damaging its fuselage.
[0135] This embodiment uses a multimodal sensor to perceive the current flight environment of the UAV. Based on the data perceived by the multimodal sensor, the shape of objects is obtained, the type of objects is estimated based on the shape, and whether an object is an obstacle is determined based on the type of object. The multimodal sensor in this embodiment includes LiDAR, a depth camera, and millimeter-wave radar.
[0136] In summary, this invention embeds equivariance into the prediction model, making the prediction model an equivariant prediction model. When predicting speed, the model will consider physical symmetry, thereby reducing prediction errors without relying on data augmentation when geometric transformations such as rotation and translation occur in the movement scene of the agent.
[0137] The equivariant architecture of this invention can utilize symmetry priors to reduce the need for large-scale training datasets, making the model suitable for low-data-volume scenarios.
[0138] This embodiment also provides an intelligent agent trajectory prediction device, such as... Figure 4 As shown, the device comprises the following components:
[0139] Data acquisition module 01 is used to acquire the historical movement data of the intelligent agent;
[0140] The speed prediction module 02 is used to apply an equivariant prediction model to the historical movement data to predict the speed of the agent and obtain the predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data.
[0141] The trajectory prediction module 03 is used to obtain the predicted trajectory of the agent based on the predicted speed, and the predicted trajectory is used to characterize the future trajectory of the agent.
[0142] In one implementation, the velocity prediction module includes the following components:
[0143] A data decomposition unit is used to determine the historical position, historical velocity, and historical acceleration in the historical movement data;
[0144] The prediction unit is used to apply an equivariant prediction model to the historical position, the historical velocity, and the historical acceleration to predict the velocity of the agent and obtain the predicted velocity.
[0145] In one implementation, the isovariant prediction model consists of an encoder and a decoder, and the prediction unit includes the following components:
[0146] A hidden state calculation component is used to apply the encoder to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration.
[0147] A velocity prediction component is used to apply a decoder to the hidden state to predict the velocity of the agent, thereby obtaining a predicted velocity. The decoder has the property of ensuring that the predicted velocity maintains the same geometric transformation as the hidden state.
[0148] In one implementation, the velocity prediction component includes the following components:
[0149] A decoding component applies a decoder to the hidden state to decode the hidden state through an equivariant linear layer of the decoder, thereby obtaining decoded features;
[0150] An integral component is used to integrate the decoded features with respect to time to predict the speed of the agent, thereby obtaining the predicted speed.
[0151] In one implementation, the device further includes the following components:
[0152] The sample acquisition module is used to acquire the sample movement data of the agent at the sample time and the sample velocity of the agent after the sample time.
[0153] The training prediction module is used to apply the equivariant prediction model to the sample movement data, and the equivariant prediction model predicts the training speed of the agent.
[0154] The loss function calculation module is used to determine the loss function of the equivariant prediction model based on the sample speed and the training speed.
[0155] The training module is used to train the equal variation prediction model based on the loss function.
[0156] Based on the above embodiments, the present invention also provides a terminal device, the principle block diagram of which can be as follows: Figure 5As shown, the terminal device includes a processor, memory, network interface, and display screen connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements an intelligent agent trajectory prediction method. The display screen can be a liquid crystal display (LCD) or an e-ink display.
[0157] Those skilled in the art will understand that Figure 5 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. A specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0158] In one embodiment, a terminal device is provided, comprising a memory, a processor, and an agent trajectory prediction program stored in the memory and executable on the processor. When the processor executes the agent trajectory prediction program, it implements the following operation instructions:
[0159] Acquire historical movement data of the intelligent agent;
[0160] An equivariant prediction model is applied to the historical movement data to predict the speed of the agent, resulting in a predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data.
[0161] Based on the predicted speed, the predicted trajectory of the agent is obtained, and the predicted trajectory is used to characterize the future trajectory of the agent.
[0162] Specifically, applying an equal variation prediction model to the historical mobility data to predict the agent's speed yields the predicted speed, including:
[0163] Determine the historical position, historical velocity, and historical acceleration from the historical movement data;
[0164] An equivariant prediction model is applied to the historical position, historical velocity, and historical acceleration to predict the agent's velocity, thus obtaining the predicted velocity.
[0165] The isovariant prediction model consists of an encoder and a decoder. It applies the isovariant prediction model to the historical position, historical velocity, and historical acceleration to predict the agent's velocity, resulting in a predicted velocity, including:
[0166] The encoder is applied to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration.
[0167] A decoder is applied to the hidden state to predict the agent's speed, resulting in a predicted speed. The decoder has the property of ensuring that the predicted speed maintains the same geometric transformation as the hidden state.
[0168] The encoder is an isovariant recurrent neural network, and the decoder is a decoder constructed based on isovariant recurrent differential equations.
[0169] Specifically, applying a decoder to the hidden state to predict the agent's speed and obtaining the predicted speed includes:
[0170] A decoder is applied to the hidden state to decode the hidden state through an equivariant linear layer of the decoder, thereby obtaining decoded features;
[0171] The decoded features are integrated with respect to time to predict the speed of the agent, thus obtaining the predicted speed.
[0172] The training of the isovariant prediction model includes:
[0173] Obtain the sample movement data of the agent at the sample time and the sample velocity of the agent after the sample time;
[0174] The equivariant prediction model is applied to the sample movement data, and the equivariant prediction model predicts the training speed of the agent;
[0175] Based on the sample velocity and the training velocity, determine the loss function of the isovariant prediction model;
[0176] The training of the isovariant prediction model is completed based on the loss function.
[0177] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.
[0178] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for predicting the trajectory of an intelligent agent, characterized in that, include: Acquire historical movement data of the intelligent agent; An equivariant prediction model is applied to the historical movement data to predict the speed of the agent, resulting in a predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data. The equivariant prediction model consists of an encoder and a decoder. The encoder is an equivariant recurrent neural network, and the decoder is a decoder constructed based on equivariant neural frequent differential equations. The model has equivariance by introducing an equivariant linear layer. Based on the predicted speed, the predicted trajectory of the agent is obtained.
2. The intelligent agent trajectory prediction method as described in claim 1, characterized in that, Applying an equal variation prediction model to the historical mobility data to predict the agent's speed, the predicted speed is obtained, including: Determine the historical position, historical velocity, and historical acceleration from the historical movement data; An equivariant prediction model is applied to the historical position, historical velocity, and historical acceleration to predict the agent's velocity, thus obtaining the predicted velocity.
3. The intelligent agent trajectory prediction method as described in claim 2, characterized in that, Applying an equivariant prediction model to the historical position, historical velocity, and historical acceleration, the agent's velocity is predicted, resulting in a predicted velocity, including: The encoder is applied to the historical position, the historical velocity, and the historical acceleration to obtain a hidden state that characterizes the historical trajectory features of the agent. The encoder has the property that the hidden state maintains the same geometric transformation as the historical position, historical velocity, and historical acceleration. A decoder is applied to the hidden state to predict the agent's speed, resulting in a predicted speed. The decoder has the property of ensuring that the predicted speed maintains the same geometric transformation as the hidden state.
4. The intelligent agent trajectory prediction method as described in claim 3, characterized in that, Applying a decoder to the hidden state to predict the agent's speed, the predicted speed is obtained, including: A decoder is applied to the hidden state to decode the hidden state through an equivariant linear layer of the decoder, thereby obtaining decoded features; The decoded features are integrated with respect to time to predict the speed of the agent, thus obtaining the predicted speed.
5. A device for predicting the trajectory of an intelligent agent, characterized in that, The device comprises the following components: The data acquisition module is used to acquire the agent's historical movement data; A speed prediction module is used to apply an equivariant prediction model to the historical movement data to predict the speed of the agent and obtain the predicted speed. The equivariant prediction model has the characteristic that the predicted speed maintains the same geometric transformation as the historical movement data. The equivariant prediction model consists of an encoder and a decoder. The encoder is an equivariant recurrent neural network, and the decoder is a decoder built based on equivariant neural frequent differential equations. The model has equivariance by introducing an equivariant linear layer. The trajectory prediction module is used to obtain the predicted trajectory of the agent based on the predicted speed.
6. A terminal device, characterized in that, The terminal device includes a memory, a processor, and an agent trajectory prediction program stored in the memory and executable on the processor. When the processor executes the agent trajectory prediction program, it implements the steps of the agent trajectory prediction method as described in any one of claims 1-4.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an agent trajectory prediction program, which, when executed by a processor, implements the steps of the agent trajectory prediction method as described in any one of claims 1-4.