Body-aware physical model fusion based trajectory prediction method, device and medium

Abstract

The application discloses a projectile trajectory prediction method and device based on embodied intelligent physical model fusion, and relates to the field of artificial intelligence.The method comprises the following steps: obtaining a historical trajectory sequence of a projectile, wherein the historical trajectory sequence comprises position information and speed information of multiple continuous historical moments; inputting the historical trajectory sequence into a sequence encoder to extract time sequence features and obtain a context feature vector; inputting the context feature vector into a residual prediction head to predict an acceleration residual, so as to obtain a residual acceleration vector representing an acceleration deviation of a non-ideal aerodynamic effect; adding the residual acceleration vector and a gravitational acceleration constant to obtain a total acceleration vector; taking the position and speed at the last moment as initial states, inputting the total acceleration vector into a differentiable kinematics layer to integrate a kinematics equation, and obtaining a predicted future trajectory sequence comprising position and speed information of multiple future moments.The method combines physical hard constraints and residual learning, and realizes high-precision physical interaction of an embodied intelligent agent under complex aerodynamic interference.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, device and medium for predicting projectile trajectories by fusing embodied intelligent physical models. Background Technology

[0002] When an embodied intelligent agent (such as a humanoid robot or an autonomous mobile operating platform) performs physical interaction tasks such as dynamic target interception or avoidance, its onboard vision system needs to perform real-time trajectory perception and prediction of projectiles (such as thrown tools or accidentally fallen workpieces) to support subsequent motion planning and decision-making.

[0003] The inventors of this application have discovered that the difficulty of this type of task lies in the fact that the projectile is simultaneously affected by a variety of complex non-ideal factors during flight, such as deterministic gravitational acceleration, air resistance, lift, and the Magnus effect caused by rotation. These non-ideal factors exhibit significant uncertainty as environmental conditions and the shape of the object change, making it extremely challenging to accurately predict its future motion.

[0004] Currently, in existing technologies, data-driven trajectory prediction schemes (such as recurrent neural networks or Transformers) directly learn an end-to-end mapping from historical observed trajectories to future trajectories. These methods rely solely on statistical extrapolation, and in long-term predictions, they are prone to producing false trajectories that violate Newton's fundamental laws of kinematics, such as non-parabolic shapes or physically infeasible results like energy non-conservation. This severely weakens the reliability of embodied agents in dynamic scenarios. Furthermore, another type of scheme extrapolates based on purely physical equations of motion (considering only gravitational acceleration). While this method can ensure that the trajectory meets basic physical constraints, it cannot accurately model complex aerodynamic disturbances in real-world environments, leading to significant discrepancies between predicted and actual landing points. In typical conditions such as indoor airflow disturbances or high rotational speeds, purely physical models struggle to meet the basic accuracy requirements of trajectory prediction for grasping and interception tasks.

[0005] In actual deployment, since neither of the above two types of solutions can effectively approximate complex non-ideal aerodynamic effects while ensuring the physical rationality of the prediction results, embodied intelligent agents face specific difficulties such as interception failure or inability to avoid due to inaccurate prediction of the future motion of the projectile when performing high-speed dynamic interaction tasks.

[0006] Therefore, how to provide an effective solution that enables embodied intelligent agents to achieve accurate physical interaction when facing projectiles with complex aerodynamic interference is a practical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] This invention provides a method, device, and medium for predicting projectile trajectories by fusing embodied intelligent physical models. The technical problem it aims to solve is: how to provide an effective solution that enables embodied intelligent agents to achieve accurate physical interaction when facing projectiles with complex aerodynamic interference.

[0008] In a first aspect, the present invention provides a projectile trajectory prediction method based on the fusion of embodied intelligent physical models, comprising: Obtain the historical trajectory sequence of the projectile, the historical trajectory sequence containing the position and velocity information of the projectile at each of multiple consecutive historical moments; The historical trajectory sequence is input into a sequence encoder for temporal feature extraction to obtain a context feature vector; The context feature vector is input into the residual prediction head to perform acceleration residual prediction, and a residual acceleration vector is obtained. The residual acceleration vector is used to characterize the acceleration deviation caused by non-ideal aerodynamic effects. The residual acceleration vector is added to the preset gravitational acceleration constant to obtain the total acceleration vector; The total acceleration vector is input into the differentiable kinematics layer. The position and velocity information corresponding to the last historical moment in the historical trajectory sequence are used as the initial state. The kinematic equations are numerically integrated to obtain the predicted future trajectory sequence. The predicted future trajectory sequence contains the position and velocity information corresponding to each of the multiple future moments.

[0009] Optionally, obtaining the historical trajectory sequence of the projectile includes: Acquire continuous frame images from airborne vision sensors; Target detection is performed on the projectiles in the consecutive frame images to determine the detection bounding box of the projectile in each frame, and the center point of the detection bounding box in each frame is taken as the image position of the projectile in that frame; Cross-frame association matching is performed on the projectiles detected between each frame, and the detection results belonging to the same projectile are associated according to the frame order to obtain the image position sequence of the same projectile in each frame. Each image position in the image position sequence is mapped to the world coordinate system to obtain the world coordinate position corresponding to each frame, which serves as the position information for each frame. The position information of adjacent frames is differentially calculated, and the differential result is divided by the time interval between adjacent frames to obtain the velocity information corresponding to each frame. The location information and velocity information corresponding to multiple consecutive historical frames are assembled in chronological order to obtain the historical trajectory sequence.

[0010] Optionally, the step of inputting the historical trajectory sequence into a sequence encoder for temporal feature extraction to obtain a context feature vector includes: The position and velocity information of each moment in the historical trajectory sequence are concatenated into a single-step state vector to obtain a state vector sequence. The state vector sequence is input into the sequence encoder, which is an LSTM network. The LSTM network performs gated recursive processing on each single-step state vector in the state vector sequence through its forget gate, input gate, and output gate to update the hidden state and cell state of the LSTM network. The hidden state output by the LSTM network at the last time step is used as the context feature vector. Alternatively, the state vector sequence is input into the sequence encoder, which is a Transformer network. The position of each single-step state vector in the state vector sequence is encoded and then input into the encoder of the Transformer network. The encoder of the Transformer network models the temporal dependency between each single-step state vector through a multi-head self-attention mechanism and outputs the encoded feature vector corresponding to each time step. The encoded feature vector corresponding to each time step is pooled, and the aggregated feature vector obtained after pooling is used as the context feature vector.

[0011] Optionally, the step of inputting the context feature vector into the residual prediction head to perform acceleration residual prediction and obtain the residual acceleration vector includes: The context feature vector is input into the residual prediction head, which is a multi-layer fully connected network. The multi-layer fully connected network includes an input layer, at least one hidden layer, and an output layer connected in sequence. The hidden layer uses the ReLU activation function, and the output layer uses a linear activation function. The input layer receives the context feature vector, performs a linear transformation on the weight matrix and bias vector of each hidden layer, and then performs a nonlinear mapping on the ReLU activation function. The output layer outputs a three-dimensional vector, which is used as the residual acceleration vector. The three components of the residual acceleration vector correspond to the acceleration deviation values ​​in the three orthogonal axes of the world coordinate system.

[0012] Optionally, the step of inputting the total acceleration vector into a differentiable kinematics layer, using the position and velocity information corresponding to the last historical moment in the historical trajectory sequence as the initial state, and numerically integrating the kinematic equations to obtain the predicted future trajectory sequence includes: Using the position information of the last historical moment in the historical trajectory sequence as the initial position and the velocity information of the last historical moment in the historical trajectory sequence as the initial velocity, the initial conditions for constructing the kinematic equation system are established. The kinematic equation system includes the first derivative of position with respect to time being equal to velocity and the first derivative of velocity with respect to time being equal to the total acceleration vector. The total acceleration vector is used as the acceleration term in the kinematic equations. Numerical integration is performed using a fourth-order Runge-Kutta integrator. Within each integration step, the fourth-order Runge-Kutta integrator calculates the first derivative estimate at the beginning of the integration step, the second derivative estimate and the third derivative estimate at half the integration step, and the fourth derivative estimate at the end of the integration step. The first derivative estimate, the second derivative estimate, the third derivative estimate and the fourth derivative estimate are weighted and averaged to obtain the final slope of the integration step. The position and velocity of the current step are updated based on the final slope. Repeatedly perform numerical integration until the preset prediction time domain length is reached. Arrange the positions and velocities output by each integration step in chronological order to obtain the predicted future trajectory sequence.

[0013] Optionally, the method further includes a step of performing two-stage training on the sequence encoder and the residual prediction head: In Phase 1, the sequence encoder and the residual prediction head are pre-trained using a projectile motion dataset generated by a pure physics simulation engine. The pure physics simulation engine is configured with only the gravitational acceleration term and the simulation of air resistance and rotation effects is turned off, generating projectile trajectory data driven only by gravity. The projectile trajectory data is used as training samples to iteratively train the sequence encoder and the residual prediction head until the sequence encoder and the residual prediction head converge. The pure physics simulation engine is a computer software system that performs motion simulation based on physical laws. In Phase Two, the pre-trained sequence encoder and residual prediction head are deployed to the embodied agent. The pre-trained sequence encoder and residual prediction head are fine-tuned using the airborne vision sensor of the embodied agent in the real working environment. The learning rate in Phase Two is lower than that in Phase One.

[0014] Optionally, the single-iteration training steps adopted in either stage one or stage two include: Acquire training samples, which include historical trajectory sequence samples of the projectile and their corresponding real future trajectory sequence samples; The historical trajectory sequence samples are input into the sequence encoder for temporal feature extraction to obtain the sample context feature vector; The sample context feature vector is input into the residual prediction head to perform acceleration residual prediction, and the sample residual acceleration vector is obtained. The sample residual acceleration vector is added to the gravitational acceleration constant to obtain the total sample acceleration vector; The total acceleration vector of the sample is input into the differentiable kinematics layer. The position and velocity information of the last moment in the historical trajectory sequence sample are used as the initial state. The kinematic equation is numerically integrated to obtain the sample's predicted future trajectory sequence. Calculate the displacement error between the predicted position at each moment in the sample predicted future trajectory sequence and the actual position at the corresponding moment in the real future trajectory sequence sample, and average the displacement error over all moments to obtain the average displacement error, and use the average displacement error as the main loss term. Based on the position and velocity information at each moment in the sample predicted future trajectory sequence, and the mass parameters of the projectile obtained, the kinetic energy and potential energy at each moment are calculated. The difference between the sum of kinetic energy and potential energy at the end moment and the start moment in the sample predicted future trajectory sequence is calculated, and the portion of the absolute value of the difference that exceeds the preset energy deviation threshold is taken as the energy deviation constraint loss term. The total loss is obtained by weighted summing of the main loss term and the energy deviation constraint loss term. The total loss is backpropagated layer by layer along the differentiable kinematics layer, the residual prediction head, and the sequence encoder using the backpropagation algorithm, and the parameters of the sequence encoder and the residual prediction head are updated synchronously.

[0015] Optionally, after Phase Two is completed and the sequence encoder and the residual prediction head are deployed to the embodied agent, an online adaptive correction step is also included: During deployment and operation, actual observations of the projectile's true trajectory fragments are obtained, and these true trajectory fragments include the location information and corresponding timestamps for each observation time. Extract the predicted trajectory subsequence within the time interval corresponding to the actual trajectory segment from the predicted future trajectory sequence, and calculate the online displacement error between the predicted trajectory subsequence and the actual trajectory segment; Determine whether the online displacement error exceeds a preset error threshold; If the online displacement error does not exceed the error threshold, keep the current network parameters unchanged and continue to output the predicted trajectory for subsequent time moments; If the online displacement error exceeds the error threshold, an online parameter update is triggered. The online parameter update is as follows: only the parameters of the residual prediction head are updated by single-step gradient descent, while keeping the parameters of the sequence encoder and the differentiable kinematics layer unchanged, and the updated residual prediction head is used for trajectory prediction at subsequent time steps. The variation trend of each component of the updated residual acceleration vector is analyzed to determine the variation amplitude of the residual acceleration in each orthogonal axis direction. The axis direction whose variation amplitude exceeds the preset amplitude threshold is identified as the dominant interference direction, and the dominant interference direction is output.

[0016] Secondly, the present invention also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0017] Thirdly, the present invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0018] This invention provides a projectile trajectory prediction method, device, and medium that integrates an embodied intelligent physical model. By forcibly adding the residual acceleration vector to the gravitational acceleration constant and then inputting it into a differentiable kinematics layer for integration, Newton's equations of motion are embedded in the network structure as an unskippable fixed computational layer. This ensures that the predicted trajectory strictly follows Newton's second law at the fundamental physics level, fundamentally avoiding the problem of non-physically reasonable trajectories generated by pure data-driven models during long-term extrapolation. Furthermore, since the sequence encoder extracts temporal features from historical trajectories, the residual prediction head only needs to learn the residual acceleration caused by complex non-ideal factors such as air resistance, lift, and rotational effects, rather than learning the full acceleration from zero. The network can achieve accurate compensation for complex aerodynamic disturbances with fewer parameters and lower data requirements. It is evident that the combined effect of the aforementioned physical hard constraint architecture and residual learning mechanism enables the embodied agent to achieve a prediction accuracy of less than 0.1 meters in landing point error when facing projectiles with complex aerodynamic disturbances, while maintaining the physical rationality of the predicted trajectory. This provides a reliable trajectory prediction basis for physical interaction tasks such as dynamic target interception and obstacle avoidance, thereby effectively supporting the embodied agent to achieve accurate and stable physical interaction in an open working environment. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a projectile trajectory prediction method based on embodied intelligent physical model fusion, provided in an embodiment of the present invention; Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0023] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0024] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0025] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0026] Please see Figure 1This invention provides a projectile trajectory prediction method based on the fusion of an embodied intelligent physical model. This method embeds Newton's equations of motion as an unskippable hard constraint into a neural network prediction architecture. This allows the network to learn only the residual acceleration caused by non-ideal aerodynamic effects that are difficult to model accurately using traditional physical models. This ensures the physical rationality of the predicted trajectory while effectively compensating for complex aerodynamic factors. Specifically, the method includes the following steps: S1, Obtain the historical trajectory sequence of the projectile, the historical trajectory sequence containing the position information and velocity information of the projectile at each of multiple consecutive historical moments.

[0027] In specific implementation, the executing entity of the method described in this embodiment can be an embodied intelligent agent, which is a physical entity device with autonomous perception, decision-making, and motion control capabilities, such as a humanoid robot, an autonomous mobile operation platform, a robotic arm system, or other intelligent devices capable of performing interactive tasks in a physical environment. This embodied intelligent agent is equipped with an onboard vision sensor and an onboard computing unit. The onboard vision sensor is used to acquire environmental images in real time; it can be a red-green-blue depth camera (RGB-D camera), a high-speed monocular camera, or a binocular stereo vision camera. The onboard computing unit is used to run neural network models and perform numerical calculations; it can be an embedded graphics processor, a neural network processing unit, or a field-programmable gate array (FPGA) or other hardware platform with parallel computing capabilities, which is not specifically limited in this invention.

[0028] First, the embodied agent acquires a historical trajectory sequence of the projectile, which includes the position and velocity information of the projectile at multiple consecutive historical moments. The projectile can be a tool actively thrown by a robotic arm, an accidentally fallen workpiece, a flying sphere, or other target objects in free flight. In practice, the embodied agent's onboard vision sensor continuously acquires environmental images at a fixed sampling frequency, such as 30 frames per second or 60 frames per second. For each frame of the acquired image, the onboard computing unit runs a target detection algorithm to identify the projectile in the image and determine its image position. Further, the target detection algorithm can employ a deep learning-based target detection model, such as a single-frame detector, a region convolutional neural network, or a candidate region-based target detection algorithm; this invention is not specifically limited in this regard. The target detection model outputs a rectangular detection bounding box, defined by the center point coordinates, width, and height. The center point of the detection bounding box of the projectile detected in each frame is determined as the image position of the projectile in that frame.

[0029] Furthermore, since the projectile's position may change significantly between consecutive frames, and other moving objects or noise interference may exist in the image, cross-frame association matching is required for the projectiles detected in each frame to determine whether the detection results in different frames correspond to the same projectile. Further, cross-frame association matching can be implemented using a Kalman filter combined with nearest neighbor matching: first, the Kalman filter predicts the projectile's position in the current frame based on the detection position in the previous frame; then, the actual detected image position in the current frame is compared with the predicted position. If the Euclidean distance between the two is less than a preset association threshold, the detection result is determined to correspond to the same projectile as the detection result in the previous frame. Further, if a projectile is not detected in a certain frame, an interpolation method based on a motion model can be used to complete the image position of that frame to ensure the continuity of the trajectory sequence. Further, through frame-by-frame association, all detection results belonging to the same projectile are connected in chronological order to form a sequence of image positions of the same projectile in each frame.

[0030] Furthermore, each image position in the image position sequence is a two-dimensional positional information represented in pixel coordinates within the image coordinate system. To be used for subsequent physical model calculations, this position needs to be mapped to the world coordinate system. This mapping process is further implemented using the pre-calibrated intrinsic and extrinsic parameter matrices of the airborne vision sensor. The intrinsic parameter matrix describes the sensor's inherent optical parameters such as focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the sensor's position and orientation in the world coordinate system. For each image position, combined with the corresponding depth value, an inverse projection transformation is used to map it from the image coordinate system to the world coordinate system, obtaining the world coordinate position for each frame. The depth value can be derived from depth data directly measured by a depth camera or from a depth estimate recovered from a monocular image based on multi-view geometric methods. This world coordinate position is a three-dimensional vector containing coordinate values ​​along the X, Y, and Z axes of the world coordinate system. This three-dimensional vector serves as the positional information of the projectile in that frame.

[0031] Furthermore, after obtaining the position information corresponding to each frame, the position information of adjacent frames is differentially calculated to obtain velocity information. Specifically, for two adjacent image frames, the position information of the subsequent frame is subtracted from the position information of the preceding frame to obtain a position difference vector. Then, each component of this position difference vector is divided by the time interval between adjacent frames to obtain the velocity information corresponding to the subsequent frame. This time interval is determined by the frame rate of the visual sensor; for example, when the frame rate is 30 frames per second, the time interval is approximately 33.3 milliseconds. Further, through the above differential calculation, each frame corresponds to a three-dimensional velocity information, which contains velocity components along the three axes of the world coordinate system. For the first frame in the sequence, since there is no preceding frame, its velocity information can be obtained using backward differential or directly set to zero.

[0032] Furthermore, the position and velocity information corresponding to multiple consecutive historical frames are assembled in chronological order from early to late to construct a historical trajectory sequence within a fixed time window. The length of this historical trajectory sequence, i.e., the number of historical frames it contains, can be preset according to the actual application scenario; for example, it can contain data from the past 10 to 30 frames. This historical trajectory sequence serves as input data for subsequent prediction processes.

[0033] In some preferred embodiments, obtaining the historical trajectory sequence of the projectile includes: acquiring continuous frame images collected by an airborne vision sensor; performing target detection on the projectile in the continuous frame images, determining the detection bounding box of the projectile in each frame, and taking the center point of the detection bounding box in each frame as the image position of the projectile in that frame; performing cross-frame association matching on the projectiles detected between frames, associating the detection results belonging to the same projectile according to the frame order to obtain the image position sequence of the same projectile in each frame; mapping each image position in the image position sequence to the world coordinate system to obtain the world coordinate position corresponding to each frame, as the position information corresponding to each frame; performing differential calculation on the position information of adjacent frames, dividing the differential result by the time interval between adjacent frames to obtain the velocity information corresponding to each frame; and assembling the position information and velocity information corresponding to each of multiple consecutive historical frames in chronological order to obtain the historical trajectory sequence.

[0034] In practice, the embodied intelligent agent first acquires continuous frame images using its onboard visual sensor. The onboard visual sensor continuously acquires environmental images at a fixed frame rate, such as 30 frames per second or 60 frames per second. Each frame is a two-dimensional digital image, composed of a pixel matrix of multiple pixels, each pixel carrying color information for three channels: red, green, and blue. The frames are arranged in chronological order of acquisition and stored in the cache of the onboard computing unit. Each frame carries a corresponding timestamp for subsequent temporal correlation and speed calculation.

[0035] Furthermore, target detection is performed on projectiles in consecutive frame images to determine the detection bounding box of the projectile in each frame, and the center point of the detection bounding box in each frame is taken as the image position of the projectile in that frame. In specific implementation, the onboard computing unit runs a pre-trained target detection neural network. This target detection neural network adopts a target detection architecture with a convolutional neural network as the backbone, such as the YOLO series network or a single-pass multi-box detector. Further, the target detection network takes a single frame image as input, and after a series of processes including image feature extraction by convolutional layers, candidate target region generation by a candidate region generation network, and detection results output by a classification and regression head, it outputs the detection results of all candidate regions in the image that may contain projectiles. Further, for each candidate region, the target detection network outputs a rectangular detection bounding box and a confidence score. The rectangular detection bounding box is defined by four parameters: the x-coordinate of the center point, the y-coordinate of the center point, the width, and the height. Further, the confidence score represents the reliability of the detection result, with a value ranging from 0 to 1. By setting a confidence threshold (e.g., 0.5), detection results with confidence scores higher than this threshold are filtered out. Then, the detection result with the highest confidence score is selected from all detection results higher than the threshold as the final detection result for that frame. The corresponding detection bounding box is the detection bounding box of the projectile in that frame. The geometric center point of this detection bounding box is taken as the image position of the projectile in that frame, and this image position is represented in pixel coordinates in the image coordinate system.

[0036] Furthermore, cross-frame association matching is performed on the projectiles detected in each frame, associating the detection results belonging to the same projectile in frame order to obtain the image position sequence of the same projectile in each frame. The purpose of cross-frame association matching is to solve the correspondence problem of target detection between consecutive frames, that is, to correctly associate the positions of projectiles detected in different frames with the same physical target. Further, in one specific implementation, cross-frame association matching is implemented using a Kalman filter combined with the Hungarian algorithm. The Kalman filter is an optimal state estimation algorithm used for state prediction and updating of linear Gaussian systems. For projectile tracking scenarios, the state vector is defined as a four-dimensional vector containing image position and image velocity. The prediction step of the Kalman filter predicts the state of the projectile in the current frame based on the kinematic equations, and the update step corrects the predicted state using the target detection results of the current frame. Further, after obtaining the detection results of each frame, the Hungarian algorithm is used to solve for the optimal match between the detection results and existing trajectories. The Hungarian algorithm achieves globally optimal association by constructing a cost matrix and assigning the detection results to the existing trajectory with the minimum cost. Through the above association matching, all detection results belonging to the same projectile are connected in chronological order to form an image position sequence of the same projectile in each frame.

[0037] Furthermore, each image position in the image position sequence is mapped to the world coordinate system to obtain the world coordinate position corresponding to each frame. This mapping process is implemented using the pre-calibrated intrinsic and extrinsic parameter matrices of the visual sensor. The intrinsic parameter matrix describes the inherent parameters of the visual sensor, such as focal length, principal point coordinates, and distortion coefficients, while the extrinsic parameter matrix describes the spatial position and attitude angle of the visual sensor in the world coordinate system. According to the pinhole camera imaging model, the mapping relationship between pixel coordinates in the image coordinate system and three-dimensional spatial coordinates in the world coordinate system is described by the projection equation. Further, for depth cameras, the corresponding depth value can be directly obtained for each pixel; for monocular cameras, the depth value can be obtained through motion reconstruction algorithms or deep learning-based monocular depth estimation methods. Combining the depth value, the pixel coordinates are converted into three-dimensional coordinates in the world coordinate system through inverse projection transformation. Specifically, the pixel coordinates and depth value are substituted into the inverse matrix of the camera projection matrix for calculation to obtain the three-dimensional coordinates of that point in the world coordinate system. These three-dimensional coordinates serve as the position information of the projectile in that frame.

[0038] Furthermore, the position information of adjacent frames is differentially calculated, and the difference result is divided by the time interval between adjacent frames to obtain the velocity information corresponding to each frame. In the specific implementation, for frame t and frame (t+1), the position information of frame (t) is subtracted from the position information of frame (t+1) to obtain a position difference vector. Then, the three components of this position difference vector are divided by the time interval between adjacent frames, which is equal to the reciprocal of the frame rate. For example, when the frame rate is 30 frames per second, this time interval is approximately 33.3 milliseconds. Further, the velocity information of frame (t+1) is obtained through the above calculation. Differential calculation is performed on all adjacent frames in the same way, so that each frame in the sequence except the first frame corresponds to a velocity information. For the first frame, since there is no previous frame, its velocity information can be obtained by backward difference, i.e., by using the position difference between the first and second frames, or it can be directly set as a zero vector.

[0039] Furthermore, the position and velocity information corresponding to multiple consecutive historical frames are assembled in chronological order to obtain a historical trajectory sequence. In the specific implementation, starting from the current moment, a preset number of historical frames are traced back, and the position and velocity information of each frame is extracted. The number of historical frames extracted can be set according to application requirements, for example, 10, 20, or 30 frames from the past. Further, the extracted position and velocity information of each frame is arranged in chronological order from earliest to latest, forming a historical trajectory sequence. The data structure of this historical trajectory sequence can be a three-dimensional tensor, with its dimension being the number of historical frames multiplied by the state dimension at each moment. This historical trajectory sequence is used as input data for subsequent prediction processes.

[0040] This embodiment utilizes an airborne vision sensor to achieve end-to-end visual perception from raw image acquisition to trajectory assembly in the world coordinate system, ensuring that the trajectory data required for subsequent physical model fusion has accurate physical dimensions and spatial coordinate references. Furthermore, cross-frame correlation matching guarantees continuous tracking of the same physical target in time, and coordinate mapping and differential calculation transform image pixel information into physically meaningful spatial position and velocity information, providing a data foundation for physical model constraints.

[0041] S2, input the historical trajectory sequence into the sequence encoder for temporal feature extraction to obtain the context feature vector.

[0042] In practice, the onboard computing unit of the embodied intelligent agent inputs the historical trajectory sequence into a sequence encoder for temporal feature extraction, obtaining a context feature vector. The sequence encoder is a pre-trained neural network model whose function is to extract features and model the context of time-series data. Specifically, the sequence encoder concatenates the position and velocity information at each moment in the historical trajectory sequence to form a single-step state vector, and then inputs each single-step state vector into the encoder network in chronological order. The encoder network captures the inherent laws governing the evolution of the projectile's motion state over time through its internal recurrent or attention structures, extracting temporal dependencies and motion pattern features from the historical trajectory. Further, after encoding, the motion information within a fixed-length time window is compressed and encoded into a fixed-dimensional context feature vector. This context feature vector is a dense numerical vector containing comprehensive information such as the projectile's motion pattern, velocity change trend, acceleration characteristics, and trajectory shape over a past period, providing sufficient contextual basis for subsequent residual acceleration prediction.

[0043] In some preferred embodiments, the step of inputting the historical trajectory sequence into a sequence encoder for temporal feature extraction to obtain a context feature vector includes: concatenating the position and velocity information of each moment in the historical trajectory sequence into a single-step state vector to obtain a state vector sequence; inputting the state vector sequence into the sequence encoder, which is an LSTM network, wherein the LSTM network performs gated recursive processing on each single-step state vector in the state vector sequence through its forget gate, input gate, and output gate to update the hidden state and cell state of the LSTM network; and using the hidden state output by the LSTM network at the last time step as the context feature vector; or, inputting the state vector sequence into the sequence encoder, which is a Transformer network, wherein each single-step state vector in the state vector sequence is position-encoded and then input into the encoder of the Transformer network, wherein the encoder of the Transformer network models the temporal dependency between each single-step state vector through a multi-head self-attention mechanism and outputs the encoded feature vector corresponding to each time step; and performing pooling processing on the encoded feature vector corresponding to each time step, and using the aggregated feature vector obtained after pooling as the context feature vector.

[0044] In specific implementation, this embodiment provides two optional sequence encoder implementation architectures: an implementation based on a long short-term memory network and an implementation based on a Transformer network, which are described below.

[0045] In the first optional implementation, the sequence encoder employs a Long Short-Term Memory (LSTM) network. LSM is a special type of recurrent neural network structure that effectively solves the gradient vanishing and gradient exploding problems inherent in traditional recurrent neural networks when processing long sequences by introducing a gating mechanism, enabling it to effectively capture and remember long-distance temporal dependencies. The core computational unit of the LSM network consists of four parts: a forget gate, an input gate, an output gate, and a cell state. The cell state is the core memory unit of the LSM network, running throughout the entire sequence processing process, responsible for storing and transmitting long-term information. The forget gate controls which information is discarded from the cell state at the previous time step; its output value is between 0 and 1, where 0 indicates complete discard and 1 indicates complete retention. The input gate controls which new information is stored in the cell state at the current time step. The output gate controls how the hidden state at the current time step is extracted from the updated cell state.

[0046] In the specific implementation, the position and velocity information at each moment in the historical trajectory sequence are first concatenated. Specifically, the position information at moment t is a three-dimensional vector, and the velocity information is a three-dimensional vector. These two vectors are concatenated end-to-end to form a six-dimensional single-step state vector. Then, the single-step state vectors are arranged in chronological order to obtain a state vector sequence.

[0047] Furthermore, the state vector sequence is input into the Long Short-Term Memory (LSTM) network. The LSM network processes each single-step state vector in the state vector sequence sequentially, step by step. At each time step, the LSM network receives the current input state vector, the hidden state from the previous time step, and the cell state from the previous time step, and performs gating computation. The forget gate calculates its activation value based on the current input and the previous hidden state; this value controls the proportion of forgotten information. The input gate calculates its activation value based on the current input and the previous hidden state, and simultaneously calculates candidate cell state values. The cell state is then updated by multiplying the forget gate activation value by the cell state from the previous time step, and adding the product of the input gate activation value and the candidate cell state value. The output gate calculates its activation value based on the current input and the previous hidden state, and then calculates the current hidden state, i.e., the product of the output gate activation value and the cell state mapped using the hyperbolic tangent function. Furthermore, through the above recursive gating process, the LSM network updates its hidden state and cell state step by step. After processing the last single-step state vector in the state vector sequence, the hidden state output by the Long Short-Term Memory (LSTM) network contains the temporal context information of the entire historical trajectory sequence. Furthermore, the hidden state output at this last time step is used as the context feature vector. The dimension of this context feature vector is equal to the dimension of the LSM network's hidden state, for example, it can be set to 128, 256, or 512 dimensions.

[0048] In the second alternative implementation, the sequence encoder employs a Transformer network. The Transformer network is a sequence modeling architecture entirely based on a self-attention mechanism. It abandons the recurrent structure and directly models the dependency between any two positions in the sequence through the attention mechanism. First, the position and velocity information at each moment in the historical trajectory sequence are concatenated into a single-step state vector, and the state vector sequence is obtained in the same way as described above.

[0049] Furthermore, positional encoding is performed on each single-step state vector in the state vector sequence. The purpose of positional encoding is to provide the Transformer network with the positional information of each single-step state vector in the time series, because the Transformer network's self-attention mechanism itself does not have the ability to perceive sequence order. In one specific implementation, a sine-cosine positional encoding method is used. The generated positional encoded vector is added to the corresponding single-step state vector to obtain the positionally encoded input vector. All positionally encoded input vectors are stacked in time order to form a two-dimensional matrix, which serves as the input to the Transformer encoder.

[0050] Furthermore, the Transformer encoder comprises a multi-head self-attention layer and a feedforward neural network layer. The multi-head self-attention layer first maps the input matrix into a query matrix, a key matrix, and a value matrix using three linear transformation matrices. Then, it calculates an attention score matrix, where each element represents the attention weight between corresponding positions. Multiplying the attention score matrix by the value matrix yields the output of the self-attention layer. The multi-head self-attention mechanism executes multiple different attention heads in parallel, each with independent linear transformation parameters. The outputs of these attention heads are then concatenated and linearly transformed to obtain the final output of the multi-head self-attention layer. Further, the feedforward neural network layer performs a non-linear transformation on the output of the self-attention layer, typically consisting of two linear transformation layers and a non-linear activation function. The encoder's output is the encoded feature vector corresponding to each time step, with each feature vector incorporating information from all positions throughout the sequence.

[0051] Furthermore, to obtain a fixed-length context feature vector, pooling is performed on the encoded feature vectors corresponding to each time step. Pooling can be performed using average pooling, which calculates the average of the encoded feature vectors across all time steps; or max pooling, which takes the maximum value across all time steps in each dimension. The aggregated feature vector obtained after pooling is then used as the context feature vector.

[0052] Furthermore, regardless of whether a long short-term memory network or a Transformer network is used to implement the sequence encoder, the final output context feature vector is a dense vector of fixed dimensions. This vector fully condenses the temporal information and motion pattern features in the historical trajectory sequence, providing high-quality input for the subsequent residual prediction head.

[0053] This embodiment provides two optional sequence encoder architectures: Long Short-Term Memory (LSTM) networks and Transformer networks. These are suitable for historical trajectory sequences of varying lengths and different computational resource constraints, respectively. LSM networks effectively capture long-distance temporal dependencies through gating mechanisms, making them suitable for processing medium-length trajectory sequences. Furthermore, Transformer networks directly model global dependencies through self-attention mechanisms, making them suitable for processing longer trajectory sequences and offering higher parallel computation efficiency. The complementary design of these two architectures allows this method to flexibly adapt to different embodied intelligent agents' hardware platforms and real-time requirements.

[0054] S3, input the context feature vector into the residual prediction head to perform acceleration residual prediction, and obtain the residual acceleration vector. The residual acceleration vector is used to characterize the acceleration deviation caused by non-ideal aerodynamic effects.

[0055] In practice, the context feature vector is input into the residual prediction head to predict the acceleration residual, resulting in a residual acceleration vector. The residual prediction head is a neural network module connected to the sequence encoder. Its input is the context feature vector output by the sequence encoder, and its output is a three-dimensional residual acceleration vector. The residual prediction head learns the complex mapping relationship between historical motion patterns and current acceleration deviations by performing multi-layer nonlinear mapping on the context feature vector. This residual acceleration vector is used to characterize the acceleration deviation caused by non-ideal aerodynamic effects, specifically including the hindering effect of air resistance on the projectile's flight trajectory, the disturbance of lift on the projectile's vertical motion, and the lateral offset caused by the Magnus effect generated by the projectile's own rotation—complex factors that traditional pure physical models cannot accurately model. Furthermore, the three-dimensional vector output by the residual prediction head has three components corresponding to acceleration deviation values ​​in the three orthogonal axes of the world coordinate system, with explicit physical units (meters per second squared) and physical meaning.

[0056] In some preferred embodiments, the step of inputting the context feature vector into the residual prediction head for acceleration residual prediction to obtain the residual acceleration vector includes: inputting the context feature vector into the residual prediction head, wherein the residual prediction head is a multilayer fully connected network, the multilayer fully connected network comprising an input layer, at least one hidden layer and an output layer connected in sequence, the hidden layer employing the ReLU activation function, the output layer employing a linear activation function, the input layer receiving the context feature vector, performing a linear transformation on the weight matrix and bias vector of each hidden layer and then performing a nonlinear mapping on the ReLU activation function, and the output layer outputting a three-dimensional vector, which is used as the residual acceleration vector, wherein the three components of the residual acceleration vector correspond to the acceleration deviation values ​​in the three orthogonal axis directions in the world coordinate system.

[0057] In practice, the residual prediction head adopts a multi-layer fully connected network structure, which is a basic form of feedforward neural network. Its characteristic is that all neurons in adjacent layers are interconnected. The multi-layer fully connected network consists of an input layer, at least one hidden layer, and an output layer connected in sequence. The input layer receives external input data, the hidden layer performs feature transformation and abstraction, and the output layer produces the final prediction result.

[0058] Furthermore, the number of neurons in the input layer is the same as the dimension of the context feature vector. The context feature vector is a fixed-dimensional numerical vector, the dimension of which is determined by the output dimension of the sequence encoder. For example, if the sequence encoder is a Long Short-Term Memory network and the hidden state dimension is 256, then the dimension of the context feature vector is 256. Each neuron in the input layer corresponds to a component of the context feature vector, receiving the value of that component as input.

[0059] Furthermore, the input layer is fully connected to the first hidden layer, meaning each neuron in the input layer is connected to every neuron in the first hidden layer. Each connection is assigned a trainable weight parameter, and each hidden layer neuron also has a trainable bias parameter. After receiving the output from the previous layer, the hidden layer performs two operations: a linear transformation and a nonlinear mapping. First, it calculates the product of the previous layer's output vector and the current layer's weight matrix, then adds the current layer's bias vector to obtain the linear transformation result. This linear transformation result is then input into a nonlinear activation function for nonlinear mapping, yielding the hidden layer's output vector. Multiple hidden layers are connected sequentially, with the input of each subsequent hidden layer being the output of the previous one, thus achieving hierarchical abstraction and progressive transformation of features.

[0060] Furthermore, the hidden layers employ a modified linear unit activation function (MRU). This activation function is defined as follows: when the input value is positive, the output equals the input value; when the input value is negative or zero, the output equals zero. The MRU has advantages such as computational simplicity, effective mitigation of the vanishing gradient problem, and accelerated network convergence, and is widely used in the hidden layers of deep neural networks. The number of hidden layers can be configured according to the actual application scenario and the computing resources of the onboard computing unit; for example, two or three hidden layers can be set. The number of neurons in each hidden layer can be the same or different; for example, it can decrease layer by layer for feature dimensionality reduction or increase layer by layer for feature expansion. In one specific implementation, two hidden layers are set: the first hidden layer contains 128 neurons, and the second hidden layer contains 64 neurons.

[0061] Furthermore, the last hidden layer is also fully connected to the output layer. The output layer has three neurons, corresponding to the three orthogonal axes in the world coordinate system: the X, Y, and Z axes. The output layer uses a linear activation function, meaning no nonlinear mapping is applied to the linear transformation result. The expression for the linear activation function is that its output is directly equal to its input; that is, it directly outputs the linear transformation result as the output vector. The reason for using a linear activation function is that the components of the residual acceleration vector need to take values ​​freely within a positive and negative range to flexibly represent acceleration deviations caused by non-ideal aerodynamic effects of various directions and magnitudes. Furthermore, using activation functions with range limitations, such as sigmoid or tanh, would restrict the value space of the residual acceleration, reducing the network's expressive power.

[0062] Furthermore, in the specific operation, the input layer receives the context feature vector output by the sequence encoder, which is then passed to the first hidden layer. The first hidden layer performs a linear transformation and corrects the nonlinear mapping of the linear unit on the input vector, before passing the result to the second hidden layer. The second hidden layer continues with the linear transformation and corrects the nonlinear mapping of the linear unit, passing the result to the output layer. The output layer performs a linear transformation on the received vector, outputting a three-dimensional vector. This three-dimensional vector is used as the residual acceleration vector, with its three components corresponding to the acceleration deviation values ​​along the X, Y, and Z axes in the world coordinate system, respectively.

[0063] In this embodiment, the residual acceleration vector output by the residual prediction head has clear physical dimensions and meanings, rather than the abstract hidden layer features that are uninterpretable in traditional neural networks. This design allows the intermediate results output by the network to be directly understood by human engineers, facilitating qualitative and quantitative analysis of the prediction results. For example, it allows analysis of which component of the residual acceleration dominates in a specific scenario, thereby determining the type of dominant non-ideal factor in that scenario.

[0064] This embodiment maps abstract contextual features into three-dimensional residual acceleration vectors with explicit physical dimensions through a multi-layer fully connected network, making the network output physically interpretable. The output layer employs a linear activation function to ensure that each component of the residual acceleration vector can freely take values ​​within a positive and negative range, flexibly adapting to aerodynamic effect deviations of various directions and amplitudes. Furthermore, the physically meaningful intermediate outputs facilitate engineers' analysis of the dominant non-ideal factors in specific scenarios, enhancing the model's debuggability and reliability.

[0065] S4. Add the residual acceleration vector to the preset gravitational acceleration constant to obtain the total acceleration vector.

[0066] In practice, the residual acceleration vector is added to a preset gravitational acceleration constant to obtain the total acceleration vector. In this step, the onboard computing unit performs a mandatory physical model fusion calculation. Specifically, the onboard computing unit reads the pre-stored gravitational acceleration constant from its internal memory. The gravitational acceleration constant is a known physical constant, vertically downwards, with a magnitude of 9.8 meters per second squared. The three components of the residual acceleration vector represent the acceleration deviations along the X, Y, and Z axes of the world coordinate system, respectively. Further, the three components of the residual acceleration vector are added to the corresponding components of the gravitational acceleration constant along the three axes of the world coordinate system to obtain the three components of the total acceleration vector. This addition operation is mandatory; the network output must pass through this physical addition layer before proceeding to subsequent steps. The network cannot bypass this physical model embedding layer. Through this mandatory operation, Newton's equations of motion, as fundamental physical laws, are rigidly embedded into the network architecture, structurally ensuring the rationality of the prediction results at the fundamental physical level.

[0067] S5, the total acceleration vector is input into the differentiable kinematics layer, and the position and velocity information corresponding to the last historical moment in the historical trajectory sequence are used as the initial state. The kinematic equations are numerically integrated to obtain the predicted future trajectory sequence, which contains the position and velocity information corresponding to each of the multiple future moments.

[0068] In practice, the total acceleration vector is input into the differentiable kinematics layer. Using the position and velocity information corresponding to the last historical moment in the historical trajectory sequence as the initial state, the kinematic equations are numerically integrated to obtain the predicted future trajectory sequence. Furthermore, the differentiable kinematics layer is a differentiable numerical integration module that internally implements the discretization solution of the kinematic equations. Before starting the integration, the initial conditions of the kinematic equations are first constructed: the position information of the last historical moment in the historical trajectory sequence is used as the initial position, and the velocity information at that moment is used as the initial velocity. Further, the kinematic equations contain two first-order ordinary differential equations: the first equation states that the first derivative of position with respect to time equals velocity, and the second equation states that the first derivative of velocity with respect to time equals acceleration, where the acceleration term is the aforementioned total acceleration vector. Further, after substituting the total acceleration vector into the second equation, the kinematic equations constitute a closed system concerning the evolution of state variables (including position and velocity) over time.

[0069] Furthermore, to solve the kinematic equations in discrete time steps, the differentiable kinematic layer employs numerical integration to discretize the continuous-time equations. In the specific implementation, starting from the initial moment, the solution is progressively advanced according to a preset time step. A state update is calculated once within each time step, yielding the predicted position and velocity at that moment. The integration step size can be set according to accuracy requirements, for example, 0.02 seconds or 0.05 seconds. Integration calculations are repeated until a preset prediction time domain length is reached, for example, predicting the trajectory within the next 2 seconds. Further, the positions and velocities output at each integration step are arranged in chronological order to construct a sequence of predicted future trajectories. This sequence of predicted future trajectories contains the position and velocity information of the projectile at multiple future moments.

[0070] Furthermore, since all operations in the differentiable kinematics layer are differentiable, including addition and numerical integration, the entire network architecture forms a fully differentiable computational chain from input to output. This means that during subsequent training, prediction errors can be propagated back from the output layer to the input layer by layer through backpropagation, achieving end-to-end joint optimization of network parameters. Furthermore, because the kinematic equations use the total acceleration vector obtained after forced addition, the predicted trajectory strictly follows Newton's laws of motion at the fundamental physics level, avoiding energy divergence or non-physical trajectory patterns common in purely data-driven models.

[0071] This embodiment forcibly adds the residual acceleration vector to the gravitational acceleration constant before inputting it into a differentiable kinematics layer for integration. This embeds Newton's equations of motion as a fixed, non-skippable computational layer into the network structure, ensuring that any predicted trajectory strictly adheres to Newton's second law at the fundamental physics level. This fundamentally eliminates the problem of physically unreasonable trajectories generated by purely data-driven models during long-term extrapolation. Furthermore, since the network only needs to learn the residual acceleration rather than the full acceleration, the number of network parameters is small and the training data requirements are low, achieving accurate compensation for complex aerodynamic effects while maintaining physical plausibility.

[0072] In some preferred embodiments, the step of inputting the total acceleration vector into a differentiable kinematics layer, using the position and velocity information corresponding to the last historical moment in the historical trajectory sequence as the initial state, and numerically integrating the kinematic equations to obtain the predicted future trajectory sequence includes: using the position information of the last historical moment in the historical trajectory sequence as the initial position, and the velocity information of the last historical moment in the historical trajectory sequence as the initial velocity, constructing initial conditions for a system of kinematic equations, wherein the system of kinematic equations includes the first derivative of position with respect to time equal to velocity, and the first derivative of velocity with respect to time equal to the total acceleration vector; and using the total acceleration vector as the acceleration term in the system of kinematic equations. A fourth-order Runge-Kutta integrator is used for numerical integration. Within each integration step, the fourth-order Runge-Kutta integrator calculates the first derivative estimate at the starting point of the integration step, the second derivative estimate at half the integration step, the third derivative estimate, and the fourth derivative estimate at the ending point of the integration step. The first derivative estimate, the second derivative estimate, the third derivative estimate, and the fourth derivative estimate are weighted and averaged to obtain the final slope of the integration step. The position and velocity of the current step are updated based on the final slope. Numerical integration is repeated until a preset prediction time domain length is reached. The position and velocity output of each integration step are arranged in chronological order to obtain the predicted future trajectory sequence.

[0073] In practice, the differentiable kinematics layer uses a fourth-order Runge-Kutta integrator as the numerical integration tool. The fourth-order Runge-Kutta integrator is a commonly used high-precision single-step method in the numerical solution of ordinary differential equations. It achieves a much higher accuracy than the simple Euler method by calculating the derivative estimates at four different positions within each integration step and performing a weighted average.

[0074] Before starting integration, initial conditions for the kinematic equations must first be constructed. Differentiable kinematic layers obtain the position and velocity information of the last historical moment in the historical trajectory sequence. This position information is used as the initial position of the kinematic equations, and the velocity information is used as the initial velocity. The initial position and initial velocity together constitute the complete state description of the kinematic equations at the initial moment.

[0075] Furthermore, the kinematic equations consist of two first-order ordinary differential equations. The first equation describes the rate of change of position with time, i.e., the first derivative of position with respect to time equals velocity. The second equation describes the rate of change of velocity with time, i.e., the first derivative of velocity with respect to time equals acceleration, where the acceleration term is the aforementioned total acceleration vector. Substituting the total acceleration vector as the acceleration term into the second equation, the kinematic equations become a closed differential equation system concerning the evolution of the state vector over time.

[0076] Furthermore, to solve the system of differential equations in discrete time steps, the fourth-order Runge-Kutta integrator advances stepwise in fixed time steps, starting from the initial time. Assume the state vector is known at the beginning of the current integration step. The fourth-order Runge-Kutta integrator performs the following four evaluation processes within this step.

[0077] Furthermore, the initial evaluation is conducted at the starting point. The rate of change of velocity and the rate of change of position at the starting point are calculated. The rate of change of velocity is equal to the total acceleration vector at the current moment; the rate of change of position is equal to the velocity vector at the current moment.

[0078] Furthermore, the second evaluation is performed at the halfway point of the step. First, the state vector is advanced by half a step from the starting point using the derivative estimate obtained from the first evaluation, resulting in the first intermediate state. In this first intermediate state, the rate of change of velocity and the rate of change of position are calculated. The rate of change of velocity is equal to the total acceleration vector evaluated in this intermediate state, and the rate of change of position is equal to the velocity vector in this intermediate state.

[0079] Furthermore, the third evaluation is also performed at the halfway point of the step, but the state is advanced again using the derivative estimate obtained from the second evaluation. Specifically, the state vector is advanced by half a step from the starting point using the derivative estimate obtained from the second evaluation, resulting in the second intermediate state. In this second intermediate state, the rate of change of velocity and the rate of change of position are calculated.

[0080] Furthermore, the fourth evaluation is performed at the end of the step. Using the derivative estimate obtained from the third evaluation, the state vector is advanced from the starting point for the entire step length to obtain the final state estimate. In this final state, the rate of change of velocity and the rate of change of position are calculated.

[0081] Furthermore, after obtaining the four derivative estimates, the fourth-order Runge-Kutta integrator performs a weighted average of these four estimates to obtain the final slope for the integration step size. The weighting method is as follows: the first and fourth derivative estimates have a weight of one-sixth, and the second and third derivative estimates have a weight of two-sixths. Then, the state of the current step is updated according to the final slope: the new position is equal to the current position plus the weighted average of the rate of change of position multiplied by the step size, and the new velocity is equal to the current velocity plus the weighted average of the rate of change of velocity multiplied by the step size. This completes the state update for one integration step size.

[0082] Furthermore, after completing one integration step, the newly obtained position and velocity are used as the starting state for the next integration step, and the above fourth-order Runge-Kutta integration calculation process is repeated. Each integration step advances the process once, and for each step, the predicted position and velocity are output. Integration continues until the total time taken reaches the preset prediction time domain length. Furthermore, the prediction time domain length can be set according to actual application requirements. For example, for a robotic arm grasping task, it can be set to predict the trajectory within the next 1 second; for a collision avoidance task, it can be set to predict the trajectory within the next 2 or 3 seconds. The positions and velocities output from each integration step are arranged in chronological order to construct the predicted future trajectory sequence.

[0083] Furthermore, the single-step truncation error of the fourth-order Runge-Kutta integrator is of the same order as the fifth power of the step size, exhibiting high numerical accuracy. In practical applications, an appropriate integration step size can be selected based on the trade-off between accuracy and computational efficiency. For example, a step size of 0.01 seconds can be set for high accuracy requirements, while a step size of 0.02 seconds or 0.05 seconds can be set for scenarios with high real-time requirements.

[0084] Furthermore, a key characteristic of differentiable kinematics layers is that all their operations are differentiable. Each step of the addition, multiplication, and fourth-order Runge-Kutta integrator is differentiable, meaning the entire numerical integration process constitutes a differentiable computational graph. This characteristic allows the gradient of the loss function with respect to the network parameters to be precisely backpropagated through the differentiable kinematics layer to the residual predictor and sequence encoder during subsequent training, achieving end-to-end joint optimization training and avoiding the gradient blocking problem caused by non-differentiability in traditional numerical integration methods during the machine learning process.

[0085] This embodiment achieves high-precision numerical solutions to the kinematic equations using a fourth-order Runge-Kutta integrator. The single-step truncation error is of the same order as the fifth power of the step size, maintaining sufficient integration accuracy even with large step sizes. Furthermore, all operations in the differentiable kinematic layer are differentiable, allowing the integration error to be backpropagated end-to-end to the preceding network module, achieving global joint optimization and avoiding the cumulative error caused by staged optimization.

[0086] In some preferred embodiments, the method further includes a two-stage training step for the sequence encoder and the residual prediction head: Stage 1, pre-training the sequence encoder and the residual prediction head using a projectile motion dataset generated by a pure physics simulation engine. The pure physics simulation engine is configured only with the gravitational acceleration term and the simulation of air resistance and rotational effects is disabled, generating projectile trajectory data driven only by gravity. The projectile trajectory data is used as training samples to iteratively train the sequence encoder and the residual prediction head until they converge. The pure physics simulation engine is a computer software system that simulates motion based on physical laws. Stage 2, deploying the pre-trained sequence encoder and the residual prediction head to an embodied agent, and fine-tuning the pre-trained sequence encoder and the residual prediction head using projectile trajectory data collected by the embodied agent's onboard vision sensor in a real working environment. The learning rate in Stage 2 is lower than the learning rate in Stage 1.

[0087] In practice, the sequence encoder and residual prediction head, as neural network models, require a large number of trainable parameters to have their optimal values ​​determined through a data-driven training process. To ensure the network can effectively learn complex aerodynamic effects while maintaining physical plausibility, this embodiment provides a two-stage training strategy.

[0088] Phase one is the pre-training phase. In this phase, a large-scale projectile motion dataset is generated using a pure physics simulation engine. A pure physics simulation engine is a computer software system that simulates motion based on physical laws. It integrates a rigid body dynamics solver, capable of accurately calculating the trajectory of a projectile under ideal physical conditions based on set initial physical parameters. During the pre-training phase, the pure physics simulation engine is specially configured: only the gravitational acceleration term is enabled, set to 9.8 m / s², vertically downwards, while air resistance and rotational effect simulations are disabled. Under this configuration, the projectile trajectory data generated by the pure physics simulation engine completely follows Newton's laws of motion under ideal conditions, i.e., projectile motion driven solely by gravity, with a standard parabolic trajectory. This dataset does not contain any complex non-ideal aerodynamic effects, such as air resistance, lift, or the Magnus effect caused by rotation.

[0089] Furthermore, the pure physics simulation engine generates diverse training data by randomizing initial conditions. Specifically, it randomly generates the initial three-dimensional coordinates of the projectile's position, the magnitude and direction of the initial velocity (including different launch elevation and azimuth angles), and the projectile's mass parameters. By combining different initial conditions, the simulation engine can generate a large number of training samples with different trajectory patterns, covering various possible projectile motion scenarios. Each training sample contains a complete historical trajectory sequence and a corresponding real future trajectory sequence.

[0090] Furthermore, after obtaining a large-scale simulation dataset, the sequence encoder and residual prediction head are pre-trained using training samples from this dataset. In each training iteration, a batch of training samples is sampled from the dataset, and the historical trajectory sequence portion of the samples is input into the network for forward propagation. The forward propagation process is consistent with the inference process described in the previous embodiments: the sequence encoder extracts temporal features to obtain a context feature vector, the residual prediction head predicts the residual acceleration, the residual acceleration is added to the gravitational acceleration to obtain the total acceleration, and the differential kinematics layer integrates to obtain the predicted trajectory. Then, the error between the predicted trajectory and the true future trajectory in the samples is calculated, the gradient is calculated using the backpropagation algorithm, and the network parameters are updated using the gradient descent optimization algorithm. The above iterative process is repeated to gradually reduce the prediction error. Pre-training continues until the prediction error of the network on the validation set no longer decreases significantly, that is, the network reaches a convergent state. After pre-training, the network has learned the laws of projectile motion under ideal physical conditions, and it has established accurate dynamic priors about gravity-driven projectile motion.

[0091] Furthermore, Phase Two is the fine-tuning phase. In this phase, the pre-trained sequence encoder and residual prediction head are deployed to the embodied agent, and the pre-trained network is fine-tuned using projectile trajectory data collected by the embodied agent's onboard vision sensors in a real-world operating environment. Unlike purely physical simulation data, trajectory data collected in a real-world operating environment includes the combined effects of various non-ideal aerodynamic factors such as air resistance, lift, and rotation, reflecting the complex dynamic characteristics of the actual physical world.

[0092] Furthermore, during the real-world data acquisition process, the embodied agent performs projectile operations or observes projectile phenomena in the operational environment. Airborne vision sensors acquire image sequences in real time, and target detection and tracking algorithms extract the actual trajectory data of the projectile. Because acquiring real data is costly, the number of real trajectory samples collected is typically far less than that of simulation data. Therefore, a key design consideration in the fine-tuning stage is to achieve effective parameter adjustment under limited real-world data conditions, while avoiding disrupting the fundamental physical priors learned during the pre-training phase.

[0093] Furthermore, to achieve the above objectives, a lower learning rate is used in the fine-tuning phase compared to the pre-training phase. Specifically, the learning rate in the pre-training phase can be set to the order of one-thousandth, while the learning rate in the fine-tuning phase is reduced to the order of one-hundred-thousandth or one-millionth. This lower learning rate allows for smaller step sizes in parameter updates during fine-tuning, preventing excessive disruption or catastrophic forgetting of the physical prior knowledge learned in the pre-training phase when training with limited real data. Furthermore, during fine-tuning, the network architecture remains identical to that in the pre-training phase; the residual acceleration vector output by the residual prediction head is forcibly added to the gravitational acceleration constant before being integrated into the differentiable kinematics layer to obtain the predicted trajectory. Moreover, through fine-tuning with a low learning rate, the network's residual prediction head gradually learns accurate compensation for non-ideal aerodynamic effects in real-world scenarios, while maintaining the fundamental physical framework established in the pre-training phase.

[0094] Furthermore, through the above two-stage training strategy, the sequence encoder and residual prediction head obtain accurate basic dynamic knowledge from large-scale pure physical simulation data in the pre-training stage, and only need a small amount of real data in the fine-tuning stage to learn a residual compensation model for a specific working environment.

[0095] It should be further explained that in the aforementioned pre-training stage, since the pure physical simulation data only includes gravity-driven forces and does not include air resistance or rotational effects, the theoretical value of the residual acceleration is zero from the perspective of the physical model. Those skilled in the art may argue that the network cannot effectively learn residual prediction capabilities at this stage; however, its actual purpose is not to directly learn non-zero residuals, but rather to construct a basic physical prior through large-scale ideal trajectory data. This process has two aspects: first, the sequence encoder, starting from random parameters, learns to effectively extract motion temporal features through extensive trajectory training; this feature extraction capability is independent of the residual magnitude; second, after optimization, the output of the residual prediction head converges to near the zero vector, making the network's default state an ideal physical model "driven only by gravity." This default state itself constitutes a reasonable physical prior. Furthermore, in the subsequent fine-tuning stage, the real residual signal is usually small relative to gravitational acceleration. The residual prediction head starts from a near-optimal zero-residual state rather than a random starting point, and can complete the learning and adaptation from a small amount of signal using only a few real samples. If the pre-training stage is omitted, the network needs to learn both fundamental physics and residual compensation from random parameters simultaneously, which significantly reduces data efficiency and convergence speed. Therefore, the core of this two-stage decoupling strategy of "first establishing physical priors and then learning residual compensation" lies in pre-embedding physical laws as prior knowledge into the network, thereby reducing dependence on a large amount of real labeled data.

[0096] This embodiment employs a two-stage training strategy: first, pre-training with pure physical simulation, and then fine-tuning with real data. In the pre-training stage, the network establishes accurate physical dynamics priors, while in the fine-tuning stage, it is adapted using a small amount of real data with a low learning rate. This approach ensures the accuracy of the basic physical framework, significantly reduces the cost of real data acquisition, and effectively avoids destructive forgetting of pre-trained knowledge when fine-tuning with limited real data.

[0097] In some preferred embodiments, the single-iteration training step adopted in either stage one or stage two includes: acquiring training samples, the training samples containing historical trajectory sequence samples of the projectile and their corresponding true future trajectory sequence samples; inputting the historical trajectory sequence samples into the sequence encoder for temporal feature extraction to obtain a sample context feature vector; inputting the sample context feature vector into the residual prediction head for acceleration residual prediction to obtain a sample residual acceleration vector; adding the sample residual acceleration vector to the gravitational acceleration constant to obtain a sample total acceleration vector; inputting the sample total acceleration vector into the differentiable kinematics layer, using the position and velocity information of the last moment in the historical trajectory sequence samples as the initial state, performing numerical integration on the kinematic equations to obtain a sample predicted future trajectory sequence; calculating the values ​​of each moment in the sample predicted future trajectory sequence. The displacement error between the predicted position and the actual position at the corresponding moment in the real future trajectory sequence sample is calculated, and the average displacement error is obtained by averaging the displacement errors at all moments. The average displacement error is used as the main loss term. Based on the position and velocity information at each moment in the sample predicted future trajectory sequence, as well as the mass parameters of the projectile, the kinetic energy and potential energy at each moment are calculated. The difference between the sum of kinetic energy and potential energy at the end moment and the start moment in the sample predicted future trajectory sequence is calculated, and the portion of the absolute value of the difference that exceeds a preset energy deviation threshold is used as the energy deviation constraint loss term. The main loss term and the energy deviation constraint loss term are weighted and summed to obtain the total loss. The total loss is backpropagated layer by layer along the differentiable kinematics layer, the residual prediction head, and the sequence encoder using the backpropagation algorithm, and the parameters of the sequence encoder and the residual prediction head are updated synchronously.

[0098] In practice, a single iteration of training is a complete training process executed on a training batch, including three stages: forward propagation to calculate prediction results, calculation of various loss functions, and backpropagation to update network parameters. This process is repeated in each iteration of training during the pre-training and fine-tuning phases.

[0099] First, training samples are acquired. These samples consist of historical trajectory sequences of the projectile and their corresponding real future trajectory sequences. During the pre-training phase, the training samples come from a dataset generated by a pure physics simulation engine; during the fine-tuning phase, the training samples are derived from data collected by the embodied agent's onboard visual sensors in a real-world operating environment. In each training sample, the length of the historical trajectory sequence is consistent with the length of the historical trajectory sequence used in the prediction inference phase, and the length of the real future trajectory sequence is consistent with the prediction time domain length.

[0100] Furthermore, the historical trajectory sequence samples are input into a sequence encoder for temporal feature extraction to obtain a sample context feature vector. The specific processing flow of this step is consistent with the processing method of the sequence encoder in the aforementioned embodiments: the sequence encoder performs temporal modeling and feature abstraction on the position and velocity information in the historical trajectory sequence samples, compressing and encoding the motion information within a fixed time window into a fixed-dimensional sample context feature vector. This vector contains comprehensive information such as the projectile's motion pattern and velocity change trend within the sample's historical time period.

[0101] Furthermore, the sample context feature vector is input into the residual prediction head for acceleration residual prediction, resulting in a sample residual acceleration vector. The residual prediction head performs multi-layer fully connected network mapping on the sample context feature vector, outputting a three-dimensional sample residual acceleration vector, with the three components corresponding to the acceleration deviations in the three axes of the world coordinate system.

[0102] Furthermore, the sample residual acceleration vector is added to the gravitational acceleration constant to obtain the sample total acceleration vector. This addition operation is a mandatory physical model fusion step, and its processing method is consistent with that in the previous embodiment, that is, the three components of the residual acceleration vector are added to the corresponding components of the gravitational acceleration constant in the three axes of the world coordinate system.

[0103] Furthermore, the total acceleration vector of the sample is input into the differentiable kinematics layer. Using the position and velocity information of the last moment in the historical trajectory sequence sample as the initial state, the kinematic equations are numerically integrated to obtain the sample's predicted future trajectory sequence. This step is processed in the same way as the integration process of the differentiable kinematics layer in the aforementioned embodiment: using the state of the last moment of the sample's historical trajectory as the initial value, and the total acceleration vector of the sample as the acceleration term, numerical integration is performed through a fourth-order Runge-Kutta integrator to obtain the sample's predicted future trajectory sequence. This sequence contains the position and velocity information corresponding to each of the multiple predicted future moments of the sample.

[0104] Furthermore, the average displacement error is calculated as the main loss term. Specifically, for each time step in the predicted future trajectory sequence of the samples, the displacement error between the predicted position vector at that time and the actual position vector at the corresponding time step in the actual future trajectory sequence samples is calculated. The displacement error is defined as the Euclidean distance between the predicted position and the actual position. The displacement error values ​​at all times are summed and divided by the total number of times to obtain the average displacement error. The average displacement error directly reflects the network's performance in terms of position prediction accuracy and is a core indicator for measuring trajectory prediction accuracy. This average displacement error is used as the main loss term.

[0105] Further, the energy deviation constraint loss term is calculated. Under ideal physical conditions, the mechanical energy of a projectile should be conserved during flight under gravity alone. Although non-conservative forces exist in real projectile motion, as a physical constraint, the predicted trajectory should satisfy the law of conservation of energy as much as possible. Before calculating the energy deviation constraint loss term, the mass parameters of the projectile are first obtained. The mass parameters can be directly obtained from the labeled information of the training samples. For simulation data, the mass is a known parameter set by the simulation engine; for real data, the mass can be obtained through pre-measurement or estimation. Then, based on the position and velocity information and mass parameters at each moment in the sample predicted future trajectory sequence, the kinetic and potential energy at each moment are calculated. The formula for calculating kinetic energy is half the product of mass and the square of velocity. The formula for calculating potential energy is the product of mass, the gravitational acceleration constant, and altitude, where altitude is the coordinate value of the position information in the vertical direction of the world coordinate system. For the start and end moments in the sample predicted future trajectory sequence, the sum of the kinetic and potential energies at these two moments is calculated to obtain the mechanical energy at the start and end moments. The difference between the mechanical energy at the endpoint and the mechanical energy at the starting point is calculated. The absolute value of this difference is taken to obtain the energy change. If the energy change is greater than a preset energy deviation threshold, the portion of the energy change exceeding the preset energy deviation threshold is used as an energy deviation constraint loss term. This loss term aims to suppress abnormal deviations in the predicted trajectory that significantly deviate from the reasonable range of energy change on a physical macroscopic level during training, preventing the model from generating non-physical trajectory divergences. It should be noted that in real projectile scenarios, although non-ideal aerodynamic effects such as air resistance cause mechanical energy attenuation, the dissipation amplitude is usually small and the change is gradual; simultaneously, the model prediction itself introduces small random errors. In this embodiment, the energy deviation constraint loss term is used as a soft constraint regularization term in the total loss. Its physical meaning is not to force the predicted trajectory to strictly satisfy absolute energy conservation, but rather to allow mechanical energy to fluctuate freely within a certain threshold range (e.g., small attenuation due to air resistance) by setting the preset energy deviation threshold, and only penalizes extreme abnormal deviations exceeding this reasonable physical limit (e.g., predicting energy that violates the macroscopic boundary of energy conservation by amplification or excessive attenuation). Through this soft constraint mechanism, the network retains its ability to learn complex aerodynamics while effectively preventing the risk of trajectory divergence that may occur when the end-to-end model extrapolates over long time domains.

[0106] Furthermore, the main loss term and the energy bias constraint loss term are weighted and summed to obtain the total loss. The weighted summation involves adding the main loss term to the energy bias constraint loss term and multiplying it by a weight coefficient. This weight coefficient adjusts the relative importance of physical constraints during training. A larger weight coefficient value results in a stronger influence of physical constraints on training and higher physical plausibility of the predicted trajectory; a smaller weight coefficient value results in a stronger dominance of data fitting and potentially higher prediction accuracy. The selection of this weight coefficient can be optimized during training using cross-validation.

[0107] Furthermore, the backpropagation algorithm is used to propagate the gradient of the total loss layer by layer along the differentiable kinematics layer, the residual prediction head, and the sequence encoder, synchronously updating the parameters of the sequence encoder and the residual prediction head. Since all operations in the differentiable kinematics layer are differentiable, the gradient of the total loss with respect to all trainable parameters in the network can be accurately calculated using the chain rule. Using the calculated gradient, the parameters of each layer are updated through the gradient descent optimization algorithm, reducing the total loss value in the next iteration. Commonly used optimization algorithms include stochastic gradient descent and its variants, such as kinetic gradient descent with momentum and adaptive moment estimator optimizer. By repeatedly executing the above single-iteration training steps, the prediction accuracy and physical plausibility of the network gradually improve with the increase of training rounds until the preset training termination condition is reached.

[0108] This embodiment achieves dual constraints of data-driven and physics-driven approaches by designing a joint loss function that combines data fitting loss and physical constraint loss. The average displacement error ensures prediction accuracy, while the energy conservation constraint loss makes the predicted trajectory more physically plausible. The weighted summation mechanism allows for flexible adjustment of the relative importance of data fitting and physical constraints according to specific task requirements. The presence of a differentiable kinematics layer enables the total loss to propagate end-to-end back to the sequence encoder and residual prediction head, achieving global joint optimization.

[0109] In some preferred embodiments, after the completion of Phase Two and the deployment of the sequence encoder and the residual prediction head to the embodied agent, an online adaptive correction step is further included: during deployment and operation, acquiring the actual observed trajectory segments of the projectile, the actual trajectory segments containing the position information of each observation time and its corresponding timestamp; extracting the predicted trajectory subsequence within the time interval corresponding to the actual trajectory segment from the predicted future trajectory sequence, calculating the online displacement error between the predicted trajectory subsequence and the actual trajectory segment; determining whether the online displacement error exceeds a preset error threshold; if the online displacement error does not exceed... The error threshold is used to keep the current network parameters unchanged and continue to output the predicted trajectory for subsequent time steps. If the online displacement error exceeds the error threshold, an online parameter update is triggered. The online parameter update is as follows: only the parameters of the residual prediction head are updated by single-step gradient descent, keeping the parameters of the sequence encoder and the differentiable kinematics layer unchanged, and the updated residual prediction head is used for trajectory prediction for subsequent time steps. The change trend of each component of the updated residual acceleration vector is analyzed to determine the change amplitude of the residual acceleration in each orthogonal axis direction. The axis direction whose change amplitude exceeds a preset amplitude threshold is identified as the dominant interference direction and the dominant interference direction is output.

[0110] In practice, the actual operating environment during the deployment and operation of the embodied agent may differ from the environmental conditions during the training phase. For example, sudden lateral airflow may alter the flight path of the projectile, changes in ambient temperature may cause changes in air density, thus affecting air resistance, and substances adhering to the projectile's surface may alter its aerodynamic characteristics. These environmental changes may lead to a decrease in the prediction accuracy of the pre-trained model, affecting the success rate of task execution. This embodiment addresses the above problems through an online adaptive correction mechanism, which utilizes real-time observation data during deployment and operation to perform lightweight online updates to the network.

[0111] First, during deployment and operation, actual trajectory segments of the projectile are acquired based on observations. The onboard vision sensors of the embodied agent continuously acquire environmental images and track the actual movement of the projectile during operation, generating the projectile's actual trajectory within the observation time window. After the projectile has flown through several frames, the sensors have obtained the projectile's true position information at several consecutive moments. These observed true position information are arranged chronologically to form a true trajectory segment. This true trajectory segment contains the position information at each observation moment and its corresponding timestamp. Unlike the offline dataset used in the training phase, this true trajectory segment is acquired in real time during deployment and operation, reflecting the actual physical state and dynamic characteristics of the current operating environment.

[0112] Furthermore, the predicted trajectory subsequences within the time intervals corresponding to the actual trajectory segments are extracted from the predicted future trajectory sequence. Specifically, based on the timestamp information of each observation moment in the actual trajectory segment, the location information matching that timestamp is searched in the already generated predicted future trajectory sequence. Since the location information of each moment in the predicted future trajectory sequence corresponds to a specific timestamp, the spatiotemporal correspondence between the predicted trajectory subsequences and the actual trajectory segments can be achieved through timestamp matching. The predicted positions of all matching moments within this time interval are extracted to form the predicted trajectory subsequences.

[0113] Furthermore, the online displacement error between the predicted trajectory subsequence and the true trajectory segment is calculated. For each observation time in the true trajectory segment, the displacement error between the predicted position and the true position at that time is calculated. The displacement error is defined as the Euclidean distance between the predicted position vector and the true position vector. The average displacement error over all observation times is taken to obtain the online displacement error. This online displacement error reflects the real-time prediction performance of the model under the current operating environment.

[0114] Furthermore, it is determined whether the online displacement error exceeds a preset error threshold. This error threshold is pre-set during the training phase or before deployment based on specific task requirements. For example, for grasping tasks requiring high precision, the error threshold can be set to 0.05 meters or 0.1 meters, depending on the size of the grasped target and the tolerance range of the robotic arm. The setting of the error threshold reflects the task's tolerance for trajectory prediction accuracy.

[0115] Furthermore, if the online displacement error does not exceed the error threshold, it indicates that the model's prediction accuracy under the current environment still meets the task requirements, and no parameter adjustment is needed. In this case, the current network parameters remain unchanged, and the existing model continues to output the predicted trajectory for subsequent time steps. This branch design avoids unnecessary parameter updates, ensuring the stability and real-time performance of the system under normal operating conditions.

[0116] Furthermore, if the online displacement error exceeds the error threshold, it indicates that the model's prediction accuracy under the current environment has decreased to an unacceptable level, requiring an online parameter update. The online parameter update has the following specific design: single-step gradient descent updates are performed only on the parameters of the residual prediction head, while keeping the parameters of the sequence encoder and the differentiable kinematics layer unchanged. This selective update design is based on the following technical considerations.

[0117] Furthermore, the sequence encoder is responsible for extracting temporal features. Its parameters have been fully optimized with large-scale data during offline training, exhibiting strong robustness to general environmental changes. The differentiable kinematics layer performs deterministic calculations of physical laws, with its parameters being fixed physical constants (including the gravitational acceleration constant), representing invariant physical laws that should not change with environmental conditions. The residual prediction head is responsible for modeling residual acceleration caused by non-ideal aerodynamic effects. This part is precisely the part most directly affected by changes in environmental conditions: for example, a sudden lateral airflow will significantly change the residual acceleration component along a certain axis, but has no effect on gravitational acceleration itself. Therefore, only updating the parameters of the residual prediction head is needed to achieve effective adaptation to environmental changes, while maintaining the temporal feature extraction capability and the stability of the basic physical framework established during the pre-training phase to the greatest extent.

[0118] Furthermore, in the specific implementation, online training samples are constructed using real trajectory fragments. The earlier portion of the historical trajectory sequence from the real trajectory fragment is used as the input to the online training samples, and the later portion of the real trajectory fragment is used as the supervision label. One forward and backward propagation calculation is performed to update the residual prediction head using single-step gradient descent. Since the number of parameters in the residual prediction head is much smaller than the number of parameters in the entire network, the computational cost of single-step gradient descent update is very small, making it suitable for real-time execution on the limited computing resources of onboard computing units. The updated residual prediction head is then used for trajectory prediction at subsequent time steps.

[0119] Furthermore, after completing the online parameter update, the changing trends of each component of the updated residual acceleration vector are further analyzed. Since the three components of the residual acceleration vector have clear physical dimensions and meanings, corresponding to acceleration deviations along the X, Y, and Z axes in the world coordinate system, the type of change in the current environmental conditions can be inferred by analyzing the changing trends of each component. The specific analysis method is as follows: obtain the values ​​of each component of the residual acceleration vector before and after the update, calculate the difference between each component at the current time and the previous time, and use this difference as the change amplitude of that component. Then, compare the change amplitude of each component with a preset amplitude threshold. The axis direction where the change amplitude exceeds the preset amplitude threshold is identified as the dominant interference direction, and this dominant interference direction is output. For example, if the change amplitude of the residual acceleration in the X-axis direction of the world coordinate system increases significantly (exceeding the preset X-axis component threshold), while the change amplitudes in the Y-axis and Z-axis directions are relatively small, it can be inferred that there is lateral airflow interference in the current environment, and the dominant interference direction is the X-axis direction. This output can be used as part of the environmental perception information of the embodied intelligent agent, and can be used as a reference for the upper-level task planning or decision-making modules to achieve autonomous attribution and strategy adjustment based on physically interpretable information.

[0120] Furthermore, this embodiment, through the aforementioned online adaptive correction mechanism, enables the embodied agent to autonomously perceive environmental changes and adjust network parameters based on real-time observation data during deployment and operation, maintaining long-term stability of prediction accuracy. Simultaneously, through physically interpretable analysis of the residual acceleration component variation trend, the embodied agent is provided with environmental diagnostic capabilities that surpass traditional "black box" models.

[0121] This embodiment calculates online displacement error by collecting real-time trajectory segments and determines whether it exceeds a preset threshold. Only when the threshold is exceeded is a lightweight online parameter update triggered, updating only the residual prediction head without affecting the physical layer and temporal coding layer. This allows the embodied agent to quickly adapt to environmental changes while maintaining the stability of the basic physical framework, avoiding the computational overhead and risk of degradation of the basic physical framework caused by full retraining. Simultaneously, by analyzing the changing trends of each component of the residual acceleration, the dominant interference direction is output, providing physically meaningful diagnostic information for the embodied agent's environmental perception and task decision-making.

[0122] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0123] Please see Figure 2 , Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server, wherein the server can be a standalone server or a server cluster composed of multiple servers.

[0124] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0125] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it enables the processor 502 to execute a projectile trajectory prediction method that integrates an embodied intelligent physical model.

[0126] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0127] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a projectile trajectory prediction method that integrates an embodied intelligent physical model.

[0128] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0129] The processor 502 is used to run a computer program 5032 stored in a memory to implement the steps of the projectile trajectory prediction method based on the fusion of an embodied intelligent physical model provided in any of the above method embodiments.

[0130] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0131] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0132] Therefore, the present invention also provides a storage medium. This storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the steps of the projectile trajectory prediction method based on embodied intelligent physical model fusion provided in any of the above method embodiments.

[0133] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0134] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0135] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0136] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0137] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0139] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, this invention is also intended to include these modifications and variations as long as they fall within the scope of the claims of this invention and their equivalents.

[0140] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A projectile trajectory prediction method based on the fusion of embodied intelligent physical models, characterized in that, include: Obtain the historical trajectory sequence of the projectile, the historical trajectory sequence containing the position and velocity information of the projectile at each of multiple consecutive historical moments; The historical trajectory sequence is input into a sequence encoder for temporal feature extraction to obtain a context feature vector; The context feature vector is input into the residual prediction head to perform acceleration residual prediction, and a residual acceleration vector is obtained. The residual acceleration vector is used to characterize the acceleration deviation caused by non-ideal aerodynamic effects. The residual acceleration vector is added to the preset gravitational acceleration constant to obtain the total acceleration vector; The total acceleration vector is input into the differentiable kinematics layer. The position and velocity information corresponding to the last historical moment in the historical trajectory sequence are used as the initial state. The kinematic equations are numerically integrated to obtain the predicted future trajectory sequence. The predicted future trajectory sequence contains the position and velocity information corresponding to each of the multiple future moments.

2. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 1, characterized in that, The acquisition of the historical trajectory sequence of the projectile includes: Acquire continuous frame images from airborne vision sensors; Target detection is performed on the projectiles in the consecutive frame images to determine the detection bounding box of the projectile in each frame, and the center point of the detection bounding box in each frame is taken as the image position of the projectile in that frame; Cross-frame association matching is performed on the projectiles detected between each frame, and the detection results belonging to the same projectile are associated according to the frame order to obtain the image position sequence of the same projectile in each frame. Each image position in the image position sequence is mapped to the world coordinate system to obtain the world coordinate position corresponding to each frame, which serves as the position information for each frame. The position information of adjacent frames is differentially calculated, and the differential result is divided by the time interval between adjacent frames to obtain the velocity information corresponding to each frame. The location information and velocity information corresponding to multiple consecutive historical frames are assembled in chronological order to obtain the historical trajectory sequence.

3. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 1, characterized in that, The step of inputting the historical trajectory sequence into a sequence encoder for temporal feature extraction to obtain a context feature vector includes: The position and velocity information of each moment in the historical trajectory sequence are concatenated into a single-step state vector to obtain a state vector sequence. The state vector sequence is input into the sequence encoder, which is an LSTM network. The LSTM network performs gated recursive processing on each single-step state vector in the state vector sequence through its forget gate, input gate, and output gate to update the hidden state and cell state of the LSTM network. The hidden state output by the LSTM network at the last time step is used as the context feature vector. Alternatively, the state vector sequence is input into the sequence encoder, which is a Transformer network. The position of each single-step state vector in the state vector sequence is encoded and then input into the encoder of the Transformer network. The encoder of the Transformer network models the temporal dependency between each single-step state vector through a multi-head self-attention mechanism and outputs the encoded feature vector corresponding to each time step. The encoded feature vector corresponding to each time step is pooled, and the aggregated feature vector obtained after pooling is used as the context feature vector.

4. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 1, characterized in that, The step of inputting the context feature vector into the residual prediction head to perform acceleration residual prediction and obtain the residual acceleration vector includes: The context feature vector is input into the residual prediction head, which is a multi-layer fully connected network. The multi-layer fully connected network includes an input layer, at least one hidden layer, and an output layer connected in sequence. The hidden layer uses the ReLU activation function, and the output layer uses a linear activation function. The input layer receives the context feature vector, performs a linear transformation on the weight matrix and bias vector of each hidden layer, and then performs a nonlinear mapping on the ReLU activation function. The output layer outputs a three-dimensional vector, which is used as the residual acceleration vector. The three components of the residual acceleration vector correspond to the acceleration deviation values ​​in the three orthogonal axes of the world coordinate system.

5. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 1, characterized in that, The step of inputting the total acceleration vector into a differentiable kinematics layer, using the position and velocity information corresponding to the last historical moment in the historical trajectory sequence as the initial state, and performing numerical integration on the kinematic equations to obtain the predicted future trajectory sequence includes: Using the position information of the last historical moment in the historical trajectory sequence as the initial position and the velocity information of the last historical moment in the historical trajectory sequence as the initial velocity, the initial conditions for constructing the kinematic equation system are established. The kinematic equation system includes the first derivative of position with respect to time being equal to velocity and the first derivative of velocity with respect to time being equal to the total acceleration vector. The total acceleration vector is used as the acceleration term in the kinematic equations. Numerical integration is performed using a fourth-order Runge-Kutta integrator. Within each integration step, the fourth-order Runge-Kutta integrator calculates the first derivative estimate at the beginning of the integration step, the second derivative estimate and the third derivative estimate at half the integration step, and the fourth derivative estimate at the end of the integration step. The first derivative estimate, the second derivative estimate, the third derivative estimate and the fourth derivative estimate are weighted and averaged to obtain the final slope of the integration step. The position and velocity of the current step are updated based on the final slope. Repeatedly perform numerical integration until the preset prediction time domain length is reached. Arrange the positions and velocities output by each integration step in chronological order to obtain the predicted future trajectory sequence.

6. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 1, characterized in that, The method further includes a step of performing two-stage training on the sequence encoder and the residual prediction head: In Phase 1, the sequence encoder and the residual prediction head are pre-trained using a projectile motion dataset generated by a pure physics simulation engine. The pure physics simulation engine is configured with only the gravitational acceleration term and the simulation of air resistance and rotation effects is turned off, generating projectile trajectory data driven only by gravity. The projectile trajectory data is used as training samples to iteratively train the sequence encoder and the residual prediction head until the sequence encoder and the residual prediction head converge. The pure physics simulation engine is a computer software system that performs motion simulation based on physical laws. In Phase Two, the pre-trained sequence encoder and residual prediction head are deployed to the embodied agent. The pre-trained sequence encoder and residual prediction head are fine-tuned using the airborne vision sensor of the embodied agent in the real working environment. The learning rate in Phase Two is lower than that in Phase One.

7. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 6, characterized in that, The single-iteration training steps used in either Phase One or Phase Two include: Acquire training samples, which include historical trajectory sequence samples of the projectile and their corresponding real future trajectory sequence samples; The historical trajectory sequence samples are input into the sequence encoder for temporal feature extraction to obtain the sample context feature vector; The sample context feature vector is input into the residual prediction head to perform acceleration residual prediction, and the sample residual acceleration vector is obtained. The sample residual acceleration vector is added to the gravitational acceleration constant to obtain the total sample acceleration vector; The total acceleration vector of the sample is input into the differentiable kinematics layer. The position and velocity information of the last moment in the historical trajectory sequence sample are used as the initial state. The kinematic equation is numerically integrated to obtain the sample's predicted future trajectory sequence. Calculate the displacement error between the predicted position at each moment in the sample predicted future trajectory sequence and the actual position at the corresponding moment in the real future trajectory sequence sample, and average the displacement error over all moments to obtain the average displacement error, and use the average displacement error as the main loss term. Based on the position and velocity information at each moment in the sample predicted future trajectory sequence, and the mass parameters of the projectile obtained, the kinetic energy and potential energy at each moment are calculated. The difference between the sum of kinetic energy and potential energy at the end moment and the start moment in the sample predicted future trajectory sequence is calculated, and the portion of the absolute value of the difference that exceeds the preset energy deviation threshold is taken as the energy deviation constraint loss term. The total loss is obtained by weighted summing of the main loss term and the energy deviation constraint loss term. The total loss is backpropagated layer by layer along the differentiable kinematics layer, the residual prediction head, and the sequence encoder using the backpropagation algorithm, and the parameters of the sequence encoder and the residual prediction head are updated synchronously.

8. The projectile trajectory prediction method based on the fusion of embodied intelligent physical models according to claim 6, characterized in that, After Phase Two is completed and the sequence encoder and the residual prediction head are deployed to the embodied agent, an online adaptive correction step is also included: During deployment and operation, actual observations of the projectile's true trajectory fragments are obtained, and these true trajectory fragments include the location information and corresponding timestamps for each observation time. Extract the predicted trajectory subsequence within the time interval corresponding to the actual trajectory segment from the predicted future trajectory sequence, and calculate the online displacement error between the predicted trajectory subsequence and the actual trajectory segment; Determine whether the online displacement error exceeds a preset error threshold; If the online displacement error does not exceed the error threshold, keep the current network parameters unchanged and continue to output the predicted trajectory for subsequent time moments; If the online displacement error exceeds the error threshold, an online parameter update is triggered. The online parameter update is as follows: only the parameters of the residual prediction head are updated by single-step gradient descent, while keeping the parameters of the sequence encoder and the differentiable kinematics layer unchanged, and the updated residual prediction head is used for trajectory prediction at subsequent time steps. The variation trend of each component of the updated residual acceleration vector is analyzed to determine the variation amplitude of the residual acceleration in each orthogonal axis direction. The axis direction whose variation amplitude exceeds the preset amplitude threshold is identified as the dominant interference direction, and the dominant interference direction is output.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-8.

Images