Method for trajectory prediction of a trans-medium vehicle and computer program product

Abstract

The application discloses a method for predicting a motion trajectory of a cross-medium vehicle and a computer program product. The method comprises the following steps: acquiring an observed state parameter of the cross-medium vehicle at a current time point; inputting the observed state parameter at the current time point into a trained trajectory prediction model and outputting a predicted state parameter at a next time point, the trained trajectory prediction model comprising a trained state prediction model and a trained recurrent neural network, the trained state prediction model being used for fusing a predicted state parameter at a previous time point and the observed state parameter at the current time point to obtain a corrected predicted state parameter at the current time point, and calculating a predicted state at the next time point according to the corrected predicted state parameter at the current time point, and the trained recurrent neural network being used for calculating a weight coefficient during fusion; and determining a motion trajectory of the cross-medium vehicle according to the predicted state parameter of the cross-medium vehicle at the next time point. The application can improve the accuracy of the motion trajectory prediction of the cross-medium vehicle.

Description

Technical Field

[0001] This application belongs to the field of automatic control and navigation technology, and in particular relates to a method for predicting the motion trajectory of a cross-medium vehicle and a computer program product. Background Technology

[0002] Transmedia vehicles are a new type of intelligent equipment capable of freely switching between and efficiently navigating different media such as air and water. Their seamless air-to-water transition capability has significant application value in fields such as military reconnaissance, marine exploration, and disaster relief. When traversing the water-air interface, transmedia vehicles face multiple dynamic challenges, including abrupt changes in flow field characteristics, a sharp decline in control stability, and complex environmental disturbances. Therefore, high-precision prediction of their trajectory is a key technological prerequisite for achieving autonomous navigation, risk avoidance, and mission optimization.

[0003] Currently, Kalman filtering-based prediction methods are commonly used for trajectory prediction. However, the accurate prediction of such methods relies on a precise mathematical model describing the vehicle's motion and the statistical characteristics of internal and external disturbances during its movement. But in the scenario of cross-medium navigation, where motion patterns change drastically, there is neither a unified and accurate mathematical model to describe the entire cross-medium navigation process, nor can the statistical patterns of various complex disturbances be known in advance. Therefore, current methods for predicting the trajectory of cross-medium vehicles suffer from insufficient prediction accuracy. Summary of the Invention

[0004] This application provides a method and computer program product for predicting the trajectory of a cross-medium vehicle, which can solve the problem of insufficient accuracy in predicting the trajectory of a cross-medium vehicle and improve the accuracy of trajectory prediction.

[0005] On one hand, embodiments of this application provide a method for predicting the trajectory of a cross-medium vehicle, including:

[0006] Obtain the observation status parameters of the cross-medium vehicle at the current time point;

[0007] The observed state parameters at the current time point are input into the trained trajectory prediction model, and the trained trajectory prediction model is used to output the predicted state parameters of the cross-medium vehicle at the next time point. The trained trajectory prediction model includes a trained state prediction model and a trained recurrent neural network. The trained state prediction model includes the predicted state parameters of the cross-medium vehicle at the previous time point. The trained state prediction model is used to fuse the predicted state parameters of the previous time point and the observed state parameters at the current time point to obtain the corrected predicted state parameters at the current time point, and to calculate the predicted state at the next time point based on the corrected predicted state parameters at the current time point. The trained recurrent neural network is used to calculate the weight coefficients between the predicted state parameters of the previous time point and the observed state parameters at the current time point during fusion.

[0008] The trajectory of the cross-medium vehicle is determined based on the predicted state parameters of the cross-medium vehicle at the next time point.

[0009] On the other hand, embodiments of this application provide a motion trajectory prediction device for a cross-medium vehicle, comprising:

[0010] The acquisition module is used to acquire the observation status parameters of the cross-medium vehicle at the current time point;

[0011] The prediction module is used to input the observed state parameters at the current time point into a trained trajectory prediction model, and use the trained trajectory prediction model to output the predicted state parameters of the cross-medium vehicle at the next time point. The trained trajectory prediction model includes a trained state prediction model and a trained recurrent neural network. The trained state prediction model includes the predicted state parameters of the cross-medium vehicle at the previous time point. The trained state prediction model is used to fuse the predicted state parameters at the previous time point and the observed state parameters at the current time point to obtain the corrected predicted state parameters at the current time point, and to calculate the predicted state at the next time point based on the corrected predicted state parameters at the current time point. The trained recurrent neural network is used to calculate the weighting coefficients between the predicted state parameters at the previous time point and the observed state parameters at the current time point during fusion.

[0012] The determination module is used to determine the motion trajectory of the cross-medium vehicle based on the predicted state parameters of the cross-medium vehicle at the next time point.

[0013] In another aspect, embodiments of this application provide an electronic device, including: a processor and a memory storing computer program instructions, wherein the processor reads and executes the computer program instructions stored in the memory to implement the motion trajectory prediction method for a cross-medium vehicle as described above.

[0014] In another aspect, embodiments of this application provide a computer-readable storage medium storing computer program instructions; when executed by a processor, the computer program instructions implement the motion trajectory prediction method for a cross-medium vehicle described above.

[0015] In another aspect, embodiments of this application provide a computer program product, including a computer program that, when executed, implements the motion trajectory prediction method for a cross-medium vehicle as described above.

[0016] The trajectory prediction method and computer program product for cross-medium vehicles in this application obtain the corrected predicted state parameters for the current time point by fusing the predicted state parameters of the previous time point and the observed state parameters of the current time point using a trained state prediction model. This balances the weights of observation and prediction, reduces the impact of model mismatch and sensor noise, and uses a trained recurrent neural network to calculate the weight coefficients during fusion. This allows the weight coefficients to be dynamically adjusted with environmental changes, without relying on predefined noise statistics, thus calculating more accurate corrected predicted state parameters for the current time point and improving prediction accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for predicting the trajectory of a cross-medium vehicle according to an embodiment of this application;

[0019] Figure 2 This is a schematic diagram of the iterative state estimation process based on Kalman filtering;

[0020] Figure 3 This is a schematic diagram of the structure of a preset trajectory prediction model provided in one embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a preset recurrent neural network provided in one embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the motion trajectory prediction device for a cross-medium vehicle provided in one embodiment of this application;

[0023] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. Detailed Implementation

[0024] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0025] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0026] First, the background technology involved in this application will be explained.

[0027] Cross-medium vehicles are a new type of intelligent equipment capable of freely switching between and efficiently navigating in different media such as air and water. They offer the advantage of overcoming the limitations of a single medium and achieving seamless air-to-water transitions, overcoming the limitations of traditional vehicles such as aircraft (which can only operate in the air) and submarines (which can only operate underwater). This greatly expands the scope and flexibility of mission execution, thus showing broad application prospects in areas such as marine reconnaissance, marine exploration, disaster relief, and environmental monitoring. However, cross-medium vehicles face multiple dynamic and control challenges during cross-medium flight, including abrupt changes in flow field, decreased control stability, and complex environmental disturbances. Therefore, high-precision trajectory prediction is urgently needed to achieve autonomous navigation, cooperative control, and mission efficiency optimization, thereby improving their adaptability and reliability in complex multi-medium environments.

[0028] Currently, due to the complex operating environment of cross-medium vehicles, they face multiple challenges during cross-medium flight, including abrupt changes in flow field characteristics, decreased control stability, and complex environmental disturbances. Specifically, in terms of flow field, the density difference between air and water is as high as approximately 800 times, requiring the shape design to consider the lift and drag characteristics of both media, easily leading to flow separation and stall. Simultaneously, the dynamics exhibit high nonlinearity and strong coupling, rendering traditional single-medium models ineffective, and establishing a unified cross-medium dynamics model is extremely difficult, severely limiting prediction accuracy. In terms of control, the coupling effects of low-altitude turbulence and interface eddies are significant, and changes in the center of mass cause abrupt changes in pitch / roll moments. Control surface efficiency decreases nonlinearly with air density, making stability difficult to maintain. In terms of the environment, airborne vehicles are affected by wind fields and turbulence, while underwater they are constrained by waves, currents, and density stratification. These time-varying disturbances have high uncertainty and are difficult to monitor in real time, resulting in insufficient prediction accuracy for current methods of predicting the trajectory of cross-medium vehicles when they cross the water-air interface.

[0029] To address the problems of the prior art, this application provides a method and computer program product for predicting the trajectory of a cross-medium vehicle. The method for predicting the trajectory of a cross-medium vehicle provided in this application will be described below.

[0030] Figure 1 This is a flowchart illustrating a method for predicting the trajectory of a cross-medium vehicle according to an embodiment of this application. Figure 1 As shown, in one embodiment of this application, the execution subject of the motion trajectory prediction method for a cross-medium vehicle can be an electronic device, and the motion trajectory prediction method for a cross-medium vehicle includes steps 101 to 103.

[0031] Step 101: Obtain the observation status parameters of the cross-medium vehicle at the current time point.

[0032] The observed state parameters at the current time point refer to the physical state measurements of the cross-medium vehicle collected in real time by sensors at the current time point, including position, velocity, acceleration, attitude angles, etc. The observed state parameters can reflect the real-time motion state of the vehicle.

[0033] In some embodiments, sensor systems deployed on the transmedium vehicle can collect real-time sensor data at the current point in time and transmit the sensor data to an electronic device via a data transmission protocol. The electronic device then preprocesses the sensor data to obtain the observation status parameters of the transmedium vehicle at the current point in time.

[0034] The sensor system can include inertial measurement units (IMUs), pressure sensors, temperature sensors, wind speed sensors, and vision sensors. The IMU measures the acceleration, angular velocity, and attitude of the transmedium vehicle and can be positioned at critical locations on the vehicle. Pressure sensors measure the water pressure underwater; the pressure increases with depth. Temperature sensors measure seawater temperature to analyze the impact of temperature changes on the vehicle's materials and operational safety; these sensors can be placed on critical components, such as near mechanical parts and the propulsion system. Wind speed sensors detect wind speed and direction during operation and can be positioned on the sides of the transmedium vehicle to measure wind speed and direction in real time. Vision sensors provide real-time visual data, helping operators observe the environment of the flight area, such as obstacles and underwater conditions. These sensors can be cameras, positioned at the front or sides of the vehicle to acquire real-time video streams of the surrounding environment.

[0035] Data transmission protocols can include Transmission Control Protocol (TCP) and Message Queuing Telemetry Transport (MQTT). Preprocessing can include techniques such as filtering and normalization.

[0036] In some embodiments, cross-medium vehicles and their sensing units are typically equipped with multiple independent clock sources. To ensure data consistency across time, a unified time synchronization mechanism is required. A high-precision time protocol, such as PTP, can be used to calibrate each clock source, thereby providing a consistent time reference for all acquired data. Here, the sensing unit refers to the sensor in the sensor system.

[0037] In some embodiments, timestamps can be embedded in the data acquired by all sensors and synchronized to the same reference clock. This enables precise timing alignment between multi-source heterogeneous data, providing a reliable basis for subsequent trajectory prediction.

[0038] In some embodiments, in bandwidth-constrained communication environments, compression algorithms, such as Huffman coding or LZ77 series algorithms, can be used to compress transmitted data to reduce bandwidth usage and improve transmission efficiency.

[0039] Step 102: Input the observed state parameters at the current time point into the trained trajectory prediction model, and use the trained trajectory prediction model to output the predicted state parameters of the cross-medium vehicle at the next time point. The trained trajectory prediction model includes a trained state prediction model and a trained recurrent neural network. The trained state prediction model includes the predicted state parameters of the cross-medium vehicle at the previous time point. The trained state prediction model is used to fuse the predicted state parameters at the previous time point and the observed state parameters at the current time point to obtain the corrected predicted state parameters at the current time point, and calculate the predicted state at the next time point based on the corrected predicted state parameters at the current time point. The trained recurrent neural network is used to calculate the weight coefficients between the predicted state parameters at the previous time point and the observed state parameters at the current time point during fusion.

[0040] Predicted state parameters refer to the state estimates of a cross-medium vehicle at a specific point in time, calculated using a trained trajectory prediction model. These parameters include position, velocity, acceleration, and attitude angles. Predicted state parameters are predictions of future states based on models and historical data.

[0041] The corrected predicted state parameters at the current time point refer to the optimal state estimate obtained by fusing the predicted state parameters from the previous time point and the observed state parameters at the current time point. These parameters include position, velocity, acceleration, and attitude angles. Corrected predicted state parameters can eliminate biases from single predictions or observations, thus improving the accuracy of state estimation.

[0042] Weighting coefficients are parameters used in the fusion process to balance the relative importance of the predicted state parameters from the previous time point and the observed state parameters from the current time point. These weighting coefficients are calculated from the trained recurrent neural network.

[0043] The trained trajectory prediction model is a machine learning model trained using historical data. It consists of a trained state prediction model and a trained recurrent neural network, and can output predicted values ​​based on input state parameters.

[0044] The trained state prediction model is a mathematical model built based on the principles of cross-medium vehicle dynamics, such as kinematic equations, and is used to perform state prediction and fusion operations.

[0045] A trained recurrent neural network (RNN) is a type of neural network with internal memory units, such as a recurrent neural network (RNN), a long short-term memory network (LSTM), or a gated recurrent unit (GRU). Trained RNNs are used to process sequential data and output weight coefficients to optimize the fusion process.

[0046] In some embodiments, the trained state prediction model includes predicted state parameters of the transmedium vehicle at the previous time point. The trained state prediction model can predict the state parameters of a cross-medium vehicle at the previous time point. Observational state parameters at the current time point The parameters are fused to generate the corrected predicted state parameters for the current time point. The fusion process is similar to the update step of Kalman filtering, but the predicted state parameters from the previous time point are changed during fusion. and the observation state parameters at the current time point The weighting coefficients are calculated by the trained recurrent neural network. Furthermore, the trained state prediction model can be based on the dynamic equations of the cross-medium vehicle, such as the air attitude equations or the underwater state equations, according to the corrected predicted state parameters at the current time point. Calculate the predicted state at the next time point. .

[0047] The air attitude equation of the cross-medium vehicle describes the vehicle's motion attitude. For example, the center of mass is used as the origin of rotation, the current attitude is used as the starting attitude S, the input attitude is used as the target attitude E, and the step size between the starting attitude and the target attitude is based on the input time difference. dt The interpolation speed is adjusted based on the difference between the current attitude S and the target attitude E, and the vehicle transitions to the target attitude frame by frame. The following steps 201 to 205 can be used to establish the air attitude equation of the cross-medium vehicle:

[0048] Step 201: Calculate the interpolation speed. For large attitude differences, use fine interpolation; for small attitude differences, use fast interpolation.

[0049] .

[0050] Step 202: Calculate the interpolation ratio based on the time difference.

[0051] .

[0052] Step 203, calculate the target rotation ( ) and current rotation ( The difference between ) and, through Limit it to a reasonable range of angles:

[0053] .

[0054] Step 204, based on the interpolation speed ( ) Calculate the rotation increment applied to the current frame, and through Speed ​​is limited to the range [0,1].

[0055] .

[0056] Step 205: Add the interpolation increment to the current rotation, and then... Correct the results to ensure the final rotation amount is valid.

[0057] .

[0058] in, interpSpeed Interpolate the velocity for the aircraft. currentPose The current flight attitude of the vehicle. targetPose The target attitude of the vehicle. DeltaInterpSpeed Interpolate the speed increment for the vehicle. DeltaMove For the vehicle rotation increment, Delta The difference between the target rotation and the current rotation. Clamp () represents the attitude constraint function for the aircraft. Normalized () is the normalization function. max () is the maximization function.

[0059] The underwater state equations for cross-medium vehicles describe their underwater operational states, such as their diving and surfacing states. These equations characterize the influence of the vehicle's velocity and acceleration components on its position and attitude. They can be expressed by the following formula:

[0060] ,

[0061] ,

[0062] ,

[0063] .

[0064] in, P For the thrust of the waterjet propulsion, F The resultant force of buoyancy and drag on the aircraft. G For the weight of the spacecraft, M The buoyancy torque, α for F Angle with the vehicle's axis, The pitch angle, The distance from the front hydrofoil to the center of gravity. To bypass k Moment of inertia of the shaft for i velocity components in the direction, for j The velocity component in the direction.

[0065] Step 103: Determine the motion trajectory of the cross-medium vehicle based on the predicted state parameters of the cross-medium vehicle at the next time point.

[0066] In some embodiments, by analyzing the predicted state parameters of the cross-medium vehicle at the next time point, information such as the vehicle's position, velocity, acceleration, and attitude at the next time point can be obtained. Furthermore, the vehicle's trajectory can be determined based on its position at the current time point and its position at the next time point. For example, interpolation or sequence generation algorithms can be used to connect the current position and the position at the next time point into a trajectory curve to obtain the vehicle's trajectory.

[0067] In some embodiments, as time steps to the next time point t+1, the observation state parameters of the cross-medium vehicle at the next time point can be obtained. Therefore, the observed state parameters at the next time point can be... Input a trained trajectory prediction model and use the trained trajectory prediction model to output the predicted state parameters of the cross-medium vehicle at time t+2. Therefore, the state of the cross-medium vehicle can be continuously predicted over time, and the trajectory of the cross-medium vehicle can be obtained.

[0068] The trajectory prediction method for cross-medium vehicles provided in this application uses a trained state prediction model to fuse the predicted state parameters from the previous time point and the observed state parameters from the current time point to obtain the corrected predicted state parameters for the current time point. This method can balance the weights of observation and prediction, reduce the impact of model mismatch and sensor noise, and use a trained recurrent neural network to calculate the weight coefficients during fusion. This allows the weight coefficients to be dynamically adjusted with environmental changes, without relying on predefined noise statistics, to calculate more accurate corrected predicted state parameters for the current time point, thereby improving prediction accuracy.

[0069] In some embodiments, in order to accurately determine the motion trajectory, step 103 is further refined to include:

[0070] Step 201: Analyze the corrected predicted state parameters at the current time point to obtain the corrected position information of the cross-medium vehicle at the current time point;

[0071] Step 202: Analyze the predicted state parameters at the next time point to obtain the predicted position information of the cross-medium vehicle at the next time point;

[0072] Step 203: Determine the motion trajectory based on the corrected location information and the predicted location information.

[0073] In some embodiments, the state parameters are typically a multi-dimensional vector containing various state variables such as position, velocity, acceleration, and attitude. Therefore, the corrected position component for the current time point can be extracted from the corrected predicted state parameters using an index or field access. Furthermore, operations such as coordinate system transformation can be used to convert the corrected position component for the current time point into location information in a geographic coordinate system, thus obtaining the corrected position information for the current time point. Similarly, the predicted position component for the next time point can be extracted from the predicted state parameters using an index or field access. Operations such as coordinate system transformation can then be used to convert the predicted position component for the next time point into location information in a geographic coordinate system, thus obtaining the predicted position information for the next time point.

[0074] In some embodiments, the corrected location information at the current time point can be one or more coordinate points in geospatial space, and the predicted location information at the next time point can also be one or more coordinate points in geospatial space. Therefore, the trajectory line from the current time point to the next time point can be generated based on the coordinate changes from the corrected location information at the current time point to the predicted location information at the next time point, and the motion trajectory of the cross-medium vehicle can be predicted.

[0075] The motion trajectory prediction method for cross-medium vehicles provided in this application predicts the motion trajectory using the corrected position information at the current time point and the predicted position information at the next time point. Since the corrected position information at the current time point reduces the error at the current point, and the predicted position at the next time point provides an estimate of the future point, the accuracy of the predicted motion trajectory can be improved, and the actual and expected path of the vehicle can be more accurately reflected.

[0076] In some embodiments, to further improve the accuracy of the predicted motion trajectory, the trained state prediction model also includes the observed predicted state parameters of the cross-medium vehicle at the current time point and the observed state parameters at the previous time point. After step 102, the model further includes:

[0077] Step 301: Calculate at least one of the following: observation difference at the current time point, new update information at the current time point, forward transition difference at the current time point, and forward update difference at the current time point; wherein, the observation difference at the current time point is the difference between the observation state parameters at the previous time point and the observation state parameters at the current time point, the new update information at the current time point is the difference between the observation state parameters at the current time point and the observation prediction state parameters at the current time point, the forward transition difference at the current time point is the difference between the prediction state parameters at the current time point and the prediction state parameters at the previous time point, and the forward update difference at the current time point is the difference between the corrected prediction state parameters at the current time point and the prediction state parameters at the current time point;

[0078] Step 302: Input at least one of the observation difference at the current time point, the new update information at the current time point, the forward transition difference at the current time point, and the forward update difference at the current time point into the trained recurrent neural network, and use the trained recurrent neural network to output the weight coefficients.

[0079] Step 303: Using the trained trajectory prediction model, the predicted state parameters of the previous time point and the observed state parameters of the current time point are fused based on the weight coefficients to obtain the corrected predicted state parameters of the current time point. The predicted state parameters of the next time point are calculated based on the corrected predicted state parameters of the current time point and then output.

[0080] Observational difference at the current time point , is the observed state parameter at the previous time point. Observational state parameters at the current time point The difference between observations reflects the change in the observed data over time, capturing the dynamic characteristics of the environment. The observation differences at the current time point form a sequence that reflects the changing trends and noise characteristics of the observed data itself. By calculating the differences between consecutive observations, some predictable trends can be removed, making it easier for recurrent neural networks to learn the noise statistics, which are related to the state transition process.

[0081] Observational prediction state parameters at the current time point It is the observation value that the trained state prediction model should have at the current time point, which is predicted by the corrected prediction state parameters at the previous time point. It is used to characterize the data that the sensor should measure under the ideal model.

[0082] New updates at the current time , is the observation state parameter at the current time point. With observed and predicted state parameters The difference between the observed and predicted values, also known as the innovation in Kalman filtering, characterizes the discrepancy between the actual observations and the model predictions. This difference contains crucial information about model prediction error and observation noise, directly reflecting the uncertainty of state estimation and serving as a key basis for calculating the Kalman gain.

[0083] The forward transition difference at the current time point is the predicted state parameter at the current time point. Predicted state parameters at the previous time point The difference is used to characterize the amount of change brought about by state prediction based solely on the model. It includes information about the state transition process and the impact of process noise, which helps recurrent neural networks understand the dynamic evolution characteristics of the system.

[0084] The forward update difference at the current time point is the corrected predicted state parameter at the current time. Predicted state parameters at the current time point The difference is used to quantify the degree to which the state estimate is corrected after incorporating new observation data. It directly reflects the corrective force of the observation data on the prediction results and is a direct measure of the change in uncertainty of the state estimate.

[0085] The loss value at the current time point is a parameter used to quantify the estimation error of the trajectory prediction model at the current time point.

[0086] In some embodiments, one or more features calculated in the previous step are input into a trained recurrent neural network (RNN). Based on these feature sequences, the trained RNN outputs weight coefficients, which are functionally equivalent to or used to calculate the Kalman gain in Kalman filtering. However, traditional Kalman gain relies on accurate noise covariance matrices Q and R, but these matrices are unknown or time-varying in complex cross-media environments. The innovation of this invention lies in that, instead of performing explicit matrix operations, it uses a data-driven approach, where a recurrent neural network, such as a GRU or LSTM, implicitly learns and outputs an optimal weight coefficient based on the input features to achieve the best fusion of predicted and observed values.

[0087] The trained trajectory prediction model uses the weight coefficients output by the recurrent neural network to fuse the predicted state parameters from the previous time point and the observed state parameters from the current time point, resulting in more accurate corrected predicted state parameters for the current time point. Subsequently, based on this corrected state, the predicted state for the next time point is calculated.

[0088] The trajectory prediction method for cross-medium vehicles provided in this application calculates multi-dimensional dynamic features such as observation differences, new and updated information, forward transfer differences, and forward update differences. It then utilizes a trained recurrent neural network to implicitly learn from these features and output optimal weight coefficients, achieving adaptive dynamic adjustment of the fusion ratio between model predictions and real-time observation data. Since these weight coefficients are obtained based on data-driven learning rather than relying on a pre-set, potentially mismatched, noise statistical model, they can more accurately respond to complex flow field changes and environmental disturbances during cross-medium flight, effectively overcoming the model mismatch problem. Ultimately, this mechanism significantly improves the intelligence and robustness of the state correction process, laying a reliable state estimation foundation for generating high-precision trajectories.

[0089] In some embodiments, in order to accurately calculate the predicted state parameters at the next time point, the trained trajectory prediction model has a trained state transition function, which, in detail 303, includes:

[0090] Step 401: Input the corrected predicted state parameters at the current time point into the trained state transition function, use the trained state transition function to calculate the predicted state parameters at the next time point, and output them.

[0091] In some embodiments, after obtaining the corrected optimal state estimate for the current time point, the trained trajectory prediction model calculates the predicted state parameters for the next time point using a trained state transition function f(•). For example, .

[0092] A state transition function is a mathematical model that describes how the system state evolves from the current moment to the next. In the context of cross-medium vehicles, this function is essentially its dynamic equation. The specific physical manifestation of the state transition function f(x) can be found in the aerial attitude equations and underwater state equations described in the above embodiments.

[0093] In classical Kalman filtering, the state transition function is a fixed mathematical model predetermined according to physical laws. Examples include the air attitude equation and the underwater state equation mentioned above. In the technical solution of this application, the trained state transition function can be obtained by training a preset state transition function. The trainable parameters of the preset state transition function can be certain empirical coefficients, which can be updated together with the parameters of the recurrent neural network using the gradient descent method.

[0094] For example, suppose the underwater equation of state contains an empirical coefficient Cd related to fluid resistance, which is a pre-defined constant in traditional models. However, in this invention, the empirical coefficient Cd related to fluid resistance can be regarded as a trainable parameter in the state transition function. Through learning from a large amount of data, the state prediction model can automatically adjust the value of Cd, making the prediction of the state transition function more accurate in complex flow fields across media.

[0095] The trajectory prediction method for cross-medium vehicles provided in this application deeply integrates the model-driven method based on physical laws and the data-driven method by using a trained state transition function to perform the prediction step. Since the parameters of the state transition function are optimized and calibrated based on real data during the training process, it not only retains the interpretability and extrapolation reliability of the physical model, but also significantly reduces the dependence on the prior accuracy of the model. It can effectively compensate for model mismatch errors caused by abrupt changes in the cross-medium flow field and changes in the center of mass. Therefore, the trained trajectory prediction model can more realistically reflect the dynamic evolution law of the vehicle, thereby providing a more accurate and reliable prior state estimate for subsequent update steps, and thus improving the accuracy and robustness of the trained trajectory prediction model.

[0096] In some embodiments, to obtain a more accurate trained trajectory prediction model, the trained trajectory prediction model has a trained observation function, and step 102 further includes:

[0097] Step 501: Input the corrected predicted state parameters at the current time point into the trained observation function, use the trained observation function to calculate the predicted observation state parameters at the next time point and output them;

[0098] Step 502: Obtain the observation status parameters of the cross-medium vehicle at the next time point;

[0099] Step 503: Calculate the loss value of the trained trajectory prediction model at the current time point based on the predicted observation parameters and the observation state parameters at the next time point.

[0100] Step 504: Update the parameters of the trained trajectory prediction model based on the loss value at the current time point.

[0101] In some embodiments, the trained state prediction model inputs the corrected predicted state parameters at the current time point into the trained observation function, and uses the trained observation function to calculate the predicted observed state parameters at the next time point. The trained observation function h(·) is a mathematical mapping used to convert the state parameters of a cross-medium vehicle, such as position and velocity, into predicted values ​​measured by sensors.

[0102] In some embodiments, when time actually progresses to t+1, the sensors on the vehicle will actually measure a sensing data point, such as position, velocity, acceleration, attitude angle, etc. This allows the acquisition of the observed state parameters of the transmedium vehicle at the next time point. The result calculated in step 501 and Compare the results and calculate the loss value of the trained trajectory prediction model at the current time point.

[0103] In some embodiments, the loss value can be calculated using the variational lower bound ELBO with a variational autoencoder as the loss function. ELBO can measure not only the reconstruction loss but also the overall loss. and The differences can also be used to constrain the distribution of latent variables inferred by the trained recurrent neural network through KL divergence, thus making it more suitable for online learning with small samples and avoiding overfitting.

[0104] For example, a trained recurrent neural network calculates the potential state variables of a cross-medium vehicle at the current time point based on the observation data sequence of the current time point and multiple historical time points. The potential state variables are a probabilistic description of the cognitive state of the cross-medium vehicle at the current time point, i.e., a complete probability distribution, which is used to characterize the best estimate of the state parameters of the cross-medium vehicle based on all existing observation data.

[0105] To perform probability calculations, a specific cognitive state sample is randomly selected from the probabilistic distribution of cognitive states at the current time point. This sample is then input into a trained state prediction model. The trained model uses a trained observation function to calculate and output reconstructed observation state parameters. These reconstructed parameters are a probabilistic guess, based on the current model parameters, of what the actual sensor readings should be.

[0106] To evaluate the generation accuracy of the observation function in the state prediction model, it is necessary to assess the extent to which the cognitive states extracted from the distribution inferred from the recurrent neural network can regenerate data that closely approximates the actual sensor readings through the observation function. If all reconstructed observations closely match the actual observations, the model is considered highly accurate. Therefore, by sampling multiple times and calculating the average expected value of the results, an assessment of the reconstruction accuracy is obtained.

[0107] To constrain and optimize the inference behavior of recurrent neural networks (RNNs), prevent their behavior from becoming extreme, and improve their generalization ability, it is necessary to evaluate the difference between the cognitive state distribution output by the RNN and an ideal distribution. The ideal distribution is a pre-defined, simple, and canonical state distribution. The ideal distribution is used to avoid overfitting the RNN to noisy training data. Therefore, the canonicality assessment of the cognitive distribution is obtained by calculating the KL divergence between the probabilistic distribution of the cognitive state at the current time point and the ideal distribution.

[0108] In summary, the difference between the reconstruction accuracy assessment and the cognitive distribution normalization assessment can be determined as the loss value of the trained trajectory prediction model at the current time point.

[0109] In some embodiments, a more accurate trained trajectory prediction model can be obtained by simultaneously adjusting the parameters of the recurrent neural network and the observation function in the state prediction model using the gradient descent method, so as to predict the trajectory of cross-medium navigation more accurately in the future.

[0110] The trajectory prediction method for cross-medium vehicles provided in this application, through the calculation of loss values ​​and the updating of parameters of the trained trajectory prediction model, enables the trained trajectory prediction model to continuously improve its state estimation accuracy after deployment, thereby matching the complex and dynamically changing state parameters of the cross-medium vehicle during operation, and avoiding the problem of decreased prediction accuracy caused by model mismatch, environmental disturbances and data scarcity.

[0111] In some embodiments, to further improve the accuracy of the trained trajectory prediction model, step 504 is further refined to include:

[0112] Step 601: Update the parameters of the trained recurrent neural network using gradient descent.

[0113] Step 602: Update the state transition coefficients of the trained state prediction model using gradient descent.

[0114] Gradient descent is an optimization algorithm that iteratively finds the minimum value of a function. It works by calculating the gradient (derivative) of the function with respect to the parameters and adjusting the parameters in the opposite direction of the gradient, thereby gradually reducing the function value.

[0115] The parameters of a trained recurrent neural network refer to the adjustable variables that constitute a recurrent neural network, such as GRU or LSTM, including the connection weights and bias terms between neurons in each layer.

[0116] The state transition coefficients of a trained state prediction model are adjustable variables embedded in the state transition function f(·). These coefficients are key parameters in the physics-based dynamic equations. They directly control the physical accuracy of the state prediction. For example, in the air attitude equations, the state transition coefficient might be a coefficient related to aerodynamic damping. In the underwater state equations, the state transition coefficients might include an empirical coefficient Cd related to fluid drag F, or a coefficient related to thruster efficiency P.

[0117] The trajectory prediction method for cross-medium vehicles provided in this application update the parameters of a trained recurrent neural network using gradient descent. This allows for targeted optimization of the weights and biases used to calculate the Kalman gain within the recurrent neural network, thereby dynamically adjusting the data fusion strategy and improving its adaptability to complex noise environments across media in real time. Simultaneously, by updating the state transition coefficients of the trained state prediction model using gradient descent, key parameters in the vehicle's dynamic equations can be fine-tuned online, effectively compensating for model mismatch caused by medium switching and sudden changes in the flow field.

[0118] The trajectory prediction method for cross-medium vehicles provided in this application provides an explicit and efficient optimization path for the online learning of the trained trajectory prediction model through a dual update mechanism of the parameters of the trained recurrent neural network and the state transition coefficients of the trained state prediction model, enabling the trained state prediction model to accurately and reliably predict the trajectory of cross-medium vehicles.

[0119] In some embodiments, to obtain an accurate trained trajectory prediction model, before step 101, the method further includes:

[0120] Step 701: Obtain training data, which includes the observation state parameters of the cross-medium vehicle at multiple consecutive historical time points;

[0121] Step 702: Obtain a preset trajectory prediction model. The preset trajectory prediction model includes a preset state prediction model and a preset recurrent neural network. The preset state prediction model has a preset state transition function and a preset observation function.

[0122] Step 703: Train the preset trajectory prediction model using training data until the loss value of the preset trajectory prediction model reaches a preset threshold, and obtain the trained trajectory prediction model.

[0123] In some embodiments, the observed state parameters of the cross-medium vehicle at multiple consecutive historical time points are time-series data collected by sensors installed on the cross-medium vehicle, such as acceleration sequences measured by IMU, temperature sequences measured by temperature sensors, and pressure sensor reading sequences.

[0124] The preset trajectory prediction model includes a preset state prediction model and a preset recurrent neural network. The preset state prediction model is constructed based on the physical state equations of the cross-medium vehicle, such as the underwater state equation and the aerial attitude equation mentioned above. The preset recurrent neural network is a neural network with a defined structure, such as a two-layer GRU network, but with randomly initialized parameters.

[0125] In some embodiments, training a preset trajectory prediction model using training data is an iterative process, and each round of training can be as follows:

[0126] Forward Propagation and State Estimation: A continuous observation sequence is extracted from the training data and input into a pre-defined trajectory prediction model. The pre-defined trajectory prediction model uses a pre-defined state transition function in a pre-defined state prediction model to predict the state, obtaining an observation prediction sequence. Then, at least one of the following is calculated based on the observation sequence and the observation prediction sequence: observation difference sequence, newly updated information sequence, forward transition difference sequence, and forward update difference sequence. A pre-defined recurrent neural network is used to calculate the weight coefficients, i.e., the Kalman gain, based on at least one of the following: observation difference sequence, newly updated information sequence, forward transition difference sequence, and forward update difference sequence. Finally, the prediction and observation are fused to obtain the state estimate value at each time point in the sequence.

[0127] Observation Prediction and Loss Calculation: Based on the state estimates at each time point in the above sequence, the corresponding observation sequence is predicted using a preset observation function in the preset state prediction model. Subsequently, the loss value of the evidence lower bound ELBO loss function is calculated through variational inference: ELBO loss function = reconstruction loss - KL divergence. Here, the reconstruction loss is the negative log-likelihood between the model-predicted observed data and the actual training data, and the KL divergence is the difference between the latent variable distribution inferred by the preset recurrent neural network and the preset prior distribution.

[0128] Backpropagation and parameter update: Based on the calculated ELBO loss, backpropagation is performed using gradient descent, while updating all internal parameters of the preset recurrent neural network and the internal coefficients of the preset state transition function and preset observation function in the preset state prediction model.

[0129] Repeat the above iterative process until the average ELBO loss of the preset trajectory prediction model on the entire training set reaches the preset threshold. At this point, the trained trajectory prediction model is obtained.

[0130] The trajectory prediction method for cross-medium vehicles provided in this application uses a preset trajectory prediction model, including a preset state prediction model and a preset recurrent neural network. The preset state prediction model has a preset state transition function and a preset observation function. The preset recurrent neural network is used to calculate the Kalman gain coefficient, which can replace the Kalman filter, which is sensitive to noise statistics. n It retains the structure and interpretability of Kalman filtering, and improves the accuracy of cross-medium vehicle trajectory prediction.

[0131] In some embodiments, to further improve the accuracy of cross-medium vehicle trajectory prediction, step 703 is further refined to include:

[0132] Step 801: Input the continuous sequence of observed state parameters from the training data into the preset trajectory prediction model;

[0133] Step 802: Using a preset state prediction model, calculate the predicted state parameter sequence and the predicted observed state parameter sequence corresponding to the observed state parameter sequence;

[0134] Step 803: Calculate the observation difference sequence, the new update information sequence, the forward transfer difference sequence, and the forward update difference sequence based on the observation state parameter sequence, the predicted state parameter sequence, and the predicted observation state parameter sequence;

[0135] Step 804: Using a pre-defined recurrent neural network, the variational posterior distribution of the latent variables at the latest time point in the observed state parameter sequence is calculated based on the observation difference sequence, the newly updated information sequence, the forward transition difference sequence, and the forward update difference sequence.

[0136] Step 805: Sample the variational posterior distribution to obtain latent variable samples;

[0137] Step 806: Using the decoder of the preset state prediction model, generate the reconstructed observation data distribution corresponding to the latent variable samples;

[0138] Step 807: Calculate the loss value of the preset trajectory prediction model based on the latest time point observation state parameters in the reconstructed observation data distribution and observation state parameter sequence.

[0139] Step 808: Based on the loss value, update the parameters of the preset state prediction model and the parameters of the preset recurrent neural network.

[0140] In some embodiments, latent variables refer to hidden information learned by a recurrent neural network and used for computation. Latent variables represent hidden information such as noise statistics and model mismatch required for state estimation of cross-medium vehicles in complex environments.

[0141] In some embodiments, the continuous sequence of observed state parameters in the training data x After inputting the preset trajectory prediction model, for time point k, the observation state parameters at time point k are used. Update the state estimation at time point k Using the state transition function f(•) from predict state of time Using the observation function h(•) from Predict the observed state parameters at time k+1 The predicted state parameter sequence and the predicted observed state parameter sequence are obtained. For time point k=0, i.e., the initial time point, the state estimate at time point k=0 is updated using the initial state parameters.

[0142] Then, based on the observed state parameter sequence, the predicted state parameter sequence, and the predicted observed state parameter sequence, at least one of the following sequences is calculated: the observed difference sequence, the newly updated information sequence, the forward transition difference sequence, and the forward update difference sequence. Among these, the observed difference... New Update Information Forward transfer difference Forward update difference The forward transition difference is the difference between two consecutive posterior state estimates, and the forward update difference is the difference between the posterior and prior state estimates. The observation difference and forward transition difference contain information about the state transition process, while the new update information and the forward update difference contain information about the uncertainty of the state estimate. The difference operation removes predictable components; therefore, the difference time series is mainly affected by the statistical characteristics of the noise to be learned.

[0143] The observation difference sequence, the newly updated information sequence, the forward transition difference sequence, and the forward update difference sequence are input into a pre-defined recurrent neural network. Based on the temporal characteristics of the input, the pre-defined recurrent neural network infers and outputs the latent variables at the current time step. z variational posterior distribution , where the variational posterior distribution is a probability distribution.

[0144] For variational posterior distribution Sampling is performed to obtain latent variable samples. The decoder of the preset state prediction model is used to generate the reconstructed observation data distribution corresponding to the latent variable samples.

[0145] Using the preset observation function h(·) of the preset state prediction model, the reconstructed observation data distribution corresponding to the latent variable samples is generated. .in, This represents the parameters of the preset trajectory prediction model.

[0146] Based on the reconstructed observation data distribution and the latest time point observation state parameters in the observation state parameter sequence, calculate the loss value of the preset trajectory prediction model; based on the loss value, update the parameters of the preset state prediction model and the parameters of the preset recurrent neural network.

[0147] The motion trajectory prediction method for cross-medium vehicles provided in this application embodiment enables the latent variables learned by the recurrent neural network to follow the laws of physical observation, thereby improving the quality of latent variable representation and enhancing the physical consistency and generalization ability of the trained motion trajectory prediction model.

[0148] In some embodiments, step 807 is further refined to include: calculating the loss value based on a variational lower bound loss function, wherein the variational lower bound loss function is:

[0149] ,

[0150] in, As the variational lower bound, x For the observed state parameter sequence, The parameters for the preset trajectory prediction model, For variational posterior parameters, z As latent variables, To reconstruct the expected value of the loss term, For variational posterior distribution, π( z ) is the prior distribution. Let KL divergence be a metric.

[0151] The variational lower bound is the training optimization objective of the variational autoencoder, which needs to be maximized to balance reconstruction accuracy and distribution regularization. It is the core optimization function of the cross-medium vehicle trajectory prediction model, adapting to small sample sizes and nonlinear scenarios to improve prediction accuracy. The input observation data consists of multi-dimensional state data (position, velocity, acceleration, tilt angle, flight altitude, etc.) of the cross-medium vehicle collected by its sensors.

[0152] Step 808 is further refined to include: maximizing the variational lower bound ELBO by using gradient descent, and updating the parameters of the preset state prediction model and the preset recurrent neural network.

[0153] The method for predicting the motion trajectory of a cross-medium vehicle provided in this application reuses the observation function in the state prediction model as the decoder of the VAE during the training phase, so that the latent variables learned by the recurrent neural network must follow the laws of physical observation, thereby improving the physical consistency and generalization ability of the trained trajectory prediction model, and enabling the trained trajectory prediction model to accurately predict the motion trajectory of the cross-medium vehicle.

[0154] Figure 2 This is a schematic diagram of the iterative state estimation process based on Kalman filtering. (Example:) Figure 2 As shown, the Kalman filter achieves continuous optimization of the dynamic system state through a closed-loop mechanism of measurement, updating, and prediction. When predicting the trajectory of a cross-medium vehicle, the state prediction function f(•) and the state transition function h(•) are known but may contain approximation errors, while the covariance matrices Q and R are unknown. Therefore, this application proposes learning the Kalman gain through a recurrent neural network and integrating it into the overall Kalman filtering process.

[0155] Figure 3 This is a schematic diagram of the structure of a preset trajectory prediction model provided in one embodiment of this application. Figure 4 This is a schematic diagram of the structure of a preset recurrent neural network provided in one embodiment of this application. Figure 3 and Figure 4As shown, the embodiment of this application provides a similar approach to the traditional extended Kalman filter, where at each time step t, the preset trajectory prediction model estimates the trajectory through two steps: prediction and update.

[0156] A three-stage closed-loop structure is adopted: initialization → measurement → update → prediction. Each step forms a complete iterative loop through data flow arrows, reflecting the temporal recursiveness. The core functions are described below:

[0157] Initialization: Set the initial state ( ) and uncertainty ( Benchmark value

[0158] Measurement: Receiving sensor data ( and measurement noise characteristics

[0159] The correction stage is achieved through Kalman gain ( ) calculate and merge observations to update the current state estimate ( ) and uncertainty ( )

[0160] Prediction: Extrapolating the state at the next time step based on the system dynamics model ( ) and uncertainty ( )

[0161] The preset trajectory prediction model provided in this application manages both uncertainty and state estimation and measurement uncertainty simultaneously. It has a time-domain recursive mechanism, which can achieve time synchronization through unit delay. Based on the idea of ​​data fusion, it combines dynamic model prediction and measured data correction, which can more accurately predict the motion trajectory of cross-medium vehicles.

[0162] In one embodiment of this application, in the prediction step, since the noise statistics Q and P of the collected environmental system are unknown, the second moment of the covariance cannot be calculated using the traditional Kalman prediction model. Therefore, the prior state estimate is calculated using a known or approximate state transition function f(•), and the predicted observation is calculated using the known h(•). The specific function expression is as follows:

[0163] ,

[0164] .

[0165] In the update step, the state transition and observation functions of the Kalman filter are parameterized using a neural network. The specific updated parameters and calculation formulas are as follows:

[0166] ① Calculate the new system measurements:

[0167]

[0168] ② Use RNN to estimate Kalman gain:

[0169]

[0170] ③ Use nonlinear neural networks to parameterize and optimize the state transition update equations:

[0171]

[0172] ④ Optimize the state prediction equation using a probabilistic statistical model:

[0173] .

[0174] Model-driven Kalman filters and their variants calculate the Kalman gain based on fundamental statistical properties. To achieve this calculation through learning, the neural network needs to be fed input features containing the information required for the Kalman gain calculation. (Kalman gain value) Relying on the statistical properties of observations and state processes, at each time step t, the recurrent neural network needs to be input with observations... and state estimation Characteristics of statistical information. The following quantities, which are related to unknown statistical relationships in state-space models, can be used as input features for recurrent neural networks:

[0175] F1: Observational difference: ;

[0176] F2: New Update Information: ;

[0177] F3: Forward transfer difference The forward transition difference is the difference between two consecutive posterior state estimates, and the available feature at time t is... ;

[0178] F4: Forward update of difference The forward update difference is the difference between the posterior state estimate and the prior state estimate. The available features at time t are... .

[0179] In some embodiments, this application also provides a hybrid state estimation algorithm combining deep learning and traditional Kalman filtering. The optimization for model training lies in using a variational autoencoder to train the loss function, i.e., maximizing the variational lower bound (ELBO), instead of minimizing the mean square error method used in traditional Kalman prediction models. The specific implementation steps are as follows:

[0180] ,

[0181] in, To reconstruct the loss, The variational posterior and prior KL divergence is decomposed into time-step local KL terms, and the gradient is estimated by Monte Carlo sampling.

[0182] This optimization model is a state-space model containing a sequence of latent states, corresponding observation sequences, and external action sequences. The evolution of the latent states is a nonlinear process, with neural networks controlling the mean of the states and the covariance of Gaussian noise, respectively; observations are generated by another neural network. The model supports arbitrary forms of nonlinear transformations, thus enabling the capture of complex temporal dynamics. This improves the accuracy of predictions for cross-medium vehicles.

[0183] In some embodiments, the trained state prediction model and the trained recurrent neural network in the trained trajectory prediction model can be configured in a modular parallel manner. Each tracked second-order statistical moment is assigned an independent GRU unit, with the structure of the GRU unit and the fully connected layers. The network consists of three cascaded GRU layers, each equipped with dedicated input and output fully connected layers. Through the non-standard interconnection between the GRU and the fully connected layers, the formula of the state-space model and the computational flow of the model-driven Kalman filter are more closely aligned, resulting in lower abstraction, a more limited range of learnable mappings, but fewer trainable parameters required.

[0184] The motion trajectory prediction method for cross-medium vehicles provided in this application combines dynamic operating environment monitoring and motion trajectory prediction to identify and adjust potential safety risks in real time, such as the impact of wind direction changes and collisions with obstacles. Through continuous analysis of the cross-medium vehicle's state and surrounding environment, real-time decisions can be made, accurately predicting the next second's position and state, performing path planning, and making decisions in advance.

[0185] The motion trajectory prediction method for cross-medium vehicles provided in this application, by combining deep learning algorithms, can not only process collected linear environmental information, but also be applied to nonlinear and model mismatch scenarios. It does not rely on linearization and can more accurately predict the motion trajectory of cross-medium vehicles.

[0186] The trajectory prediction method for cross-medium vehicles provided in this application adopts a hybrid model-driven and data-driven approach, retains the structure and interpretability of Kalman filtering, and replaces the Kalman filter, which is sensitive to noise statistics, with an RNN. n In the computational part, both the physical models f(•) and h(•) are utilized, as well as the noise characteristics of the data, to improve the accuracy of predicting the flight trajectory of cross-medium vehicles.

[0187] The trajectory prediction method for cross-medium vehicles provided in this application uses RNN to implicitly learn noise characteristics, avoiding numerical instability and model mismatch problems. It does not require noise statistics, avoids matrix inversion, has high computational efficiency, and is suitable for embedding in cross-medium vehicle equipment.

[0188] Figure 5 This is a schematic diagram of the motion trajectory prediction device for a cross-medium vehicle provided in one embodiment of this application. Figure 5 As shown, the trajectory prediction device 50 for a cross-medium vehicle includes:

[0189] The acquisition module 51 is used to acquire the observation status parameters of the cross-medium vehicle at the current time point;

[0190] The prediction module 52 is used to input the observed state parameters at the current time point into the trained trajectory prediction model, and use the trained trajectory prediction model to output the predicted state parameters of the cross-medium vehicle at the next time point. The trained trajectory prediction model includes a trained state prediction model and a trained recurrent neural network. The trained state prediction model includes the predicted state parameters of the cross-medium vehicle at the previous time point. The trained state prediction model is used to fuse the predicted state parameters at the previous time point and the observed state parameters at the current time point to obtain the corrected predicted state parameters at the current time point, and calculate the predicted state at the next time point based on the corrected predicted state parameters at the current time point. The trained recurrent neural network is used to calculate the weight coefficient between the predicted state parameters at the previous time point and the observed state parameters at the current time point during fusion.

[0191] The determination module 53 is used to determine the motion trajectory of the cross-medium vehicle based on the predicted state parameters of the cross-medium vehicle at the next time point.

[0192] In some embodiments, the determining module 53 is specifically used for:

[0193] Analyze the corrected predicted state parameters at the current time point to obtain the corrected position information of the cross-medium vehicle at the current time point;

[0194] Analyze the predicted state parameters at the next time point to obtain the predicted position information of the cross-medium vehicle at the next time point;

[0195] The trajectory is determined based on the corrected location information and the predicted location information.

[0196] In some embodiments, the trajectory prediction device 50 for a cross-medium vehicle further includes:

[0197] The first calculation module is used to calculate at least one of the following: the observation difference at the current time point, the new update information at the current time point, the forward transition difference at the current time point, and the forward update difference at the current time point; wherein, the observation difference at the current time point is the difference between the observation state parameters at the previous time point and the observation state parameters at the current time point, the new update information at the current time point is the difference between the observation state parameters at the current time point and the observation prediction state parameters at the current time point, the forward transition difference at the current time point is the difference between the prediction state parameters at the current time point and the prediction state parameters at the previous time point, and the forward update difference at the current time point is the difference between the corrected prediction state parameters at the current time point and the prediction state parameters at the current time point;

[0198] The second calculation module is used to input at least one of the observation difference at the current time point, the new update information at the current time point, the forward transition difference at the current time point, and the forward update difference at the current time point into the trained recurrent neural network, and to use the trained recurrent neural network to output weight coefficients.

[0199] The third calculation module is used to obtain the corrected predicted state parameters at the current time point by fusing the predicted state parameters of the previous time point and the observed state parameters at the current time point based on the weight coefficients of the trained trajectory prediction model, and to calculate and output the predicted state parameters at the next time point based on the corrected predicted state parameters at the current time point.

[0200] In some embodiments, the trained trajectory prediction model has a trained state transition function, and the third calculation module is specifically used to: input the corrected predicted state parameters at the current time point into the trained state transition function, calculate the predicted state parameters at the next time point using the trained state transition function, and output them.

[0201] In some embodiments, the trained trajectory prediction model has a trained observation function, and the trajectory prediction device 50 for the cross-medium vehicle further includes:

[0202] The fourth calculation module is used to input the corrected predicted state parameters at the current time point into the trained observation function, use the trained observation function to calculate the predicted observation state parameters at the next time point, and output them.

[0203] The second acquisition module is used to acquire the observation status parameters of the cross-medium vehicle at the next time point;

[0204] The fifth calculation module is used to calculate the loss value of the trained trajectory prediction model at the current time point based on the predicted observation parameters and the observation state parameters at the next time point.

[0205] The update module is used to update the parameters of the trained trajectory prediction model based on the loss value at the current time point.

[0206] In some embodiments, the update module is specifically used for:

[0207] The parameters of the trained recurrent neural network are updated using gradient descent.

[0208] The state transition coefficients of the trained state prediction model are updated using the gradient descent method.

[0209] In some embodiments, the trajectory prediction device 50 for a cross-medium vehicle further includes:

[0210] The second acquisition module is used to acquire training data, which includes the observation state parameters of the cross-medium vehicle at multiple consecutive historical time points.

[0211] The third acquisition module is used to acquire a preset trajectory prediction model. The preset trajectory prediction model includes a preset state prediction model and a preset recurrent neural network. The preset state prediction model has a preset state transition function and a preset observation function.

[0212] The training module is used to train a preset trajectory prediction model using training data until the loss value of the preset trajectory prediction model reaches a preset threshold, thus obtaining a trained trajectory prediction model.

[0213] In some embodiments, the training module is specifically used for:

[0214] Input the continuous sequence of observed state parameters from the training data into the preset trajectory prediction model;

[0215] Using a pre-defined state prediction model, the predicted state parameter sequence and the predicted observed state parameter sequence are calculated to correspond to the observed state parameter sequence.

[0216] Calculate the observation difference sequence, the new update information sequence, the forward transfer difference sequence, and the forward update difference sequence based on the observed state parameter sequence, the predicted state parameter sequence, and the predicted observed state parameter sequence;

[0217] A pre-defined recurrent neural network is used to calculate the variational posterior distribution of the latent variables at the latest time point in the observed state parameter sequence based on the observation difference sequence, the newly updated information sequence, the forward transition difference sequence, and the forward update difference sequence;

[0218] Sample the variational posterior distribution to obtain latent variable samples;

[0219] The pre-defined observation function of the pre-defined state prediction model is used to generate the reconstructed observation data distribution corresponding to the latent variable samples;

[0220] Based on the reconstructed observation data distribution and the observation state parameters at the latest time point in the observation state parameter sequence, calculate the loss value of the preset trajectory prediction model;

[0221] Based on the loss value, update the parameters of the preset state prediction model and the preset recurrent neural network.

[0222] In some embodiments, the training module is further configured to:

[0223] The loss value is calculated based on the variational lower bound loss function, which is:

[0224] ,

[0225] in, As the variational lower bound, For the observed state parameter sequence, The parameters for the preset trajectory prediction model, For variational posterior parameters, z As latent variables, To reconstruct the expected value of the loss term, Let z be the variational posterior distribution, and π(z) be the prior distribution. Let KL divergence be a metric.

[0226] The variational lower bound ELBO is maximized using gradient descent, and the parameters of the preset state prediction model and the preset recurrent neural network are updated.

[0227] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in one embodiment of this application. For example... Figure 6 As shown, the electronic device may include a processor 61 and a memory 62 storing computer program instructions.

[0228] Specifically, the processor 61 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0229] Memory 62 may include mass storage for data or instructions. For example, and not limitingly, memory 62 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 62 may include removable or non-removable (or fixed) media. Where appropriate, memory 62 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 62 is non-volatile solid-state memory.

[0230] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0231] The processor 61 reads and executes computer program instructions stored in the memory 62 to implement any of the motion trajectory prediction methods for cross-medium vehicles in the above embodiments.

[0232] In one example, the device may also include a communication interface 63 and a bus 64. For example, Figure 6 As shown, the processor 61, memory 62, and communication interface 63 are connected via bus 64 and communicate with each other.

[0233] Communication interface 63 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0234] Bus 64 includes hardware, software, or both, that couples components of an electronic device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 64 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0235] Furthermore, in conjunction with the trajectory prediction method for cross-medium vehicles in the above embodiments, this application embodiment can provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the trajectory prediction methods for cross-medium vehicles in the above embodiments.

[0236] This application also provides a computer program product, including a computer program that, when executed, implements any of the motion trajectory prediction methods for cross-medium vehicles described in the above embodiments.

[0237] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0238] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0239] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0240] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0241] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A method for predicting the trajectory of a cross-medium vehicle, characterized in that, include: Obtain the observation status parameters of the cross-medium vehicle at the current time point; The observed state parameters at the current time point are input into the trained trajectory prediction model, and the trained trajectory prediction model is used to output the predicted state parameters of the cross-medium vehicle at the next time point. The trained trajectory prediction model includes a trained state prediction model and a trained recurrent neural network. The trained state prediction model includes the predicted state parameters of the cross-medium vehicle at the previous time point. The trained state prediction model is used to fuse the predicted state parameters of the previous time point and the observed state parameters at the current time point to obtain the corrected predicted state parameters at the current time point, and to calculate the predicted state at the next time point based on the corrected predicted state parameters at the current time point. The trained recurrent neural network is used to calculate the weight coefficients between the predicted state parameters of the previous time point and the observed state parameters at the current time point during fusion. The trajectory of the cross-medium vehicle is determined based on the predicted state parameters of the cross-medium vehicle at the next time point.

2. The method according to claim 1, characterized in that, Determining the trajectory of the cross-medium vehicle based on its predicted state parameters at the next time point includes: The corrected predicted state parameters at the current time point are analyzed to obtain the corrected position information of the cross-medium vehicle at the current time point. Analyze the predicted state parameters at the next time point to obtain the predicted position information of the cross-medium vehicle at the next time point; The motion trajectory is determined based on the corrected location information and the predicted location information.

3. The method according to claim 1, characterized in that, The trained state prediction model further includes the observed predicted state parameters of the cross-medium vehicle at the current time point and the observed state parameters at the previous time point. The step of using the trained trajectory prediction model to output the predicted state parameters of the cross-medium vehicle at the next time point includes: Calculate at least one of the following: the observation difference at the current time point, the new update information at the current time point, the forward transition difference at the current time point, and the forward update difference at the current time point; wherein, the observation difference at the current time point is the difference between the observation state parameter at the previous time point and the observation state parameter at the current time point, the new update information at the current time point is the difference between the observation state parameter at the current time point and the observation prediction state parameter at the current time point, the forward transition difference at the current time point is the difference between the prediction state parameter at the current time point and the prediction state parameter at the previous time point, and the forward update difference at the current time point is the difference between the corrected prediction state parameter at the current time point and the prediction state parameter at the current time point; The training recurrent neural network takes at least one of the observation difference at the current time point, the new update information at the current time point, the forward transition difference at the current time point, and the forward update difference at the current time point as input, and outputs the weight coefficients using the training recurrent neural network. The trained trajectory prediction model is used to fuse the predicted state parameters of the previous time point and the observed state parameters of the current time point based on the weight coefficients to obtain the corrected predicted state parameters of the current time point. The predicted state parameters of the next time point are then calculated and output based on the corrected predicted state parameters of the current time point.

4. The method according to claim 3, characterized in that, The trained trajectory prediction model has a trained state transition function. The step of calculating and outputting the predicted state parameters for the next time point based on the corrected predicted state parameters for the current time point includes: The corrected predicted state parameters at the current time point are input into the trained state transition function, and the predicted state parameters at the next time point are calculated and output using the trained state transition function.

5. The method according to claim 1, characterized in that, The trained trajectory prediction model has a trained observation function. After inputting the observation state parameters at the current time point into the trained trajectory prediction model, the method further includes: The corrected predicted state parameters at the current time point are input into the trained observation function, and the predicted observation state parameters at the next time point are calculated and output using the trained observation function. Obtain the observation status parameters of the cross-medium vehicle at the next time point; Based on the predicted observation parameters and the observation state parameters at the next time point, the loss value of the trained trajectory prediction model at the current time point is calculated. The parameters of the trained trajectory prediction model are updated based on the loss value at the current time point.

6. The method according to claim 5, characterized in that, The step of updating the parameters of the trained trajectory prediction model based on the loss value at the current time point includes: The parameters of the trained recurrent neural network are updated using gradient descent. The state transition coefficients of the trained state prediction model are updated using the gradient descent method.

7. The method according to claim 1, characterized in that, Before inputting the observation state parameters at the current time point into the trained trajectory prediction model, the method further includes: Acquire training data, which includes the observation state parameters of the cross-medium vehicle at multiple consecutive historical time points; A preset trajectory prediction model is obtained, the preset trajectory prediction model includes a preset state prediction model and a preset recurrent neural network, the preset state prediction model has a preset state transition function and a preset observation function; The training data is used to train the preset trajectory prediction model until the loss value of the preset trajectory prediction model reaches a preset threshold, thus obtaining the trained trajectory prediction model.

8. The method according to claim 7, characterized in that, The step of training the preset trajectory prediction model using the training data until the loss value of the preset trajectory prediction model reaches a preset threshold to obtain a trained trajectory prediction model includes: The continuous sequence of observed state parameters from the training data is input into the preset trajectory prediction model; Using the preset state prediction model, calculate the predicted state parameter sequence and the predicted observed state parameter sequence corresponding to the observed state parameter sequence; Calculate the observation difference sequence, the new update information sequence, the forward transfer difference sequence, and the forward update difference sequence based on the observation state parameter sequence, the predicted state parameter sequence, and the predicted observation state parameter sequence; The preset recurrent neural network is used to calculate the variational posterior distribution of the latent variables at the latest time point in the observation state parameter sequence based on the observation difference sequence, the new update information sequence, the forward transition difference sequence, and the forward update difference sequence; The variational posterior distribution is sampled to obtain latent variable samples; The pre-defined observation function of the pre-defined state prediction model is used to generate the reconstructed observation data distribution corresponding to the latent variable samples; Based on the reconstructed observation data distribution and the observation state parameters at the latest time point in the observation state parameter sequence, calculate the loss value of the preset trajectory prediction model; Based on the loss value, update the parameters of the preset state prediction model and the parameters of the preset recurrent neural network.

9. The method according to claim 8, characterized in that, The step of calculating the loss value of the preset trajectory prediction model based on the observation state parameters at the latest time point in the reconstructed observation data distribution and observation state parameter sequence includes: The loss value is calculated based on the variational lower bound loss function, which is: ， in, As the variational lower bound, x The observed state parameter sequence, The parameters for the preset trajectory prediction model, For variational posterior parameters, z As latent variables, To reconstruct the expected value of the loss term, For variational posterior distribution, π( z ) is the prior distribution. Let KL divergence be a metric. The step of updating the parameters of the preset state prediction model and the parameters of the preset recurrent neural network based on the loss value includes: The variational lower bound is maximized using gradient descent, thereby updating the parameters of the preset state prediction model and the preset recurrent neural network.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1-9.

Images