A multi-mode trajectory prediction electro-optical tracking system
By combining multimodal data fusion and a Gaussian process latent force model with stochastic differential equations and Schrödinger bridge inference, the instability problem of single-mode observation in photoelectric tracking technology is solved, and high-precision trajectory prediction and stable output are achieved in complex environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHANGHAI DONGGU AVIATION TECH CO LTD
- Filing Date
- 2026-03-13
- Publication Date
- 2026-06-19
AI Technical Summary
Existing photoelectric tracking technology relies on single-modal observation, which is susceptible to image quality, environmental interference and sensor noise. This results in unstable trajectory prediction in dynamic scenarios, and multimodal data is difficult to fuse effectively, limiting prediction accuracy and robustness.
By integrating multimodal observation data such as visible light, infrared, and inertial measurement, and combining the Gaussian process potential force model and stochastic differential equations, a time-continuous trajectory distribution is generated through Schrödinger bridge inference, providing target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set.
It improves the accuracy and stability of trajectory prediction under complex scenarios and uncertain conditions, achieves strong multimodal fusion capabilities, excellent prediction continuity, and provides reliable target trajectory output.
Smart Images
Figure CN121834141B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection and continuous tracking, and more particularly to a photoelectric tracking system for multi-mode trajectory prediction. Background Technology
[0002] In the field of photoelectric tracking, target trajectory prediction is crucial for achieving high-precision target tracking and control. However, existing technologies largely rely on single-modal observations, making them susceptible to the effects of imaging quality, environmental interference, and sensor noise, resulting in unstable trajectory prediction results in dynamic scenarios. Furthermore, the temporal and spatial differences in multimodal data make it difficult for existing methods to achieve effective fusion, limiting prediction accuracy and robustness.
[0003] In terms of trajectory modeling methods, traditional dynamic models and statistical filtering algorithms often assume that the trajectory evolution process conforms to a simplified linear or Gaussian distribution, which is difficult to adapt to complex motion patterns and uncertainty propagation, and cannot guarantee the rationality of the prediction results in terms of boundary constraints and global temporal consistency. This leads to a significant increase in trajectory prediction bias in complex environments such as occlusion, noise enhancement, or multipath interference.
[0004] Furthermore, existing probabilistic inference and candidate trajectory generation methods generally suffer from a lack of simplistic computational approaches, failing to adequately represent the uncertainties under multimodal inputs and lacking the ability to model predicted trajectories using multiple hypotheses. This results in insufficient coverage of prediction results and low matching between candidate solutions and true trajectories in practical applications, thus limiting the stability of downstream tracking and control modules.
[0005] Therefore, how to provide a photoelectric tracking system with multi-mode trajectory prediction is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0006] One objective of this invention is to propose a multi-mode trajectory prediction photoelectric tracking system. This invention integrates multi-modal observation data, including visible light, infrared, and inertial measurement data, constructs a latent force prior using a Gaussian process latent force model, introduces stochastic differential equations to establish a continuous-time reference diffusion process, and generates a time-continuous trajectory distribution using Schrödinger bridge inference under boundary constraints. Furthermore, it obtains the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate sets through conditional probability inference. This invention can improve trajectory prediction accuracy and stability under complex scenarios and uncertainties, possessing advantages such as strong multi-modal fusion capability, excellent prediction continuity, and comprehensive candidate solution coverage.
[0007] A photoelectric tracking system for multi-mode trajectory prediction according to an embodiment of the present invention includes:
[0008] The multimodal observation and acquisition module is used to generate a set of multimodal observation sequences and a set of timestamps, and to extract metadata such as device attitude parameters, signal-to-noise ratio, occlusion rate, and imaging intrinsic parameters.
[0009] The observation preprocessing module is used to generate a standardized observation set and temperature scheduling parameters, in which modal quality weighting is achieved by combining confidence sequences.
[0010] The boundary distribution modeling module is used to generate a set of boundary distribution constraints, including a multi-time boundary distribution set, boundary distribution index and time mapping relationship;
[0011] The latent force modeling module is used to build a latent force model of a Gaussian process and generate the latent force distribution and kernel parameter set;
[0012] The diffusion process modeling module is used to generate a reference diffusion process parameter set using a stochastic differential equation modeling method.
[0013] The Schrödinger Bridge Inference Module is used to perform Schrödinger Bridge Inference under the constraints of the boundary distribution constraint set and the reference diffusion process parameter set to generate a time-continuous trajectory distribution.
[0014] The trajectory inference output module is used to generate target trajectory coordinate sequences, covariance sequences, and multiple hypothesis candidate sets based on time-continuous trajectory distributions.
[0015] Optionally, modules can be integrated using the following methods:
[0016] Collect raw observation data, generate a multimodal observation sequence set, and extract metadata such as device attitude parameters, signal-to-noise ratio, and occlusion rate to generate a timestamp set.
[0017] Preprocessing is performed on the multimodal observation sequence set based on the timestamp set, and the confidence sequence of each mode is calculated by combining the signal-to-noise ratio and occlusion rate metadata to generate a standardized observation set and temperature scheduling parameters;
[0018] By combining a standardized observation set, a confidence sequence, and a timestamp set, a boundary distribution constraint set is generated using a method based on weighted probability density estimation.
[0019] Based on the standardized observation set and equipment attitude parameters, a Gaussian process potential force model is established, the dynamic structure and potential force prior are determined, the kernel function is selected and the kernel hyperparameters are calibrated, and the potential force distribution and kernel parameter set are generated.
[0020] Based on the potential force distribution and kernel parameter set, a continuous-time reference diffusion process is constructed using a stochastic differential equation modeling method. The drift term and diffusion intensity are determined, and a reference diffusion process parameter set is generated.
[0021] Using the boundary distribution constraint set and the reference diffusion process parameter set as constraints, Schrödinger bridge inference is performed, and the bridge potential function pair and the controlled drift field are solved by the iterative proportional fitting method. The time-continuous trajectory distribution is generated by combining the confidence sequence and temperature scheduling parameters.
[0022] Based on the time-continuous trajectory distribution, a conditional probability inference method is used to generate the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set.
[0023] Optionally, the generation of the multimodal observation sequence set and timestamp set includes:
[0024] The raw observation data collected by the visible light sensor and infrared sensor are timestamped and synchronized, and the raw observation data collected by the inertial measurement unit is timestamped to generate a set of timestamped multimodal observation sequences.
[0025] The device attitude parameters are calculated based on the raw observation data from the inertial measurement unit, and the device attitude parameters are correlated with the timestamps of the multimodal observation sequence set.
[0026] The signal-to-noise ratio and occlusion rate metadata are calculated based on the raw observation data from the visible light sensor and the infrared sensor, and the signal-to-noise ratio and occlusion rate metadata are mapped to the timestamps of the multimodal observation sequence set.
[0027] Call the pre-stored imaging intrinsic parameters, match the timestamps of the imaging intrinsic parameters with the timestamps of the multimodal observation sequence set, and output the timestamp set.
[0028] Optionally, the generation of the standardized observation set and temperature scheduling parameters includes:
[0029] Denoising processing is performed on the multimodal observation sequence set based on the timestamp set, and the imaging data of the visible light sensor and infrared sensor and the measurement data of the inertial measurement unit are timestamp aligned according to a unified time reference;
[0030] Based on imaging intrinsic parameters, geometric correction and photometric normalization are performed on the imaging data of visible light sensors and infrared sensors;
[0031] Asynchronous sampling alignment is completed under the constraints of the timestamp set, mapping the multimodal observation sequence set to a unified time axis, and generating a comparable multimodal observation frame sequence;
[0032] The confidence sequence of each modality is calculated by combining the signal-to-noise ratio and occlusion rate metadata according to the timestamp, and the confidence sequence is applied to the multimodal observation frame sequence to generate a standardized observation set with modal quality weighting.
[0033] Temperature scheduling parameters are generated based on confidence sequences and occlusion rate metadata.
[0034] Optionally, the generation of the boundary distribution constraint set includes:
[0035] Based on the standardized observation set, confidence sequence and timestamp set, a method based on kernel density estimation and combined with modal confidence weighting is used to estimate the probability density of the standardized observation set under different timestamps, and generate a multi-time boundary distribution set.
[0036] Based on the multi-time boundary distribution set, a boundary distribution index is established by using a method based on time series segmentation and feature anchor point extraction. The boundary distribution index is then mapped to the timestamp set to generate a time mapping relationship.
[0037] Based on the multi-time boundary distribution set, the boundary distribution index and the time mapping relationship, a weighted consistency fusion method is used to generate the boundary distribution constraint set.
[0038] Optionally, the generation of the potential force distribution and kernel parameter set includes:
[0039] Based on the standardized observation set and equipment attitude parameters, a Gaussian process latent force model is established. Dynamic structural constraints are introduced into the Gaussian process latent force model to construct the latent force prior corresponding to the multimodal observation sequence set.
[0040] In the Gaussian process potential force model, a kernel function is selected, and a set of timestamps and device attitude parameters are introduced during the construction of the kernel function to ensure that the kernel function can reflect the characteristics of potential force changes with time and attitude.
[0041] The kernel hyperparameters of the kernel function are calibrated. The kernel hyperparameters are iteratively optimized using the boundary distribution constraint set, the standardized observation set, and the confidence sequence as references to generate a kernel parameter set.
[0042] By applying the kernel parameter set to the Gaussian process latent force model, inferences are made under the constraints of latent force priors and dynamic structure to generate the latent force distribution.
[0043] Optionally, the generation of the reference diffusion process parameter set includes:
[0044] Based on the potential force distribution and kernel parameter set, a stochastic differential equation modeling method is adopted. In this stochastic differential equation, drift channels and diffusion channels are defined to construct a continuous-time reference diffusion process.
[0045] The parameterization form of the potential force field is determined based on the potential force distribution. The parameterization result is then merged with the basic drift term within the drift channel to generate the drift function of the drift channel.
[0046] The kernel function response is constructed based on the set of kernel parameters and merged with the basic diffusion intensity within the diffusion channel to generate a time-continuous diffusion model.
[0047] Based on the drift function, diffusion model, and continuous-time reference diffusion process, a reference diffusion process parameter set is generated.
[0048] Optionally, the generation of the time-continuous trajectory distribution includes:
[0049] Based on the boundary distribution constraint set and the reference diffusion process parameter set, a reference path measure and boundary constraints are constructed, and the parameterization structure of the bridge potential function pair and the controlled drift field of Schrödinger bridge inference is initialized.
[0050] An iterative proportional fitting method is used to alternately update the bridge potential function pair. The trajectory evolution process is driven by the reference diffusion process parameter set on the timestamp set. The update process is adjusted by combining the boundary distribution constraint set, confidence sequence and temperature scheduling parameters until the bridge potential function pair converges.
[0051] The relationship between the controlled drift field and the controlled transfer is calculated based on the converged bridge potential function and the reference diffusion process parameter set.
[0052] Under the constraints of the controlled drift field and the reference diffusion process parameter set, the trajectory distribution is gradually evolved and corrected by combining the boundary distribution constraint set, confidence sequence and temperature scheduling parameters, to complete the temporal renormalization and generate a time-continuous trajectory distribution.
[0053] Optionally, the generation of the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set includes:
[0054] The conditional probability density is calculated based on the time-continuous trajectory distribution, which is expanded into a conditional probability density at the times corresponding to each time stamp set.
[0055] Based on the conditional probability density, a target trajectory coordinate sequence is generated using the conditional probability inference method.
[0056] Based on the conditional probability density, a weighted second-order moment operation is performed on the timestamp set to calculate the covariance sequence;
[0057] Candidate sets are generated by sampling under the constraint of conditional probability density, and a weighted result is formed by combining the confidence sequence and temperature scheduling parameters during the sampling process to generate a multi-hypothesis candidate set.
[0058] The beneficial effects of this invention are:
[0059] First, by fusing multimodal observation data, this invention effectively solves the problem that single-modal observations are susceptible to noise and occlusion interference in complex environments, and achieves stable observation and feature extraction under different sensor input conditions.
[0060] Secondly, this invention introduces a Gaussian process latent force model and a stochastic differential equation modeling method into trajectory modeling, which overcomes the shortcomings of overly simplified assumptions in traditional dynamic models, and ensures that trajectory prediction can be guaranteed in terms of dynamic consistency and temporal continuity.
[0061] Furthermore, by combining Schrödinger bridge inference with conditional probability inference, this invention not only generates a time-continuous trajectory distribution, but also provides a target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set in the output stage, thereby significantly improving the accuracy and diversity of prediction results and providing reliable support for downstream tracking and control modules. Attached Figure Description
[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0063] Figure 1 This is an overall flowchart of a photoelectric tracking system for multi-mode trajectory prediction proposed in this invention;
[0064] Figure 2 This is a schematic diagram of the structure for generating a reference diffusion process parameter set based on the Gaussian process latent force model and stochastic differential equation modeling in this invention;
[0065] Figure 3 This is a schematic diagram of the structure in this invention for performing Schrödinger bridge inference and generating a time-continuous trajectory distribution under the constraints of the boundary distribution constraint set and the reference diffusion process parameter set. Detailed Implementation
[0066] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0067] refer to Figure 1-3 A photoelectric tracking system for multi-mode trajectory prediction, comprising:
[0068] The multimodal observation and acquisition module is used to acquire raw observation data from visible light sensors, infrared sensors and inertial measurement units, generate a set of timestamped multimodal observation sequences, and extract device attitude parameters, signal-to-noise ratio, occlusion rate metadata and imaging intrinsic parameters, and output a set of timestamps.
[0069] The observation preprocessing module is used to perform preprocessing on the multimodal observation sequence set based on the timestamp set, including denoising, geometric correction, photometric normalization and asynchronous sampling alignment, and calculate the confidence sequence of each mode by combining the signal-to-noise ratio and occlusion rate metadata, and generate a standardized observation set and temperature scheduling parameters.
[0070] The boundary distribution modeling module is used to combine the standardized observation set, confidence sequence and timestamp set, and generate the boundary distribution constraint set using a weighted probability density estimation method. The boundary distribution constraint set includes a multi-time boundary distribution set, boundary distribution index and time mapping relationship.
[0071] The latent force modeling module is used to establish a Gaussian process latent force model based on a standardized set of observations and equipment attitude parameters. In the Gaussian process latent force model, the dynamic structure and latent force priors are determined, the kernel function is selected and the kernel hyperparameters are calibrated, and the latent force distribution and kernel parameter set are generated.
[0072] The diffusion process modeling module is used to construct a continuous-time reference diffusion process based on the potential force distribution and kernel parameter set, using a stochastic differential equation modeling method, to determine the drift term and diffusion intensity, and generate a reference diffusion process parameter set.
[0073] The Schrödinger bridge inference module is used to perform Schrödinger bridge inference under the constraints of the boundary distribution constraint set and the reference diffusion process parameter set. It uses an iterative proportional fitting method to solve the bridge potential function pair and the controlled drift field, and combines the confidence sequence and temperature scheduling parameters to generate a time-continuous trajectory distribution.
[0074] The trajectory inference output module is used to generate target trajectory coordinate sequences, covariance sequences, and multiple hypothesis candidate sets based on time-continuous trajectory distribution using conditional probability inference methods, for use by downstream tracking and control modules.
[0075] In this embodiment, the modules are connected through the following method:
[0076] Raw observation data from visible light sensors, infrared sensors, and inertial measurement units are collected to generate a set of timestamped multimodal observation sequences. At the same time, device attitude parameters, signal-to-noise ratio, occlusion rate metadata, and imaging intrinsic parameters are extracted, and a set of timestamps is output.
[0077] Preprocessing is performed on the multimodal observation sequence set based on the timestamp set, including denoising, geometric correction, photometric normalization and asynchronous sampling alignment. The confidence sequence of each mode is calculated by combining the signal-to-noise ratio and occlusion rate metadata, and a standardized observation set and temperature scheduling parameters are generated.
[0078] By combining the standardized observation set, confidence sequence, and timestamp set, a boundary distribution constraint set is generated using a weighted probability density estimation method, including a multi-time boundary distribution set, boundary distribution index, and time mapping relationship.
[0079] Based on the standardized observation set and equipment attitude parameters, a Gaussian process potential force model is established, the dynamic structure and potential force prior are determined, the kernel function is selected and the kernel hyperparameters are calibrated, and the potential force distribution and kernel parameter set are generated.
[0080] Based on the potential force distribution and kernel parameter set, a continuous-time reference diffusion process is constructed using a stochastic differential equation modeling method. The drift term and diffusion intensity are determined, and a reference diffusion process parameter set is generated.
[0081] Using the boundary distribution constraint set and the reference diffusion process parameter set as constraints, Schrödinger bridge inference is performed, and the bridge potential function pair and the controlled drift field are solved by the iterative proportional fitting method. The time-continuous trajectory distribution is generated by combining the confidence sequence and temperature scheduling parameters.
[0082] Based on the time-continuous trajectory distribution, a conditional probability inference method is used to generate the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set for use by the downstream tracking and control module.
[0083] In this embodiment, the generation of the multimodal observation sequence set and the timestamp set includes:
[0084] The raw observation data collected by the visible light sensor and infrared sensor are timestamped and synchronized, and the raw observation data collected by the inertial measurement unit is also timestamped, generating a set of timestamped multimodal observation sequences. The timestamping means that each frame of imaging data from the visible light sensor and infrared sensor, as well as the measurement data from each group of inertial measurement units, is assigned a unified system clock tag, thereby ensuring that different modal data can be referenced under the same time reference. Synchronization triggering means that the visible light sensor and infrared sensor sample at the same reference time point through the system's time control mechanism, reducing the error caused by asynchronous sampling.
[0085] The device attitude parameters are calculated based on the raw observation data from the inertial measurement unit (IMU), and then mapped to the timestamps of the multimodal observation sequence set. The device attitude parameters are obtained by integrating and solving the acceleration and angular velocity data collected by the IMU, including attitude angles and rotation matrices. These parameters are used to mark the spatial orientation state of the observation data at different time points. Mapping the device attitude parameters to the timestamps of the multimodal observation sequence set means that the observation data at each moment carries its spatial attitude information, realizing the joint calibration of temporal information and spatial state.
[0086] Signal-to-noise ratio (SNR) and occlusion rate metadata are calculated based on the raw observation data from visible light and infrared sensors. These metadata are then mapped to the timestamps of the multimodal observation sequence set. The SNR is obtained by calculating the ratio of the effective signal strength to the noise variance of the observed image and is used to characterize the data quality. The occlusion rate metadata is obtained by calculating the proportion of visible pixels of the target in the observed image and is used to characterize the observation integrity. Both types of metadata are assigned the same timestamp, so that the observation data at each moment carries the corresponding quality index and occlusion index.
[0087] The system calls pre-stored imaging intrinsic parameters and maps them to timestamps in a multimodal observation sequence set, outputting a timestamp set. This set includes timestamps of the original observation data and corresponding device attitude parameters, signal-to-noise ratio, occlusion rate metadata, and imaging intrinsic parameters. The imaging intrinsic parameters include parameters such as focal length, principal point coordinates, and distortion coefficients. These parameters are obtained and pre-stored during the device calibration phase. Calling the pre-stored imaging intrinsic parameters ensures the accuracy of geometric correction and observation space mapping. Mapping the imaging intrinsic parameters to timestamps in a multimodal observation sequence set ensures that each frame of observation data can be interpreted under a unified camera model. The output timestamp set not only contains time point information but also structurally binds device attitude parameters, signal-to-noise ratio, occlusion rate metadata, and imaging intrinsic parameters, achieving the fusion of temporal information with quality indicators, spatial constraints, and imaging geometry.
[0088] In this embodiment, the generation of the standardized observation set and temperature scheduling parameters includes:
[0089] Denoising processing is performed on the multimodal observation sequence set based on the timestamp set. The imaging data of visible light sensors and infrared sensors and the measurement data of the inertial measurement unit are timestamped according to a unified time reference. Denoising processing refers to applying temporal filtering and spatial filtering methods to the original imaging data and inertial measurement data to remove sensor noise and environmental interference, ensuring the stability and reliability of the input data. Timestamp alignment according to a unified time reference maps the data of different modes to the same time reference frame by using a unified system clock, ensuring a strict one-to-one correspondence between the multimodal data in the subsequent fusion process, thereby improving the cross-modal matching accuracy.
[0090] Based on imaging intrinsic parameters, geometric correction and photometric normalization are performed on the imaging data of visible light and infrared sensors to ensure the consistency of multimodal imaging data in spatial geometry and brightness distribution. Geometric correction includes compensation for imaging distortion parameters and coordinate transformation of the imaging plane to align the imaging results of different modalities geometrically. Photometric normalization, through histogram equalization or normalization factor calibration, makes the multimodal imaging data comparable in light intensity and contrast, thus providing a unified standard for subsequent feature extraction and probabilistic modeling.
[0091] Asynchronous sampling alignment is performed under the constraint of timestamp set, mapping the multimodal observation sequence set to a unified time axis and generating a comparable multimodal observation frame sequence. Asynchronous sampling alignment refers to inserting each modal observation point onto a unified time axis through time interpolation and resampling mechanism when the sampling frequencies of different modal sensors are inconsistent, so as to achieve the temporal correspondence of data points and make the multimodal observation frame sequence comparable in a unified time dimension.
[0092] By combining signal-to-noise ratio (SNR) and occlusion rate metadata, a confidence sequence for each modality is calculated based on a timestamp. This confidence sequence is then applied to a multimodal observation frame sequence to generate a standardized observation set with modal quality weighting. The calculation of the confidence sequence involves constructing weighting factors based on SNR and occlusion rate metadata, and using a weighting function to score the reliability of modal observations at different timestamps. When applying the confidence sequence to the multimodal observation frame sequence, low-quality observations are suppressed and high-quality observations are enhanced through weighted averaging or modality selection mechanisms. This achieves modal quality weighting in the standardized observation set, significantly enhancing the robustness of multimodal fusion.
[0093] Temperature scheduling parameters are generated based on the confidence sequence and occlusion rate metadata, and output along with the standardized observation set. The temperature scheduling parameters are obtained by dynamically adjusting the confidence sequence and occlusion rate metadata, and are used to control the smoothness of the distribution and the convergence speed in subsequent probabilistic inference. When the confidence is high, the temperature scheduling parameters tend to shrink to highlight the high-confidence trajectory distribution. When the confidence is low or the occlusion rate is high, the temperature scheduling parameters increase to avoid premature convergence, thereby improving the adaptability and stability of trajectory prediction.
[0094] In this embodiment, the generation of the boundary distribution constraint set includes:
[0095] Based on a standardized observation set, confidence sequence, and timestamp set, a method combining kernel density estimation and modal confidence weighting is used to estimate the probability density of the standardized observation set at different timestamps, generating a multi-time boundary distribution set. Kernel density estimation is a publicly available non-parametric probability distribution estimation method that can fit the distribution pattern of observation data without fixed distribution assumptions. The introduction of the modal confidence weighting mechanism ensures that the multi-time boundary distribution set better reflects the uncertainty characteristics of the real trajectory. The modal confidence weighting mechanism is proposed to address the issue of differences in quality, reliability, and effectiveness among different modes in the multi-modal observation sequence set. The confidence sequences generated for each modality are quantitatively evaluated based on the signal-to-noise ratio and occlusion rate metadata. The confidence values are then used as weights in probability density estimation or subsequent calculations. When the confidence is high, the observation data of that modality has a larger weight in the weighting process; when the confidence is low, the contribution of that modality is weakened accordingly. This dynamic weighting method ensures that the probability modeling process is more consistent with the actual observation scenario and avoids the bias caused by low-quality modalities in the overall distribution estimation. Compared with traditional equal-weight or fixed-weight processing methods, the modality confidence weighting mechanism can adaptively adjust the influence of each modality in a time-varying environment, thereby improving the accuracy and stability of the boundary distribution set.
[0096] Based on a multi-time boundary distribution set, a boundary distribution index is established using a method based on time series segmentation and feature anchor point extraction. The boundary distribution index is then mapped to a timestamp set to generate a time mapping relationship, ensuring that each boundary distribution index corresponds to a unified time reference. Specifically, time series segmentation is used to divide long-term observation data into several intervals to capture trajectory distribution characteristics of different time periods. Feature anchor point extraction is used to identify representative boundary feature points in each segment to establish an efficient index structure. Mapping with the timestamp set ensures that the trajectory distribution characteristics of different time periods correspond to specific times, thereby forming a time continuity constraint.
[0097] Based on the multi-time boundary distribution set, boundary distribution index and time mapping relationship, a weighted consistency fusion method is adopted to generate a boundary distribution constraint set. The weighted consistency fusion method balances the distribution contribution of different time periods by setting weights and introduces consistency constraints in the fusion process to ensure the continuity and stability of the results in a statistical sense. This avoids the instability of the predicted trajectory caused by a sudden change in distribution at a single time. The boundary distribution constraint set reflects short-term dynamic changes while maintaining the smoothness and consistency of the overall trajectory distribution.
[0098] In this embodiment, the generation of the potential force distribution and the kernel parameter set includes:
[0099] Based on a standardized set of observations and device attitude parameters, a Gaussian process latent force model is established. Dynamic structural constraints are introduced into this model to construct a latent force prior corresponding to the multimodal observation sequence set. The dynamic structural constraints embed physical motion laws into the model structure, ensuring that the latent force prior is both dependent on the observation data and conforms to mechanical consistency, thus guaranteeing the accuracy and interpretability of the latent force in both the temporal and physical dimensions. The latent force prior represents the initial assumptions about the distribution of the latent force in the unobserved state. These assumptions are set based on the statistical characteristics of the dynamic structure and the observation data, limiting the possible distribution range of the latent force in the temporal and spatial dimensions. They serve as the core condition for constraining the model's solution space during subsequent inference, ensuring that the latent force distribution not only depends on the observation data but also follows physical consistency and statistical stability in trajectory prediction tasks.
[0100] In the Gaussian process potential force model, a kernel function is selected, and a timestamp set and device attitude parameters are introduced in the process of constructing the kernel function to ensure that the kernel function can reflect the characteristics of potential force changes with time and attitude. The timestamp set is used to capture the temporal evolution law of potential force; the device attitude parameters are used to characterize the influence of attitude changes on potential force. The combination of the two enables the kernel function to take into account both temporal continuity and spatial direction correlation when describing trajectory prediction.
[0101] The kernel function is calibrated using kernel hyperparameters. The kernel hyperparameters are iteratively optimized using the boundary distribution constraint set, the standardized observation set, and the confidence sequence as references to generate a kernel parameter set. The boundary distribution constraint set provides temporal boundary conditions, the standardized observation set ensures input consistency, and the confidence sequence is used to adjust the contribution weights of different modes. The combination of these three factors avoids overfitting and improves robustness during the iterative optimization process, thus obtaining a kernel parameter set that is optimized at both the temporal and modal levels.
[0102] The kernel parameter set is applied to the Gaussian process latent force model to infer the latent force distribution under the constraints of the latent force prior and the dynamic structure. The kernel parameter set determines the correlation structure of the latent force, the latent force prior provides physical consistency constraints, and the dynamic structure ensures that the prediction results conform to the laws of motion. The three work together to achieve probabilistic inference of the latent force, so that the generated latent force distribution can simultaneously characterize the trajectory evolution trend and the uncertainty range.
[0103] In this embodiment, the generation of the reference diffusion process parameter set includes:
[0104] Based on the potential force distribution and kernel parameter set, a stochastic differential equation modeling method is adopted. In this stochastic differential equation, a drift channel and a diffusion channel are defined to construct a continuous-time reference diffusion process. The drift channel consists of the basic drift term and the potential force distribution, and the diffusion channel consists of the basic diffusion intensity and the kernel parameter set. The improved form of the stochastic differential equation is as follows:
[0105] ;
[0106] in, This represents the trajectory state vector at time t, where t represents a continuous time variable. Indicates the basic drift term. This represents the potential force field constructed from the potential force distribution. The scalar adjustment coefficient representing the potential force field. Indicates the basic diffusion intensity. This represents the kernel function response determined by the set of kernel parameters. A parameter vector representing the set of kernel parameters. The scalar adjustment coefficient representing the kernel function response. Indicates the standard Brownian motion increment;
[0107] Among them, the drift channel and diffusion channel are used to distinguish between the deterministic evolution part and the random perturbation part, so that the model can structurally and explicitly integrate the potential force distribution and the kernel parameter set; the continuous-time reference diffusion process refers to the global solution space of the stochastic differential equation in the time domain, providing a reference distribution for subsequent Schrödinger bridge inference; the scalar adjustment coefficient of the potential force field is used to adjust the weight of the potential force distribution in the drift channel; the scalar adjustment coefficient of the kernel function response is used to adjust the influence of the kernel parameter set on the diffusion intensity.
[0108] The parameterization form of the potential force field is determined based on the potential force distribution. The parameterization result is then merged with the basic drift term within the drift channel to generate the drift function of the drift channel. The parameterization form of the potential force field is obtained by mapping the potential force distribution to a function so that it can express the trend of potential force change in the continuous time dimension. It is also superimposed with the basic drift term to ensure that the drift function contains both observation-driven information and dynamic consistency. The drift function is an improved drift term superimposed with the potential force distribution. It not only describes the average trend of trajectory change but also guarantees that the trend satisfies dynamic consistency.
[0109] A kernel function response is constructed based on the kernel parameter set and merged with the basic diffusion intensity within the diffusion channel to generate a time-continuous diffusion model. The kernel function response is generated by the kernel parameter set and is used to characterize the correlation characteristics of trajectory uncertainty in time and state space. After being merged with the basic diffusion intensity, the diffusion model can dynamically reflect the noise propagation law in trajectory prediction. The diffusion model is an improved diffusion term that introduces the kernel parameter set. It can dynamically characterize the noise propagation and uncertainty evolution in trajectory prediction, avoiding the oversimplification of traditional diffusion intensity.
[0110] Based on the drift function, diffusion model, and continuous-time reference diffusion process, a reference diffusion process parameter set is generated. This parameter set is used for the configuration of the reference diffusion process in the subsequent Schrödinger bridge inference stage. By combining the drift function and diffusion model, the reference diffusion process parameter set forms a complete probabilistic description of the trajectory state evolution, ensuring that the generation and evolution of the trajectory distribution can be constrained in a structured parameter configuration manner in Schrödinger bridge inference. This improves the uncertainty modeling capability of the predicted trajectory while ensuring physical consistency.
[0111] In this embodiment, the generation of the time-continuous trajectory distribution includes:
[0112] Based on the boundary distribution constraint set and the reference diffusion process parameter set, a reference path measure and boundary constraints are constructed. The parameterized structure of the bridge potential function pair and the controlled drift field of Schrödinger bridge inference is initialized. The bridge potential function pair represents a two-function structure used in Schrödinger bridge inference to correct the consistency between the reference diffusion process and the boundary distribution constraint set. One function acts on the forward propagation of time, and the other function acts on the backward propagation of time. The parameterized structure of the controlled drift field represents the introduction of a modified form of the potential force distribution on the framework of the basic drift term, so that the drift term can simultaneously respond to the constraints of the reference diffusion process parameter set and the boundary distribution constraint set during the parameterization process.
[0113] An iterative proportional fitting method is used to alternately update the bridge potential function pairs. The trajectory evolution process is driven by the reference diffusion process parameter set on the timestamp set. The update process is adjusted by combining the boundary distribution constraint set, confidence sequence, and temperature scheduling parameters until the bridge potential function pairs converge. The iterative proportional fitting method gradually reduces the difference between the reference diffusion process and the boundary distribution constraint set by alternately updating the bridge potential function pairs. Its applicability lies in its ability to simultaneously satisfy probability flow constraints and boundary matching conditions in each update, thus ensuring that the trajectory distribution generated under the reference diffusion process parameter set is both continuous in time series and consistent with the boundary distribution constraint set at boundary times. Therefore, it can serve as an operational solution to the Schrödinger bridge problem. Convergence of the bridge potential function pairs means that during the iteration process, when the updated function pairs no longer change significantly in continuous time, convergence is determined, resulting in a stable bridge potential function pair.
[0114] The trajectory evolution process represents the gradual evolution of the trajectory distribution over time, driven by a reference diffusion process parameter set constrained by a timestamp set. Specifically, the reference diffusion process parameter set consists of a drift function and a diffusion model. The drift function reflects the deterministic evolution trend of the trajectory under the constraints of dynamic priors and potential force distribution, while the diffusion model reflects the propagation law of random uncertainty of the trajectory under the constraints of the kernel parameter set. Under the constraints of the timestamp set, the trajectory evolution process uses the timestamp set as a benchmark to gradually advance the state update of the trajectory distribution in the continuous time dimension, so that the trajectory distribution at each time point both follows the dynamic constraints of the reference diffusion process parameter set and maintains consistency with the boundary distribution constraint set. In this process, the confidence sequence and temperature scheduling parameters further act as adjustment factors in the evolution process, weighting and correcting the influence of different modal observations, thereby enabling the trajectory evolution process to dynamically adapt to the differences in the quality of multimodal observations and ensuring the temporal stability and physical rationality of the trajectory distribution globally.
[0115] The controlled drift field and controlled transfer relation are calculated based on the converged bridge potential function pair and the reference diffusion process parameter set. The controlled drift field is based on the basic drift term with a gradient correction term superimposed, and the controlled transfer relation is based on the diffusion intensity with boundary consistency correction. Specifically, calculating the controlled drift field means introducing the correction gradient generated by the bridge potential function pair on the basis of the existing drift function, so that the drift term can be biased towards the direction that satisfies the boundary distribution constraint set; calculating the controlled transfer relation means introducing consistency constraints on the diffusion model of the reference diffusion process parameter set, so that the transfer process can conform to the statistical consistency of the boundary distribution constraint set at multiple time steps.
[0116] Under the constraints of a controlled drift field and a reference diffusion process parameter set, the trajectory distribution is progressively evolved and corrected by combining the boundary distribution constraint set, confidence sequence, and temperature scheduling parameters. This results in temporal renormalization and the generation of a time-continuous trajectory distribution. The progressive evolution and correction of the trajectory distribution means that, under the constraints of the controlled drift field and diffusion model, the trajectory distribution advances gradually along the time dimension, and at each moment, the offset is corrected according to the conditions of the boundary distribution constraint set. The completion of temporal renormalization means that a dynamic adjustment mechanism of confidence sequence and temperature scheduling parameters is introduced during the advancement of the trajectory distribution, so that the final output time-continuous trajectory distribution satisfies both physical and dynamic consistency and reflects the differences in quality and reliability of multimodal observations.
[0117] In this embodiment, the generation of the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set includes:
[0118] The conditional probability density is calculated based on the time-continuous trajectory distribution. The time-continuous trajectory distribution is expanded into conditional probability density at the times corresponding to each set of timestamps. This is used to characterize the conditional distribution characteristics of the trajectory in the continuous time dimension. The calculation of conditional probability density means normalizing the time-continuous trajectory distribution under the constraint of the timestamp set. The state distribution of the trajectory at each time moment is characterized by the probability density function, so that the trajectory distribution can be represented in a conditional way in the continuous time dimension, providing a computable probabilistic basis for subsequent inference and sampling.
[0119] Based on conditional probability density, a target trajectory coordinate sequence is generated using the conditional probability inference method. The target trajectory coordinate sequence is composed of the conditional mean of the trajectory distribution that is continuous in time. The conditional probability inference method means inferring the mean of the trajectory at each time point under the constraint of the conditional probability density. The target trajectory coordinate sequence is composed of these conditional means arranged in chronological order, reflecting the most likely trajectory path of the target in the continuous time dimension.
[0120] Based on the conditional probability density, a weighted second-moment operation is performed on the timestamp set to calculate the covariance sequence. The covariance sequence reflects the uncertainty of the time-continuous trajectory distribution in the prediction. Specifically, the weighted second-moment operation on the timestamp set means that the second moment of the conditional probability density is integrated with the confidence sequence as the weight to obtain the variance and covariance matrices at each time point. These matrices are then arranged in the order of the timestamps to generate the covariance sequence, thereby quantitatively reflecting the uncertainty and modal differences in trajectory prediction.
[0121] A candidate set is generated through sampling under the constraint of conditional probability density. During the sampling process, the confidence sequence and temperature scheduling parameters are combined to form a weighted result, generating a multi-hypothesis candidate set. Here, the sampling to generate the candidate set means that a set of possible trajectory paths are obtained through random sampling under the constraint of conditional probability density, serving as multiple candidate solutions for the target trajectory. The weighted result formed by combining the confidence sequence and temperature scheduling parameters during the sampling process represents the weight adjustment of the reliability of different modal observations. At the same time, the temperature scheduling parameters are introduced to control the exploration degree of sampling, thereby ensuring that the multi-hypothesis candidate set can cover multiple trajectory possibilities while improving the consistency with the actual trajectory distribution.
[0122] Example 1:
[0123] To verify the feasibility of this invention in practice, it was applied to an optoelectronic tracking task in a complex environment. The scenario involved the simultaneous continuous observation of a high-speed moving target using a visible light sensor, an infrared sensor, and an inertial measurement unit (IMU). The experimental environment presented challenges such as varying illumination, partial occlusion, and background noise interference. The visible light sensor experienced reduced imaging resolution at long distances, the infrared sensor displayed false heat sources against a strong thermal background, and the IMU suffered from accumulated errors. Traditional methods under these conditions are prone to increased trajectory prediction bias, trajectory interruption, or multi-target confusion, failing to maintain stable tracking performance over long periods.
[0124] During the experiment, the system acquired multimodal observation data and generated a set of multimodal observation sequences. Through preprocessing, signal-to-noise ratio and occlusion rate metadata were incorporated into the calculation of the confidence sequence, ensuring that the influence of low-quality modes in subsequent trajectory modeling was mitigated. Based on this, a Gaussian process latent force model was established, incorporating equipment attitude parameters and timestamp sets to generate a latent force distribution and kernel parameter set. Then, a reference diffusion process parameter set was obtained by combining stochastic differential equation modeling. Under the constraints of the boundary distribution constraint set, Schrödinger bridge inference was performed to generate a time-continuous trajectory distribution. Finally, the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set were output through conditional probability inference. The entire system forms a multi-stage interactive chain, effectively solving the problems of observation uncertainty and insufficient trajectory continuity.
[0125] To evaluate performance, this invention was compared with traditional Kalman filtering, multimodal Bayesian filtering, and long short-term memory neural network prediction methods. Experimental data included trajectory prediction error at different noise levels, trajectory preservation rate in the presence of occlusion, uncertainty control capability in long-term tracking tasks, and real-time processing speed in complex environments. The experiment used 2000 trajectory sequences as the test set, with an average trajectory length of 300 frames, a frame interval of 0.04 seconds, an average target velocity of approximately 20 m / s, an occlusion rate range of 10%–40%, and a signal-to-noise ratio range of 5–25 dB in the test environment.
[0126] Table 1. Performance comparison of the method of the present invention with that of the traditional method.
[0127]
[0128] As shown in Table 1, the method of this invention outperforms the comparative methods in all indicators. The average trajectory prediction error is reduced to 1.35 meters, a decrease of 52.6% compared to 2.85 meters for Kalman filtering and a decrease of 29.7% compared to 1.92 meters for Long Short-Term Memory Neural Networks, indicating more accurate trajectory prediction. This improvement is attributed to the constraints of the Gaussian process latent force model on the dynamic characteristics and the optimization of trajectory distribution under global boundary conditions by Schrödinger bridge inference, which effectively suppresses prediction errors in complex environments.
[0129] In terms of occlusion retention rate, this invention achieves 81.6%, which is higher than Kalman filtering's 61.2%, multimodal Bayesian filtering's 67.5%, and long short-term memory neural networks' 72.1%. This indicates that, in the presence of occlusion, this invention can rely on the reference diffusion process parameter set and boundary distribution constraint set to complete reasonable trajectory inference, effectively ensuring the continuity and stability of the trajectory.
[0130] Regarding covariance convergence speed, this invention achieves stability in just 64 frames, a reduction of 46.7% compared to the 120 frames required for Kalman filtering and a reduction of 27.3% compared to the 88 frames required for Long Short-Term Memory Neural Networks. This demonstrates that the system can control prediction uncertainty more quickly and improve the reliability of trajectory distribution.
[0131] In terms of multi-hypothesis coverage, this invention achieves 76.2%, higher than Kalman filtering's 54.8%, multimodal Bayesian filtering's 61.3%, and long short-term memory neural networks' 68.4%. This means that this invention can output richer candidate solutions, enhances the coverage of complex motion patterns, and provides more feasible trajectory selections for downstream control.
[0132] In terms of real-time processing speed, this invention achieves 160 frames per second, which is slightly lower than the 185 frames per second of Kalman filtering. However, while ensuring high-precision prediction and multiple hypothesis coverage, the processing speed is still far higher than the threshold required for real-time applications, fully meeting the needs of engineering applications.
[0133] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A photoelectric tracking system with multi-mode trajectory prediction, characterized in that, include: The multimodal observation and acquisition module is used to generate a set of multimodal observation sequences and a set of timestamps, and to extract metadata such as device attitude parameters, signal-to-noise ratio, occlusion rate, and imaging intrinsic parameters. The observation preprocessing module is used to generate a standardized observation set and temperature scheduling parameters, and to achieve modal quality weighting by combining the confidence sequence. The boundary distribution modeling module is used to generate a set of boundary distribution constraints, including a multi-time boundary distribution set, boundary distribution index and time mapping relationship; The latent force modeling module is used to build a latent force model of a Gaussian process and generate the latent force distribution and kernel parameter set; The diffusion process modeling module is used to generate a reference diffusion process parameter set using a stochastic differential equation modeling method. The Schrödinger Bridge Inference Module is used to perform Schrödinger Bridge Inference under the constraints of the boundary distribution constraint set and the reference diffusion process parameter set to generate a time-continuous trajectory distribution. The trajectory inference output module is used to generate a target trajectory coordinate sequence, a covariance sequence, and a set of multiple hypothesis candidates based on a time-continuous trajectory distribution. The multimodal observation and acquisition module collects raw observation data, generates a set of multimodal observation sequences, and extracts metadata such as device attitude parameters, signal-to-noise ratio, and occlusion rate to generate a set of timestamps. The observation preprocessing module performs preprocessing on the multimodal observation sequence set based on the timestamp set, calculates the confidence sequence of each modality by combining the signal-to-noise ratio and occlusion rate metadata, and generates a standardized observation set and temperature scheduling parameters. The boundary distribution modeling module combines a standardized observation set, a confidence sequence, and a timestamp set, and uses a weighted probability density estimation method to generate a boundary distribution constraint set. The latent force modeling module establishes a Gaussian process latent force model based on a standardized set of observations and equipment attitude parameters, determines the dynamic structure and latent force priors, selects the kernel function and calibrates the kernel hyperparameters, and generates the latent force distribution and kernel parameter set. The diffusion process modeling module, based on the potential force distribution and kernel parameter set, uses a stochastic differential equation modeling method to construct a continuous-time reference diffusion process, determine the drift function and diffusion model, and generate a reference diffusion process parameter set; The Schrödinger bridge inference module uses the boundary distribution constraint set and the reference diffusion process parameter set as constraints to perform Schrödinger bridge inference. It uses an iterative proportional fitting method to solve the bridge potential function pair and the controlled drift field, and combines the confidence sequence and temperature scheduling parameters to generate a time-continuous trajectory distribution. The trajectory inference output module, based on the time-continuous trajectory distribution, uses a conditional probability inference method to generate a target trajectory coordinate sequence, a covariance sequence, and a set of multiple hypothesis candidates. The generation of the multimodal observation sequence set and timestamp set includes: The raw observation data collected by the visible light sensor and infrared sensor are timestamped and synchronized, and the raw observation data collected by the inertial measurement unit is timestamped to generate a set of multimodal observation sequences with timestamps. The device attitude parameters are calculated based on the raw observation data from the inertial measurement unit, and the device attitude parameters are correlated with the timestamps of the multimodal observation sequence set. The signal-to-noise ratio and occlusion rate metadata are calculated based on the raw observation data from the visible light sensor and the infrared sensor, and the signal-to-noise ratio and occlusion rate metadata are mapped to the timestamps of the multimodal observation sequence set. Call the pre-stored imaging intrinsic parameters, match the timestamps of the imaging intrinsic parameters with the timestamps of the multimodal observation sequence set, and output the timestamp set.
2. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the standardized observation set and temperature scheduling parameters includes: Denoising processing is performed on the multimodal observation sequence set based on the timestamp set, and the imaging data of the visible light sensor and infrared sensor and the measurement data of the inertial measurement unit are timestamp aligned according to a unified time reference; Based on imaging intrinsic parameters, geometric correction and photometric normalization are performed on the imaging data of visible light sensors and infrared sensors; Asynchronous sampling alignment is completed under the constraints of the timestamp set, mapping the multimodal observation sequence set to a unified time axis, and generating a comparable multimodal observation frame sequence; The confidence sequence of each modality is calculated by combining the signal-to-noise ratio and occlusion rate metadata according to the timestamp, and the confidence sequence is applied to the multimodal observation frame sequence to generate a standardized observation set with modal quality weighting. Temperature scheduling parameters are generated based on confidence sequences and occlusion rate metadata.
3. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the boundary distribution constraint set includes: Based on the standardized observation set, confidence sequence and timestamp set, a method based on kernel density estimation and combined with modal confidence weighting is used to estimate the probability density of the standardized observation set under different timestamps, and generate a multi-time boundary distribution set. Based on the multi-time boundary distribution set, a boundary distribution index is established by using a method based on time series segmentation and feature anchor point extraction. The boundary distribution index is then mapped to the timestamp set to generate a time mapping relationship. Based on the multi-time boundary distribution set, the boundary distribution index and the time mapping relationship, a weighted consistency fusion method is used to generate the boundary distribution constraint set.
4. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the potential force distribution and kernel parameter set includes: Based on the standardized observation set and equipment attitude parameters, a Gaussian process latent force model is established. Dynamic structural constraints are introduced into the Gaussian process latent force model to construct the latent force prior corresponding to the multimodal observation sequence set. In the Gaussian process potential force model, a kernel function is selected, and a set of timestamps and device attitude parameters are introduced during the construction of the kernel function to ensure that the kernel function can reflect the characteristics of potential force changes with time and attitude. The kernel hyperparameters of the kernel function are calibrated. The kernel hyperparameters are iteratively optimized using the boundary distribution constraint set, the standardized observation set, and the confidence sequence as references to generate a kernel parameter set. By applying the kernel parameter set to the Gaussian process latent force model, inferences are made under the constraints of latent force priors and dynamic structure to generate the latent force distribution.
5. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the reference diffusion process parameter set includes: Based on the potential force distribution and kernel parameter set, a stochastic differential equation modeling method is adopted. In this stochastic differential equation, drift channels and diffusion channels are defined to construct a continuous-time reference diffusion process. The parameterization form of the potential force field is determined based on the potential force distribution. The parameterization result is then merged with the basic drift term within the drift channel to generate the drift function of the drift channel. The kernel function response is constructed based on the set of kernel parameters and merged with the basic diffusion intensity within the diffusion channel to generate a time-continuous diffusion model. Based on the drift function, diffusion model, and continuous-time reference diffusion process, a reference diffusion process parameter set is generated.
6. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the time-continuous trajectory distribution includes: Based on the boundary distribution constraint set and the reference diffusion process parameter set, a reference path measure and boundary constraints are constructed, and the parameterization structure of the bridge potential function pair and the controlled drift field of Schrödinger bridge inference is initialized. An iterative proportional fitting method is used to alternately update the bridge potential function pair. The trajectory evolution process is driven by the reference diffusion process parameter set on the timestamp set. The update process is adjusted by combining the boundary distribution constraint set, confidence sequence and temperature scheduling parameters until the bridge potential function pair converges. The relationship between the controlled drift field and the controlled transfer is calculated based on the converged bridge potential function and the reference diffusion process parameter set. Under the constraints of the controlled drift field and the reference diffusion process parameter set, the trajectory distribution is gradually evolved and corrected by combining the boundary distribution constraint set, confidence sequence and temperature scheduling parameters, to complete the temporal renormalization and generate a time-continuous trajectory distribution.
7. The photoelectric tracking system for multi-mode trajectory prediction according to claim 1, characterized in that, The generation of the target trajectory coordinate sequence, covariance sequence, and multiple hypothesis candidate set includes: The conditional probability density is calculated based on the time-continuous trajectory distribution, which is expanded into a conditional probability density at the times corresponding to each time stamp set. Based on the conditional probability density, a target trajectory coordinate sequence is generated using the conditional probability inference method. Based on the conditional probability density, a weighted second-order moment operation is performed on the timestamp set to calculate the covariance sequence; Candidate sets are generated by sampling under the constraint of conditional probability density, and a weighted result is formed by combining the confidence sequence and temperature scheduling parameters during the sampling process to generate a multi-hypothesis candidate set.