Multi-source track data fusion prediction method and device

Abstract

The application provides a multi-source track data fusion prediction method and device, relates to the technical field of data fusion, and can systematically solve the problems of sparse missing measurements and time delay disorder of multi-source heterogeneous track data based on merging processing based on a detection principle and a timestamp, realizes normalized representation of multi-source data on a unified time axis, simultaneously, the prediction position of the extrapolation mechanism based on the physical kinematics principle and the displacement prediction model is weighted and fused, the kinematics priori knowledge is encoded into the reasoning process, the track jitter and physical incoherence problems possibly generated by a pure data driven model can be effectively inhibited, the prediction precision and robustness under the condition of multi-source sparse track data are obviously improved, and the accuracy of the final prediction position is improved.

Description

Technical Field

[0001] This invention relates to the field of data fusion technology, and in particular to a method and apparatus for multi-source trajectory data fusion and prediction. Background Technology

[0002] In the field of multi-sensor target tracking and prediction, such as air traffic control, maritime situational awareness, or ground vehicle monitoring, multiple sensors, including radar, network detection, photoelectric, and electronic reconnaissance sensors, are typically deployed to collaboratively observe the same moving target, thereby forming multi-source trajectory data. However, due to factors such as sensor observation cycles, transmission delays, electromagnetic interference, and coverage blind spots, multi-source trajectory data generally suffers from problems such as sparse and missing data, heterogeneous data sources, out-of-order latency, and noise contamination.

[0003] To comprehensively utilize multi-source track data to obtain more stable and accurate target tracks, data fusion technology is widely adopted in the industry. However, existing technologies often directly fuse multi-source track data, while different sensors may have drastically different error characteristics, update frequencies, and data loss patterns. For example, radar detection may be affected by multipath effects, resulting in range ambiguity, while photoelectric detection is prone to data interruption at night or in adverse weather conditions. Directly mixing these raw data with such different characteristics can easily introduce noise, leading to a decrease in the quality or even divergence of the fused target track. Summary of the Invention

[0004] This invention provides a method and apparatus for multi-source trajectory data fusion and prediction, in order to overcome the deficiencies existing in related technologies.

[0005] This invention provides a method for multi-source trajectory data fusion and prediction, comprising: Acquire trajectory observation data of the target object collected by multiple sensors within the current time window; the current time window includes the current moment and multiple historical moments. Based on the detection principle of each sensor and the timestamp of each trajectory observation data, the trajectory observation data are merged into a sparse observation data sequence on a unified time axis; The observation data at each time step within the sparse observation data sequence are feature-encoded to obtain a word embedding vector sequence. Based on the word embedding vector sequence, a displacement prediction model is applied to determine the predicted position and predicted velocity of the target object at the next time step. Based on the predicted velocity and the trajectory observation data corresponding to the current moment, the extrapolated position of the target object at the next moment is determined, and based on the predicted position and the extrapolated position, the final predicted position of the target object is determined.

[0006] According to a multi-source track data fusion prediction method provided by the present invention, the step of merging the track observation data into a sparse observation sequence on a unified time axis based on the detection principle of each sensor and the timestamp of each track observation data includes: Based on the detection principle of each sensor, the track observation data belonging to the same detection principle are merged into a single logical track to obtain multiple logical tracks. For any logical trajectory, merge the track observation data with the same timestamp and the same data in the logical trajectory, retain the track observation data with the same timestamp but different data in the logical trajectory, filter out the track observation data in the logical trajectory whose jump exceeds a preset threshold, pad the track observation data in the logical trajectory with missing data with zeros, and mark the validity of the zero-padding positions based on the mask identifier.

[0007] According to the multi-source track data fusion prediction method provided by the present invention, the step of determining the predicted position and predicted velocity of the target object at the next moment by applying a displacement prediction model based on the word embedding vector sequence includes: Determine the position embedding vector and time embedding vector of the word embedding vector at each time step in the word embedding vector sequence, and construct a word sequence based on the position embedding vector, the time embedding vector and the word embedding vector at each time step in the word embedding vector sequence; Based on the lexical sequence, the displacement prediction model is applied to obtain the predicted position and the predicted velocity.

[0008] According to the multi-source track data fusion prediction method provided by the present invention, the displacement prediction model includes a mask-aware multi-head self-attention coding network and a two-head kinematics-aware regression network; the step of obtaining the predicted position and the predicted velocity based on the word sequence by applying the displacement prediction model includes: The word sequence is input into the displacement prediction model. Each attention head in the multi-head self-attention encoding network calculates attention weights based on the word sequence and the mask identifier, calculates a context vector based on the attention weights, and generates temporal context features based on each context vector. The position sensing head and velocity sensing head in the dual-head kinematics perception and regression network output the predicted position and the predicted velocity based on the temporal context features, respectively.

[0009] According to the multi-source track data fusion prediction method provided by the present invention, the step of performing feature encoding on the observation data at each time point within the sparse observation data sequence to obtain a word embedding vector sequence includes: The observation data at each time step in the sparse observation data sequence are mapped to the Fourier feature space to generate word embedding vectors. The word embedding vectors are arranged in chronological order to obtain the word embedding vector sequence.

[0010] According to the multi-source track data fusion prediction method provided by the present invention, the training steps of the displacement prediction model include: Acquire the trajectory observation data samples of the sample objects collected by each of the aforementioned sensors within a historical time period; Based on the detection principle of each sensor and the timestamp of each trajectory observation data sample, the trajectory observation data samples are merged into sparse observation data sequence samples on a unified time axis; The observation data samples at each time step within the sparse observation data sequence sample are respectively feature-encoded to obtain word embedding vector sequence samples; The displacement prediction model is trained under supervision based on the word embedding vector sequence samples and the true location and true velocity of the sample objects within the historical time period.

[0011] According to the multi-source trajectory data fusion prediction method provided by the present invention, the step of supervised training of the displacement prediction model based on the word embedding vector sequence samples and the true position and true velocity of the sample objects within the historical time period includes: Using a sliding window of the same length as the current time window, target data samples are extracted from the sparse observation data sequence samples. Based on the word embedding vector sequence samples corresponding to the target data samples, an initial prediction model is applied to determine the predicted position sample and predicted velocity sample of the sample object at the next moment in the sliding window. Based on the predicted position samples, the predicted velocity samples, the actual position, and the actual velocity, a training loss is calculated, and the initial prediction model is iteratively trained based on the training loss to obtain the displacement prediction model.

[0012] According to a multi-source trajectory data fusion prediction method provided by the present invention, the step of calculating training loss based on the predicted position sample, the predicted velocity sample, the true position, and the true velocity includes: Based on the predicted location samples and the corresponding actual locations at the given time, the regression loss is calculated; Based on the predicted velocity samples and the actual velocities at the corresponding times, calculate the velocity consistency loss; Calculate the standard deviation of each of the true positions within the sliding window, and based on the standard deviation, weight the difference between the predicted position sample and the true position at the corresponding time to obtain the weighted distance loss; The difference between the predicted location sample and the actual location at the corresponding time is converted into the actual geographical distance, and the actual geographical distance is used as the geographical distance loss. Based on the predicted velocity samples at each time point within the sliding window, the velocity change rate at each time point within the sliding window is solved, and based on the velocity change rate at each time point within the sliding window, the acceleration penalty loss is calculated. The training loss is calculated based on the regression loss, the velocity consistency loss, the weighted distance loss, the geographic distance loss, and the acceleration penalty loss.

[0013] According to a multi-source track data fusion prediction method provided by the present invention, determining the extrapolated position of the target object at the next moment based on the predicted velocity and the track observation data corresponding to the current moment includes: If the actual position of the target object at the current moment is empty, then the predicted position corresponding to the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position; If the actual position of the target object at the current moment is not empty, then the actual position of the target object at the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position.

[0014] The present invention also provides a multi-source track data fusion prediction device, comprising: The data acquisition module is used to acquire track observation data of the target object collected by multiple sensors within the current time window; the current time window includes the current moment and multiple historical moments. The time unification module is used to merge the track observation data into a sparse observation data sequence on a unified time axis based on the detection principle of each sensor and the timestamp of each track observation data. The displacement prediction module is used to perform feature encoding on the observation data at each time step within the sparse observation data sequence to obtain a word embedding vector sequence, and based on the word embedding vector sequence, apply the displacement prediction model to determine the predicted position and predicted velocity of the target object at the next time step. The position prediction module is used to determine the extrapolated position of the target object at the next moment based on the predicted velocity and the track observation data corresponding to the current moment, and to determine the final predicted position of the target object based on the predicted position and the extrapolated position.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-source track data fusion prediction method as described above.

[0016] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-source track data fusion prediction method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the multi-source track data fusion prediction method as described above.

[0018] The multi-source track data fusion prediction method and apparatus provided by this invention can systematically solve the problems of sparse missing data and time delay disorder in multi-source heterogeneous track data by merging processing based on detection principles and timestamps, and realize the standardized representation of multi-source data on a unified time axis. At the same time, by introducing an extrapolation mechanism based on physical kinematics principles and weighted fusion with the predicted position of the displacement prediction model, kinematic prior knowledge is incorporated into the reasoning process, which can effectively suppress the trajectory jitter and physical inconsistency problems that may be generated by pure data-driven models, significantly improve the prediction accuracy and robustness under multi-source sparse track data conditions, and improve the accuracy of the final predicted position. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the multi-source trajectory data fusion prediction method provided by the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of the multi-source trajectory data fusion prediction device provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0024] Figure 1 This is a flowchart illustrating a multi-source trajectory data fusion prediction method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: S1, acquire trajectory observation data of the target object collected by multiple sensors in the current time window; the current time window includes the current moment and multiple historical moments; S2, based on the detection principle of each sensor and the timestamp of each trajectory observation data, the trajectory observation data are merged into a sparse observation data sequence on a unified time axis; S3, feature encoding is performed on the observation data at each time step within the sparse observation data sequence to obtain a word embedding vector sequence, and based on the word embedding vector sequence, a displacement prediction model is applied to determine the predicted position and predicted velocity of the target object at the next time step. S4. Based on the predicted velocity and the trajectory observation data corresponding to the current moment, determine the extrapolated position of the target object at the next moment, and based on the predicted position and the extrapolated position, determine the final predicted position of the target object.

[0025] Specifically, the multi-source trajectory data fusion prediction method provided in this embodiment of the invention is applied to scenarios where trajectory data of the same object observed by multiple sensors need to be fused and processed to predict the future motion state of the object. The execution subject of this method is a multi-source trajectory data fusion prediction device, which can be configured in a data fusion processing platform or server. The data fusion processing platform or server can be a locally deployed high-performance computing device or a cloud server, and no specific limitation is made here.

[0026] First, step S1 is executed to acquire trajectory observation data of the target object collected by multiple sensors within the current time window. These sensors may include radar, network detection equipment, photoelectric sensors, and electronic reconnaissance equipment. Different sensors have different detection principles and observation periods, resulting in differences in data format, accuracy, and temporal resolution.

[0027] The target object refers to a moving target that is simultaneously tracked and observed by multiple sensors, such as an aircraft, ship, or ground vehicle. Track observation data refers to the record of the target object's position information acquired by each sensor within its observation period, typically collected at different times. Here, position information can include longitude and latitude coordinates. In actual data acquisition, each sensor outputs track observation data in structured data format.

[0028] The current time window refers to a continuous time range selected for performing trajectory prediction. It includes the current moment and multiple historical moments. The current moment refers to the most recent acquisition of trajectory observation data, while the multiple historical moments refer to several consecutive or discontinuous observation moments preceding the current moment. The length of the current time window can be set according to the actual application scenario and prediction accuracy requirements. For example, the current time window can be composed of the ten most recent observation moments, including the current moment. When acquiring trajectory observation data, all observation records within the current time window are retrieved from the data source according to the device number and target identifier of each sensor. Due to the different observation periods of different sensors, the trajectory observation data collected by each sensor within the current time window may be unevenly distributed over time. At some moments, some sensors may not have collected data, resulting in missing data.

[0029] Then, step S12 is executed, which uses the detection principles of each sensor and the timestamps of each track observation data to merge the track observation data into a sparse observation data sequence on a unified time axis. The detection principle refers to the detection system adopted by each sensor. For example, radar uses the principle of electromagnetic wave reflection to detect targets, photoelectric sensors use the principle of optical imaging to detect targets, and electronic reconnaissance equipment uses the principle of electromagnetic signal interception to detect targets, etc.

[0030] Sensors using different detection principles differ in observation accuracy, observation period, and data format. Therefore, it is necessary to classify and group the trajectory observation data of each sensor according to its detection principle. For example, trajectory observation data collected by sensors using the same detection principle can be grouped into the same logical trajectory; that is, trajectory observation data collected by multiple sensors under the same detection system belong to the same logical trajectory. For devices not included in the inference process, they can be classified into tracks used only for statistical analysis and not involved in subsequent prediction calculations.

[0031] A timestamp is a marker indicating the time of acquisition for each track observation data point, used to identify its position on the timeline. Due to the different observation periods of multiple sensors and the influence of transmission delays and electromagnetic interference, the track observation data reported by each sensor may be out of order, meaning later-collected data may arrive at the processing end first. Therefore, it is necessary to sort and merge the track observation data based on timestamps. For example, suppose there are three logical tracks, corresponding to radar detection, electro-optical detection, and electronic reconnaissance, respectively, with ten moments within the current time window. At a certain moment, both radar and electro-optical detection collect track observation data of the target object, while electronic reconnaissance does not collect any data; in this case, the electronic reconnaissance track observation data at that moment is filled with zero values.

[0032] By grouping track observation data into a sparse observation data sequence on a unified time axis based on the detection principles and timestamps of each sensor, the alignment and unified representation of multi-source heterogeneous data in the time dimension can be achieved. This effectively solves the problems of sparse missing measurements and time delay disorder caused by different observation periods and inconsistent transmission delays of multi-source data, and provides standardized input data for subsequent feature encoding and model prediction.

[0033] Next, step S13 is executed to perform feature encoding on the observation data at each time step within the sparse observation data sequence, resulting in a word embedding vector sequence. Here, feature encoding refers to the process of converting the observation data at each time step in the sparse observation data sequence into a high-dimensional vector representation. Since the observation data at each time step in the sparse observation data sequence are all continuous values, and the displacement prediction model used later was originally designed for discrete words, directly encoding continuous coordinates in a simple way results in a spectral bias problem, making it difficult to distinguish high-frequency detail changes in the coordinates. Therefore, this embodiment of the invention uses a specialized encoding method to perform feature encoding on the observation data at each time step.

[0034] When encoding features of observation data at each time step within a sparse observation data sequence, various encoding methods can be employed, such as coordinate Fourier coding, linear coding, multilayer perceptron coding, block coding, or residual coding. Each observation data step yields a word embedding vector, and the word embedding vectors corresponding to the observation data at each time step within the sparse observation data sequence, ordered according to their respective time steps, can form a word embedding vector sequence.

[0035] For example, suppose the current time window contains ten moments, each corresponding to the latitude and longitude coordinates of three logical flight paths, meaning the input dimension for each moment is six-dimensional. After feature encoding, the observation data at each moment is mapped to a 768-dimensional word embedding vector, and the word embedding vectors corresponding to the observation data at the ten moments constitute a word embedding vector sequence.

[0036] This step involves encoding the features of the observation data at each time point within the sparse observation data sequence and applying a displacement prediction model for prediction. This effectively converts continuous coordinate data into a high-dimensional feature representation that can be processed by the displacement prediction model.

[0037] Furthermore, by utilizing word embedding vector sequences and combining them with a displacement prediction model, the predicted position and velocity of the target object at the next moment can be determined. The displacement prediction model refers to a deep learning model used to predict the future position and velocity of the target object based on its historical motion data. The displacement prediction model can employ an encoding network structure based on a multi-head self-attention mechanism.

[0038] Here, the word embedding vector sequence can be directly input into the displacement prediction model. The displacement prediction model models the correlation between the observation data at each time step in the word embedding vector sequence, capturing the motion patterns and trends of the target object, and thus outputting the predicted position and velocity of the target object at the next time step. The predicted position refers to the normalized position of the target object output by the displacement prediction model at the next time step, which may include normalized longitude and latitude coordinates. The predicted velocity refers to the normalized velocity of the target object output by the displacement prediction model at the next time step, which represents the normalized representation of the positional change of the target object between adjacent time steps.

[0039] Finally, step S14 is executed to determine the extrapolated position of the target object at the next moment using the predicted velocity and the track observation data corresponding to the current moment. This extrapolated position refers to the result obtained by physically calculating the position of the target object at the next moment based on kinematic principles, using the known predicted velocity and the track observation data corresponding to the current moment, and assuming uniform motion.

[0040] For example, the predicted velocity output by the displacement prediction model can be first converted into displacement per second; then, using the position in the trajectory observation data corresponding to the current moment as the anchor point, and following the assumption of uniform motion, the product of the anchor point position and the predicted velocity multiplied by the time step is added to obtain the extrapolated position of the target object at the next moment. Here, the time step is the time length between two adjacent moments.

[0041] After determining the extrapolated position, the predicted position output by the displacement prediction model and the extrapolated position are weighted and fused to obtain the final predicted position of the target object. The weight coefficients for both the predicted and extrapolated positions range from 0 to 1. For example, the weight coefficient for the predicted position can be 0.7, and the weight coefficient for the extrapolated position can be 0.3, meaning the final predicted position is primarily determined by the predicted position and secondarily by the extrapolated position. This fusion method balances the learning ability of the displacement prediction model with prior kinematic knowledge based on physical laws. It can correct for deviations in the displacement prediction model through physical extrapolation, while prioritizing the predicted position output by the displacement prediction model when the prediction is accurate, thereby improving the accuracy and stability of the final predicted position.

[0042] Furthermore, in practical applications, the prediction state of each track in the previous frame can be maintained independently according to the target identifier, so as to achieve parallel smooth processing of multiple target objects. That is, for multiple target objects tracked at the same time, the above extrapolation and fusion operations are performed independently without interference.

[0043] For example, assuming the displacement prediction model outputs a predicted velocity corresponding to a displacement per second of 0.01 degrees in longitude and 0.005 degrees in latitude, and the current anchor point is 120 degrees east longitude and 30 degrees north latitude, then the extrapolated position is 120.01 degrees east longitude and 30.005 degrees north latitude. If the displacement prediction model outputs a predicted position of 120.008 degrees east longitude and 30.004 degrees north latitude, with a weighting coefficient of 0.7 for the predicted position and 0.3 for the extrapolated position, then the final predicted position is the result of a weighted sum of the predicted position and the extrapolated position using weighting coefficients of 0.7 and 0.3, respectively.

[0044] The extrapolated position is determined based on the predicted velocity and the current trajectory observation data. The predicted position and the extrapolated position are then weighted and fused to obtain the final predicted position, effectively combining data-driven prediction with kinematic extrapolation based on physical laws. This fusion method uses physical extrapolation to constrain and correct the predicted position output by the displacement prediction model, effectively suppressing trajectory jitter and improving the smoothness and physical plausibility of the trajectory prediction. Furthermore, by flexibly setting the weighting coefficients, an optimal balance can be achieved between the predicted and extrapolated positions according to the actual application scenario.

[0045] The multi-source track data fusion prediction method provided in this invention can systematically solve the problems of sparse missing data and time delay disorder in multi-source heterogeneous track data by merging processing based on detection principles and timestamps, and realize the standardized representation of multi-source data on a unified time axis. At the same time, by introducing an extrapolation mechanism based on physical kinematics principles and weighted fusion with the predicted position of the displacement prediction model, kinematic prior knowledge is incorporated into the reasoning process, which can effectively suppress the trajectory jitter and physical inconsistency problems that may be generated by the pure data-driven model, significantly improve the prediction accuracy and robustness under multi-source sparse track data conditions, and improve the accuracy of the final predicted position.

[0046] Based on the above embodiments, the step of grouping the track observation data into a sparse observation sequence on a unified time axis based on the detection principle of each sensor and the timestamp of each track observation data includes: Based on the detection principle of each sensor, the track observation data belonging to the same detection principle are merged into a single logical track to obtain multiple logical tracks. For any logical trajectory, merge the track observation data with the same timestamp and the same data in the logical trajectory, retain the track observation data with the same timestamp but different data in the logical trajectory, filter out the track observation data in the logical trajectory whose jump exceeds a preset threshold, pad the track observation data in the logical trajectory with missing data with zeros, and mark the validity of the zero-padding positions based on the mask identifier.

[0047] Specifically, in determining the sparse observation sequence, the detection principles of each sensor can be used to group track observation data belonging to the same detection principle into a single logical track, resulting in multiple logical tracks. Each logical track corresponds to a data channel based on a specific detection principle.

[0048] For example, track observation data from all radar sensors can be mapped to radar logical tracks, track observation data from all electro-optical sensors can be mapped to electro-optical logical tracks, and track observation data from all electronic reconnaissance sensors can be mapped to electronic reconnaissance logical tracks. Track observation data from sensors not included in the inference process can be categorized as auxiliary tracks used only for statistical analysis and will not participate in subsequent prediction calculations.

[0049] Since sensors based on different detection principles have fundamental differences in observation accuracy, data format, and noise characteristics, merging the track observation data of sensors based on the same detection principle into a single logical track can avoid semantic confusion caused by directly mixing heterogeneous data, while providing semantically consistent data input for subsequent feature encoding and model prediction.

[0050] Then, for any logical track, all track observation data within that logical track are sorted according to their timestamps. For track observation data with the same timestamp and identical values, a deduplication and merging operation is performed, retaining only one track observation data. Here, track observation data with the same timestamp and identical values ​​refer to observation data reported by multiple sensors of the same type within the same logical track, which have the same value at the same time. This type of observation data is redundant information, and merging it can reduce computational overhead without losing effective information.

[0051] For track observation data with the same timestamp but different timestamps, they are identified as conflicting data and are rejected for merging. Multiple different observation data points for that logical track at that time are retained separately. Here, track observation data with the same timestamp but different timestamps refer to observation records reported by multiple sensors of the same type within the same logical track, with different values ​​at the same time. This type of observation data may reflect multipath effects or local measurement biases. Retaining such observation data can help displacement prediction models obtain richer observation patterns.

[0052] For track observation data where the jump between adjacent time points exceeds a preset threshold, the observation data at the next adjacent time point is determined to be noise and filtered out. Here, the preset threshold refers to the reasonable physical motion range of the position change. Track observation data with jumps exceeding the preset threshold are usually caused by electromagnetic interference, equipment failure, or target identification mismatch, and are considered noise data, which should be filtered out to prevent them from misleading the prediction process of the displacement prediction model. For example, when the single-step jump in longitude or latitude coordinates is greater than or equal to 10°, the observation data at the next time point is considered to not conform to the physical motion law of the target object and is directly discarded.

[0053] For track observations with missing data at a given moment (i.e., no valid observation data), zero-padding is performed at that location, and a mask identifier is set to invalidate this identifier. This indicates that the zero value at that location is for zero-padding to fill in missing data, not for actual observation data. This mask identifier is a validity identifier used in conjunction with the zero-padding operation to distinguish between cases where actual observation data is zero and cases where zero-padding is performed due to missing data. This ensures that the displacement prediction model can correctly identify which locations are valid observation data and which are missing data during processing, avoiding misclassification of zero-padding as actual observation data.

[0054] After the above processing, the observation data of each logical track at each time point are organized into a sparse observation data sequence with a uniform structure. Each time point in this sparse observation data sequence corresponds to the observation data in each logical track. Missing positions are filled with zero values ​​and their non-real observation data attributes are identified by a mask.

[0055] Through the above merging process, semantic unification at the logical level and structural alignment at the temporal level can be achieved for multi-source heterogeneous track data. Data with the same detection principle are merged into logical tracks, avoiding semantic pollution caused by direct mixing of heterogeneous data; redundant data is merged, and conflicting data is retained or optimized, controlling data size while ensuring information integrity; noisy data with jump anomalies is filtered out to improve data quality; missing data is padded with zeros and its validity is indicated by a mask, ensuring that the sparse observation data sequence maintains a regular tensor structure for easy model processing while retaining explicit labels of missing information for model perception and utilization. This provides a high-quality, structured input foundation for subsequent feature encoding and displacement prediction.

[0056] In this embodiment of the invention, by combining semantic hierarchical merging driven by the detection principle with a multi-dimensional data cleaning mechanism, the problems of heterogeneous mixing, redundancy and conflict, noise interference and sparse missing measurements in multi-source track data can be systematically solved. While ensuring the semantic consistency and structural regularity of the data, the identifiable markers of missing information are explicitly preserved, laying a reliable data foundation for subsequent feature encoding and displacement prediction based on deep learning, and significantly improving the adaptability and robustness of the fusion prediction method in complex multi-source sparse scenarios.

[0057] Based on the above embodiments, the step of applying a displacement prediction model based on the word embedding vector sequence to determine the predicted position and predicted velocity of the target object at the next moment includes: Determine the position embedding vector and time embedding vector of the word embedding vector at each time step in the word embedding vector sequence, and construct a word sequence based on the position embedding vector, the time embedding vector and the word embedding vector at each time step in the word embedding vector sequence; Based on the lexical sequence, the displacement prediction model is applied to obtain the predicted position and the predicted velocity.

[0058] Specifically, in applying the displacement prediction model, we can first determine the positional embedding vector and temporal embedding vector of the word embedding vector at each time step in the word embedding vector sequence. Here, the positional embedding vector refers to the vector representation used to characterize the relative order of the word embedding vectors in the word embedding vector sequence. Since the self-attention mechanism used by the displacement prediction model itself does not have the ability to perceive the order of the input word embedding vector sequence, if the word embedding vectors at each time step are directly input into the displacement prediction model, the displacement prediction model cannot distinguish the sequential relationship between different word embedding vectors. Therefore, it is necessary to use the positional embedding vector to obtain the positional information of each word embedding vector in the word embedding vector sequence. The positional embedding vector can be in the form of a learnable parameter matrix or in the form of a fixed sine and cosine encoding, with its dimension consistent with the dimension of the word embedding vector, so as to facilitate subsequent element-wise addition and fusion.

[0059] Temporal embedding vectors are vector representations used to characterize the absolute time information corresponding to each moment. Unlike positional embedding vectors, which only provide the relative order of word embedding vectors within a word embedding vector sequence, temporal embedding vectors carry the actual timestamp information of each moment. This allows displacement prediction models to perceive the distribution density and time intervals of observed data on the true time axis, which is particularly important for handling non-uniform sampling problems caused by different sensor observation periods and transmission delays. Temporal embedding vectors can also take the form of a learnable parameter matrix or a fixed sine / cosine encoding form, with dimensions consistent with word embedding vectors and positional embedding vectors.

[0060] For example, the corresponding position embedding vector can be retrieved from a predefined position encoding table or calculated through a learnable parameter matrix according to the index position in the word embedding vector sequence at a certain time, and the corresponding time embedding vector can be obtained by mapping through a time encoding function according to the real timestamp corresponding to that time.

[0061] Subsequently, a word sequence is constructed using the positional embedding vector, temporal embedding vector, and word embedding vector corresponding to each time step in the word embedding vector sequence. The word sequence can include the word corresponding to each time step. Each word can be obtained by adding the positional embedding vector, temporal embedding vector, and word embedding vector at the corresponding time step element by element. Each word can simultaneously carry the sequence order information, absolute time information, and observation feature information at the corresponding time step, and can be used as a direct input object for the displacement prediction model.

[0062] Furthermore, the word sequence can be input into the displacement prediction model. The displacement prediction model uses a multi-head self-attention mechanism to model the long-range dependencies between different time points in the word sequence, captures the motion patterns and trends of the target object at historical time points, and outputs the predicted position and velocity of the target object at the next time point.

[0063] By supplementing each time step in the word embedding vector sequence with positional and temporal embedding vectors and constructing a word sequence, the displacement prediction model can simultaneously perceive the relative order within the sequence and the absolute time distribution on the true time axis, effectively overcoming the inherent defect of the pure self-attention mechanism being insensitive to temporal information. Especially in scenarios where multiple sensor observation periods differ and data transmission is delayed and disordered, the absolute time information provided by the temporal embedding vector enables the model to correctly understand the temporal structure of non-uniformly sampled data, avoiding misjudgments of motion trends due to missing time information, thereby improving the prediction accuracy of the displacement prediction model under complex temporal conditions.

[0064] In this embodiment of the invention, a dual embedding mechanism is used to simultaneously inject relative order information within the sequence and absolute time information of the real time axis, which makes up for the lack of perception of temporal structure by the self-attention mechanism. This enables the displacement prediction model to accurately understand the temporal structure of the observed data under the conditions of non-uniform sampling of multi-source heterogeneous data and disordered time delay, and realizes the joint output and physical self-consistency of predicted position and predicted velocity. This significantly improves the modeling ability and prediction reliability of the displacement prediction model on sparse non-uniform temporal data.

[0065] Based on the above embodiments, the displacement prediction model includes a mask-aware multi-head self-attention encoding network and a two-head kinematics-aware regression network; the step of applying the displacement prediction model based on the word sequence to obtain the predicted position and the predicted velocity includes: The word sequence is input into the displacement prediction model. Each attention head in the multi-head self-attention encoding network calculates attention weights based on the word sequence and the mask identifier, calculates a context vector based on the attention weights, and generates temporal context features based on each context vector. The position sensing head and velocity sensing head in the dual-head kinematics perception and regression network output the predicted position and the predicted velocity based on the temporal context features, respectively.

[0066] Specifically, the displacement prediction model can include a mask-aware multi-head self-attention encoding network and a two-head kinematics-aware regression network. The mask-aware multi-head self-attention encoding network is the core encoding module in the displacement prediction model responsible for extracting temporal contextual features from the lexical sequence. This multi-head self-attention encoding network consists of stacked multi-layer multi-head self-attention structures, each containing multiple attention heads working in parallel. Each attention head models the relationships between lexical units at different time points in the lexical sequence from different subspaces. Mask awareness means that during the calculation of attention weights, the multi-head self-attention encoding network can receive and identify mask identifiers corresponding to zero-padding positions in the sparse observation data sequence, thereby automatically masking the influence of missing time steps or zero-padding dimensions during attention calculation, preventing invalid zero-padding data from interfering with the attention allocation of valid observation data. Multi-head self-attention involves mapping the lexical sequence to query vectors, key vectors, and value vectors through multiple sets of different linear transformations. Each attention head independently calculates attention weights and performs weighted aggregation. Finally, the context vectors output by each attention head are concatenated and linearly transformed to output temporal contextual features.

[0067] Attention weights are quantitative measures of the strength of association between different moments in a word sequence when each attention head calculates the context vector. A higher value indicates a stronger motion association between corresponding moments. The context vector is an aggregated feature vector obtained by weighting and summing the value vectors of each moment in the word sequence under the weighting effect of attention weights. This context vector incorporates motion context information from the entire current time window.

[0068] Temporal context features refer to the final encoding result output by a multi-head self-attention encoding network after multi-layer stacking. These temporal context features integrate the global dependencies and local motion patterns between different time points in the word sequence and serve as the direct input for the prediction output of a two-head kinematics-aware regression network.

[0069] A dual-head kinematics-aware regression network (DKN) is a module in a displacement prediction model responsible for mapping the temporal context features obtained from a multi-head self-attention encoding network to predicted velocity and predicted position. This DKN contains two independent regression branches: a position-aware head and a velocity-aware head, which output predicted position and predicted velocity, respectively. Kinematic awareness refers to the fact that this DKN is guided by kinematic consistency constraints during training, ensuring that the predicted velocity output by the velocity-aware head and the predicted position output by the position-aware head satisfy a physical kinematic relationship; that is, the predicted velocity should approximate the normalized representation of the position displacement at adjacent time points.

[0070] In the process of obtaining predicted position and velocity using the displacement prediction model, a word sequence can be input into the displacement prediction model. In each layer of the multi-head self-attention encoding network, after receiving the word sequence, each attention head maps the words at each time step to the corresponding query vector, key vector, and value vector through its own independent linear transformation. Subsequently, each attention head calculates the original attention score based on the dot product of the query vector and the key vector. During the calculation process, a mask identifier is introduced to correct the attention score. For moments when the mask identifier is invalid, its corresponding attention score is set to negative infinity, so that the attention weight at that position approaches zero after normalization. This automatically masks the influence of zero-padding missing data during the attention aggregation process.

[0071] After masking correction, the attention scores are normalized to obtain the final attention weights. Each attention head then performs a weighted sum of its corresponding value vectors based on its attention weight to obtain the context vector output by that attention head. The context vectors output by all attention heads in the multi-head self-attention encoding network are concatenated and fused through linear transformation and residual connections. This fusion is then passed through a feedforward network and layer normalization to generate the output of that layer of the multi-head self-attention encoding network. After layer-by-layer stacking of multiple multi-head self-attention structures, the final output is a temporal context feature. This temporal context feature fully encodes the global temporal dependencies between different time points in the word sequence, while effectively isolating interference from missing data through a mask-aware mechanism.

[0072] After obtaining the temporal context features, these features are input into a dual-head kinematics-aware regression network. The position sensing head in the network receives the temporal context features, processes them through fully connected layers and nonlinear activation functions, and outputs the predicted position of the target object at the next time step. This predicted position is represented in latitude and longitude coordinates. The velocity sensing head also receives the temporal context features, processes them through independent fully connected layers and nonlinear activation functions, and outputs the predicted velocity of the target object at the next time step. This predicted velocity is represented in normalized displacement form, i.e., a normalized expression of the positional change between adjacent time steps.

[0073] Although the position-aware head and velocity-aware head share the same temporal context features as input, they each have independent network parameters and are optimized for position regression and velocity regression tasks respectively, avoiding gradient conflict problems when a single output head undertakes two different physical quantity regression tasks at the same time.

[0074] For example, suppose a word sequence contains ten time points, each with a word dimension of 768. A logical track at the fifth time point in the mask is marked as invalid due to a missing value. In a multi-head self-attention encoding network, when each attention head calculates its attention weight, the attention score for the missing dimension at the fifth time point is set to negative infinity. After normalization, its attention weight approaches zero, so this missing value location will not interfere with the calculation of the context vector for other valid time points. The temporal context features output by the multi-head self-attention encoding network are fed into the position-aware head and the velocity-aware head, respectively. The position-aware head outputs the normalized longitude and latitude coordinates for the next time point, and the velocity-aware head outputs the normalized velocity for the next time point.

[0075] By employing a mask-aware multi-head self-attention coding network to encode the word sequence, interference from missing data (including padding) can be fundamentally isolated at the attention computation level. This allows the displacement prediction model to learn motion patterns solely from valid observation data, avoiding feature contamination issues caused by missing data participating in attention computation in traditional schemes. Furthermore, by using a dual-head kinematics-aware regression network to output predicted position and predicted velocity separately, the independent regression accuracy of the two physical quantities is ensured, while kinematic consistency constraints guarantee the physical consistency of the prediction results. This effectively suppresses jitter and drift in the predicted trajectory, improving the smoothness and reliability of the trajectory prediction.

[0076] In this embodiment of the invention, the displacement prediction model is specifically defined as a combination architecture of a mask-aware multi-head self-attention encoding network and a two-head kinematics-aware regression network. During the encoding stage, a mask identifier guides the calculation of attention weights to shield against interference from missing data. During the decoding stage, the position-aware head and velocity-aware head output the predicted position and predicted velocity respectively, and kinematic consistency constraints are applied. This scheme deeply embeds the mask-aware mechanism into the attention calculation process, enabling the displacement prediction model to adaptively focus on effective observation information when processing sparse and non-uniform observation data. Simultaneously, through the collaborative design of two-head independent regression and kinematic consistency constraints, the joint output and physical consistency of the predicted position and predicted velocity are achieved, significantly improving the feature extraction capability and physical rationality of the predicted output under multi-source sparse track data conditions.

[0077] Based on the above embodiments, the step of performing feature encoding on the observation data at each time point within the sparse observation data sequence to obtain a word embedding vector sequence includes: The observation data at each time step in the sparse observation data sequence are mapped to the Fourier feature space to generate word embedding vectors. The word embedding vectors are arranged in chronological order to obtain the word embedding vector sequence.

[0078] Specifically, since the latitude and longitude coordinates in the track observation data are essentially continuously changing physical quantities, and the displacement prediction model has limited ability to fit discrete inputs, directly inputting the observation data into the displacement prediction model can easily lead to the loss of high-frequency motion information and oversmoothing of low-frequency trends.

[0079] Therefore, a coordinate Fourier coding layer is introduced in the process of feature encoding the observation data at each time step within the sparse observation data sequence to obtain the word embedding vector sequence. The observation data corresponding to all logical tracks at each time step within the sparse observation data sequence are input into the coordinate Fourier coding layer. The coordinate Fourier coding layer maps the observation data at each time step to the Fourier feature space to generate each word embedding vector.

[0080] The Fourier feature space refers to the high-dimensional periodic feature representation space transformed from the original low-dimensional observation data through Fourier feature mapping. Fourier feature mapping expands each observation data point into a high-dimensional vector representation containing multiple frequency components by performing an inner product operation with a set of pre-defined sine and cosine basis functions. This allows the displacement prediction model to simultaneously capture the low-frequency global motion trend and high-frequency local motion details of the target object in the feature space.

[0081] In generating word embedding vectors, the coordinate Fourier coding layer extracts the observation data corresponding to all logical tracks at each time point in the sparse observation data sequence, including the longitude and latitude coordinates and mask identifiers of each logical track. For zero-padding positions, since their corresponding mask identifiers are invalid, they can be excluded from the calculation during the Fourier feature mapping process or their mapping results can be set to zero vectors to avoid introducing spurious frequency components in the frequency domain. Then, a set of sine and cosine basis functions covering different frequency ranges is selected, and the effective observation data at that time is used to perform inner product operations with the sine and cosine basis functions respectively to generate the corresponding frequency component vectors. The frequency component vectors of all logical tracks at that time are concatenated or aggregated to obtain the word embedding vector corresponding to that time point. For example, if at a certain moment all three logical tracks have valid observations, and the latitude and longitude coordinates of each logical track generate a 256-dimensional frequency component vector after Fourier feature mapping, then the word embedding vector at that moment has a dimension of 1536. If one of the logical tracks is missing a measurement at that moment, its corresponding frequency component vector is a zero vector. The final word embedding vector still maintains a uniform dimension of 1536, but the subspace corresponding to the missing logical track has zero values, which are distinguished in subsequent model processing by a mask. After repeating the above operation for all moments, the resulting word embedding vectors are arranged in ascending order of timestamps, thus forming a word embedding vector sequence.

[0082] By mapping raw observation data to the Fourier feature space to generate word embedding vectors, continuously changing physical coordinates can be transformed into high-dimensional feature representations containing multi-scale frequency information. This enables displacement prediction models to simultaneously perceive the macroscopic motion trend and microscopic motion details of the target object, effectively overcoming the problems of high-frequency information loss and low-frequency oversmoothing when directly inputting raw coordinate values. By arranging the word embedding vectors at each time step in chronological order to form a word embedding vector sequence, the integrity of the temporal structure is preserved, providing a structured input foundation for subsequent temporal dependency modeling of multi-head self-attention encoding networks. Furthermore, Fourier feature mapping, as a deterministic nonlinear transformation, does not introduce additional learnable parameters, has low computational cost, and is numerically stable, making it suitable for deployment in real-time fusion prediction scenarios of multi-source track data.

[0083] In this embodiment of the invention, word embedding vectors are generated by mapping the observation data at each time step in the sparse observation data sequence to the Fourier feature space, and then arranged in chronological order to obtain a word embedding vector sequence. This scheme utilizes Fourier feature mapping to transform the original low-dimensional continuous coordinates into a high-dimensional multi-scale frequency domain feature representation, which can overcome the spectral bias problem of continuous coordinates in traditional coding methods. This allows the displacement prediction model to simultaneously capture the low-frequency global motion trend and high-frequency local motion details of the target object in a unified feature space, effectively solving the problem of spectral information loss caused by directly inputting the original coordinate values. At the same time, combined with masking to handle zero values ​​at missing locations, the consistency of the word embedding vector sequence in terms of dimension and the purity of its semantics are ensured, laying a high-quality input foundation for subsequent feature extraction and temporal modeling of the displacement prediction model.

[0084] Based on the above embodiments, the training steps of the displacement prediction model include: Acquire the trajectory observation data samples of the sample objects collected by each of the aforementioned sensors within a historical time period; Based on the detection principle of each sensor and the timestamp of each trajectory observation data sample, the trajectory observation data samples are merged into sparse observation data sequence samples on a unified time axis; The observation data samples at each time step within the sparse observation data sequence sample are respectively feature-encoded to obtain word embedding vector sequence samples; The displacement prediction model is trained under supervision based on the word embedding vector sequence samples and the true location and true velocity of the sample objects within the historical time period.

[0085] Specifically, during the training of the displacement prediction model, trajectory observation data samples of the sample object collected by each sensor within a historical time period can be acquired first. These trajectory observation data samples refer to the original observation records of the target object, carrying known true motion states, collected by each sensor within a historical time period. The true motion states include the true position and true velocity at the corresponding moment, obtained through a high-precision reference system or post-hoc backtracking methods. The true position refers to the normalized result of the actual geographical coordinates of the sample object at each moment within the historical time period, which can be obtained through a high-precision differential positioning system or post-hoc precise trajectory determination methods, serving as a supervisory label for the position regression task. The true velocity refers to the normalized result of the actual velocity of the sample object at each moment within the historical time period, also obtained through a high-precision reference system, serving as a supervisory label for the velocity regression task.

[0086] The sample object can be a known object such as an aircraft, ship or vehicle, whose actual motion state has been calibrated by a high-precision reference system to ensure the reliability of the tag data.

[0087] Then, using the detection principles of each sensor and the timestamps of each track observation data sample, the track observation data samples are merged into a sparse observation data sequence sample on a unified time axis. The sparse observation data sequence sample refers to the structured training data obtained after processing the track observation data samples according to the same merging rules as in the inference stage, including merging based on detection principles, redundancy merging, conflict retention, jump filtering, and zero-padding and masking for missing measurements. Its data structure and processing flow are completely consistent with the sparse observation data sequence in the inference stage to ensure the distribution consistency between the training data and the inference data.

[0088] Subsequently, feature encoding is performed on the observation data samples at each time step within the sparse observation data sequence to obtain word embedding vector sequence samples. The word embedding vector sequence samples refer to the word embedding vector sequences used for model training obtained by encoding the observation data samples at each time step within the sparse observation data sequence samples using the same Fourier feature mapping and other methods as in the inference stage. Their generation method is exactly the same as in the inference stage to ensure strict alignment between the training and inference stages at the feature encoding level.

[0089] Finally, the displacement prediction model is trained under supervision using word embedding vector sequence samples and the true positions and velocities of sample objects within historical time periods. Here, word embedding vector sequence samples, along with corresponding mask identifiers, are input into the initial prediction model to obtain the predicted position samples and predicted velocity samples output by the initial prediction model. Based on the predicted position samples and true positions output by the initial prediction model, the position loss can be calculated; based on the predicted velocity samples and true velocities output by the initial prediction model, the velocity loss can be calculated. Simultaneously, kinematic consistency loss is used to constrain the physical consistency between the predicted position samples and predicted velocity samples. The position loss, velocity loss, and kinematic consistency loss are weighted and summed to obtain the training loss. The parameters of the initial prediction model are updated using the backpropagation algorithm, including all learnable parameters in the mask-aware multi-head self-attention encoding network and the two-head kinematic awareness regression network. This process is repeated until the loss of the initial prediction model on the validation set converges to a preset threshold or a preset number of training epochs are reached, completing the training.

[0090] In this embodiment of the invention, historical flight path observation data samples are acquired and sparse observation data sequence samples and word embedding vector sequence samples are generated according to the merging and encoding rules consistent with the inference stage. These are then combined with the actual position and actual velocity to perform supervised training of the displacement prediction model. Through the consistent design of the training-inference data pipeline, the data distribution deviation between the training and inference stages is eliminated, enabling the displacement prediction model to fully learn the mapping rules between multi-source sparse observation data and the target motion state during training. This ensures the prediction accuracy and physical plausibility of the model in actual inference scenarios after training.

[0091] Based on the above embodiments, the supervised training of the displacement prediction model based on the word embedding vector sequence samples and the true location and true velocity of the sample objects within the historical time period includes: Using a sliding window of the same length as the current time window, target data samples are extracted from the sparse observation data sequence samples. Based on the word embedding vector sequence samples corresponding to the target data samples, an initial prediction model is applied to determine the predicted position sample and predicted velocity sample of the sample object at the next moment in the sliding window. Based on the predicted position samples, the predicted velocity samples, the actual position, and the actual velocity, a training loss is calculated, and the initial prediction model is iteratively trained based on the training loss to obtain the displacement prediction model.

[0092] Specifically, during the supervised training of the displacement prediction model, a sliding window with the same length as the current time window can be used to extract target data samples from the sparse observation data sequence samples. Then, using the word embedding vector sequence samples corresponding to the target data samples, the initial prediction model can be applied to determine the predicted position sample and the predicted velocity sample of the sample object at the next moment in the sliding window.

[0093] Then, the training loss is calculated using the predicted location samples, predicted velocity samples, actual location, and actual velocity. This training loss can be obtained by weighting and summing the regression loss, velocity consistency loss, weighted distance loss, geographic distance loss, and acceleration penalty loss.

[0094] The regression loss is the supervised loss calculated based on the deviation between the predicted location sample and the actual location at the corresponding time. It measures the prediction accuracy of the initial prediction model in the location regression task; the smaller the value, the closer the predicted location sample output by the initial prediction model is to the actual location. The regression loss can be calculated using regression loss functions such as mean squared error, smoothed L1 loss, or Huber loss, and the specific form can be selected according to the actual training requirements. When calculating the regression loss, the latitude and longitude coordinates in the predicted location sample can be subtracted from the latitude and longitude coordinates of the actual location at the corresponding time dimension by dimension, and the mean squared error or smoothed L1 loss can be calculated to obtain the regression loss.

[0095] Velocity consistency loss is a supervised loss calculated based on the deviation between the predicted velocity sample and the actual velocity at the corresponding time. It is used to constrain the predicted velocity output by the initial prediction model to approach the actual velocity, ensuring the prediction accuracy of the velocity regression task. Velocity consistency loss can also be calculated using regression loss functions such as mean squared error or smoothed L1 loss. When calculating velocity consistency loss, the difference between the predicted velocity sample and the actual velocity at the corresponding time can be calculated dimension by dimension, yielding the mean squared error or smoothed L1 loss.

[0096] Weighted distance loss refers to the distance loss obtained by using the reciprocal of the standard deviation of the true position at each time step within the sliding window as an adaptive weighting factor to weight the difference between the predicted position sample and the true position. The core idea is that when the motion amplitude of the sample object within the sliding window is large, the standard deviation of the true position is large, and the allowable deviation of the position prediction increases accordingly, while the weight decreases. Conversely, when the motion amplitude of the sample object is small, the standard deviation of the true position is small, and the requirement for position prediction accuracy is more stringent, while the weight increases accordingly. Through this adaptive weighting mechanism, the initial prediction model can dynamically adjust its focus on position prediction accuracy under different motion states, avoiding the training loss from dominating the training direction due to large absolute deviations in large-scale motion scenarios, while enhancing sensitivity to small deviations in small-scale motion scenarios.

[0097] When calculating the weighted distance loss, we can first statistically analyze the dispersion of the true location in the longitude and latitude directions at all times within the sliding window, and calculate the longitude standard deviation and latitude standard deviation respectively. The reciprocal of the longitude standard deviation is used as the weight of the longitude difference between the predicted location sample and the true location, and the reciprocal of the latitude standard deviation is used as the weight of the latitude difference between the predicted location sample and the true location. Using the weights of the longitude difference and the latitude difference, we can perform a weighted summation of the longitude difference and the latitude difference to obtain the weighted distance loss.

[0098] Geographic distance loss refers to the distance loss calculated after converting the coordinate difference between the predicted location sample and the actual location into the physical distance in actual geographic space. Since the actual physical distance corresponding to latitude and longitude coordinates is different at different locations on the Earth's surface, directly using the coordinate difference to calculate the loss cannot accurately reflect the true physical meaning of the prediction bias. Therefore, it is necessary to use geographic distance transformation to map the coordinate difference into the actual distance in meters or kilometers, thereby ensuring the consistency of the training loss measurement in physical space.

[0099] When calculating geographic distance loss, the Haversine or Vincenty formulas can be used to convert the difference between the latitude and longitude coordinates of the predicted location sample and the actual location into the actual physical distance on the Earth's surface, in meters or kilometers. This actual geographic distance can be used as the geographic distance loss, or its mean squared error can be calculated and used as the geographic distance loss. Since the actual physical distance varies at different latitudes and longitudes, directly calculating the loss from the coordinate difference can lead to the same numerical deviation having different physical meanings at different geographical locations. Geographic distance conversion can eliminate this spatial inhomogeneity, giving the training loss a uniform metric in physical space.

[0100] Acceleration penalty loss refers to the penalty loss calculated based on the rate of change of the predicted velocity sample at each time step within the sliding window. It is used to constrain the smoothness of the predicted velocity sample output by the initial prediction model in the time dimension, prevent drastic jumps in the predicted velocity sample, and thus ensure the kinematic rationality of the predicted trajectory.

[0101] When calculating the acceleration penalty loss, the difference between the predicted velocity samples at adjacent times within the sliding window and divided by the time interval can be used to obtain the rate of change of velocity at each time moment, which is an approximation of the predicted acceleration. The sum of the mean square or absolute values ​​of the rate of change of velocity at each time moment is then calculated as the acceleration penalty loss.

[0102] Training loss refers to the comprehensive loss function value obtained by weighting and summing regression loss, velocity consistency loss, weighted distance loss, geographic distance loss, and acceleration penalty loss according to preset weight coefficients. It is used to comprehensively measure the model's overall performance across multiple dimensions, including location accuracy, velocity accuracy, physical distance accuracy, and motion smoothness, and guides the iterative optimization of the initial prediction model's parameters. The weight coefficients of each loss can be adjusted according to actual training needs to balance the contribution ratio of each loss during the training process. For example, the weight coefficients of regression loss, velocity consistency loss, weighted distance loss, geographic distance loss, and acceleration penalty loss can be 0.1, 0.5, 0.1, 0.1, and 0.2, respectively.

[0103] By introducing a multi-dimensional composite loss function comprising regression loss, velocity consistency loss, weighted distance loss, geographic distance loss, and acceleration penalty loss, the model can be jointly constrained from five dimensions: position accuracy, velocity accuracy, adaptive motion amplitude perception, physical spatial distance measurement, and motion smoothness. Specifically, the weighted distance loss adaptively adjusts the sensitivity to positional bias through standard deviation, ensuring stable training performance of the initial prediction model under different motion amplitudes. The geographic distance loss eliminates the problem of uneven measurement of latitude and longitude coordinates in physical space, giving the training loss a unified physical meaning. The acceleration penalty loss constrains the temporal smoothness of the predicted velocity, preventing velocity jumps in the predicted trajectory that do not conform to physical laws. The synergistic optimization of the five losses enables the initial prediction model to fully learn the accurate mapping relationship between multi-source sparse observation data and the target motion state during training, ensuring that the predicted position and velocity output by the model in actual inference scenarios after training are both highly accurate and satisfy physical kinematic constraints.

[0104] In this embodiment of the invention, a multi-dimensional composite loss function system is constructed, encompassing position accuracy, velocity accuracy, adaptive motion amplitude perception, physical space distance measurement, and motion smoothness. Weighted distance loss enables adaptive adjustment of position prediction accuracy under different motion amplitudes; geographical distance loss eliminates the problem of uneven measurement of latitude and longitude coordinates in physical space; and acceleration penalty loss ensures the kinematic rationality of the predicted trajectory. The synergistic optimization of these five losses provides comprehensive supervision and constraints to the model during training, significantly improving the prediction accuracy, physical rationality, and trajectory smoothness of the displacement prediction model under multi-source sparse track data conditions after training.

[0105] Based on the above embodiments, determining the extrapolated position of the target object at the next moment based on the predicted velocity and the trajectory observation data corresponding to the current moment includes: If the actual position of the target object at the current moment is empty, then the predicted position corresponding to the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position; If the actual position of the target object at the current moment is not empty, then the actual position of the target object at the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position.

[0106] Specifically, to ensure that the extrapolation position calculation process is as close as possible to the actual state of the target object, when determining the extrapolation position of the target object at the next moment, an anchor point position is introduced as the starting reference point for the extrapolation calculation, and different anchor point positions are used to calculate the extrapolation position for the following two cases: Scenario 1: The actual location of the target object at the current moment is empty. In this case, the predicted location corresponding to the current moment is used as the anchor point, and extrapolation is performed using the predicted velocity to obtain the extrapolated location. This extrapolated location can be equal to the anchor point plus the product of the predicted velocity and the time step. Since there are no actual observations available at the current moment, using the predicted location corresponding to the current moment as the anchor point ensures that the extrapolation calculation can be performed normally under any observation conditions. Moreover, since the anchor point is itself the predicted location, the extrapolated location is a secondary extrapolation result based on the predicted location, and its accuracy depends on the reliability of the predicted location.

[0107] Scenario 2: The actual position of the target object at the current moment is not empty. In this case, the actual position of the target object at the current moment is used as the anchor point, and the extrapolated position is obtained by extrapolating using the predicted velocity. In this case, since the anchor point position is the actual observation value, there is no cumulative prediction error. Therefore, the extrapolated position has high accuracy and can effectively correct any deviations that may exist in the predicted position directly output by the displacement prediction model, eliminating the impact of cumulative prediction error on the extrapolation accuracy.

[0108] In this embodiment of the invention, by adaptively selecting the anchor point position based on whether the actual position at the current moment is empty, and obtaining the extrapolated position by kinematic extrapolation based on the predicted velocity, the accumulated error can be eliminated by using the actual position as the anchor point when there are actual observations, and the extrapolation continuity can be ensured by using the predicted position as the anchor point when there are no actual observations. This allows the extrapolated position to maintain high reliability under different observation sparsity conditions, providing high-quality kinematic prior information for subsequent fusion and correction with the prediction results of the displacement prediction model, and effectively improving the accuracy and stability of the final prediction results over a long time scale.

[0109] To quantitatively analyze the final prediction results, this embodiment of the invention uses the kilometer-level Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) as metrics, defined based on the Haversine distance between the predicted and actual positions converted to kilometers. Using the multi-source track data fusion prediction method provided in this embodiment, the displacement prediction model achieves low convergence levels in both kilometer-level MAE and RMSE on the test set, meeting operational accuracy criteria. Experimental results demonstrate that the multi-source track data fusion prediction method provided in this embodiment can effectively solve the problem of high-precision prediction for sparse multi-source tracks, offering a new technical approach for multi-source track prediction.

[0110] like Figure 2 As shown, based on the above embodiments, this embodiment of the invention provides a multi-source track data fusion prediction device, including: Data acquisition module 21 is used to acquire track observation data of the target object collected by multiple sensors in the current time window; the current time window includes the current moment and multiple historical moments; The time unification module 22 is used to merge the track observation data into a sparse observation data sequence on a unified time axis based on the detection principle of each sensor and the timestamp of each track observation data. The displacement prediction module 23 is used to perform feature encoding on the observation data at each time step in the sparse observation data sequence to obtain a word embedding vector sequence, and based on the word embedding vector sequence, apply the displacement prediction model to determine the predicted position and predicted velocity of the target object at the next time step. The position prediction module 24 is used to determine the extrapolated position of the target object at the next moment based on the predicted velocity and the track observation data corresponding to the current moment, and to determine the final predicted position of the target object based on the predicted position and the extrapolated position.

[0111] Specifically, the functions of each module in the multi-source trajectory data fusion prediction device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0112] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the multi-source track data fusion prediction method provided in the above embodiments.

[0113] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0114] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the multi-source track data fusion prediction method provided in the above embodiments.

[0115] In another aspect, the present invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the multi-source track data fusion prediction method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-source trajectory data fusion prediction method, characterized in that, include: Acquire trajectory observation data of the target object collected by multiple sensors within the current time window; The current time window includes the current moment and multiple historical moments; Based on the detection principle of each sensor and the timestamp of each trajectory observation data, the trajectory observation data are merged into a sparse observation data sequence on a unified time axis; The observation data at each time step within the sparse observation data sequence are feature-encoded to obtain a word embedding vector sequence. Based on the word embedding vector sequence, a displacement prediction model is applied to determine the predicted position and predicted velocity of the target object at the next time step. Based on the predicted velocity and the trajectory observation data corresponding to the current moment, the extrapolated position of the target object at the next moment is determined, and based on the predicted position and the extrapolated position, the final predicted position of the target object is determined.

2. The multi-source trajectory data fusion prediction method according to claim 1, characterized in that, Based on the detection principles of each sensor and the timestamps of each trajectory observation data, the trajectory observation data are grouped into a sparse observation sequence on a unified time axis, including: Based on the detection principle of each sensor, the track observation data belonging to the same detection principle are merged into a single logical track to obtain multiple logical tracks. For any logical trajectory, merge the track observation data with the same timestamp and the same data in the logical trajectory, retain the track observation data with the same timestamp but different data in the logical trajectory, filter out the track observation data in the logical trajectory whose jump exceeds a preset threshold, pad the track observation data in the logical trajectory with missing data with zeros, and mark the validity of the zero-padding positions based on the mask identifier.

3. The multi-source trajectory data fusion prediction method according to claim 2, characterized in that, The step of determining the predicted position and velocity of the target object at the next moment by applying a displacement prediction model based on the word embedding vector sequence includes: Determine the position embedding vector and time embedding vector of the word embedding vector at each time step in the word embedding vector sequence, and construct a word sequence based on the position embedding vector, the time embedding vector and the word embedding vector at each time step in the word embedding vector sequence; Based on the lexical sequence, the displacement prediction model is applied to obtain the predicted position and the predicted velocity.

4. The multi-source trajectory data fusion prediction method according to claim 3, characterized in that, The displacement prediction model includes a mask-aware multi-head self-attention encoding network and a two-head kinematics-aware regression network; the step of applying the displacement prediction model based on the word sequence to obtain the predicted position and the predicted velocity includes: The word sequence is input into the displacement prediction model. Each attention head in the multi-head self-attention encoding network calculates attention weights based on the word sequence and the mask identifier, calculates a context vector based on the attention weights, and generates temporal context features based on each context vector. The position sensing head and velocity sensing head in the dual-head kinematics perception and regression network output the predicted position and the predicted velocity based on the temporal context features, respectively.

5. The multi-source trajectory data fusion prediction method according to claim 1, characterized in that, The step of performing feature encoding on the observation data at each time point within the sparse observation data sequence to obtain a word embedding vector sequence includes: The observation data at each time step in the sparse observation data sequence are mapped to the Fourier feature space to generate word embedding vectors. The word embedding vectors are arranged in chronological order to obtain the word embedding vector sequence.

6. The multi-source track data fusion prediction method according to any one of claims 1-5, characterized in that, The training steps for the displacement prediction model include: Acquire the trajectory observation data samples of the sample objects collected by each of the aforementioned sensors within a historical time period; Based on the detection principle of each sensor and the timestamp of each trajectory observation data sample, the trajectory observation data samples are merged into sparse observation data sequence samples on a unified time axis; The observation data samples at each time step within the sparse observation data sequence sample are respectively feature-encoded to obtain word embedding vector sequence samples; The displacement prediction model is trained under supervision based on the word embedding vector sequence samples and the true location and true velocity of the sample objects within the historical time period.

7. The multi-source trajectory data fusion prediction method according to claim 6, characterized in that, The supervised training of the displacement prediction model based on the word embedding vector sequence samples and the true location and true velocity of the sample objects within the historical time period includes: Using a sliding window of the same length as the current time window, target data samples are extracted from the sparse observation data sequence samples. Based on the word embedding vector sequence samples corresponding to the target data samples, an initial prediction model is applied to determine the predicted position sample and predicted velocity sample of the sample object at the next moment in the sliding window. Based on the predicted position samples, the predicted velocity samples, the actual position, and the actual velocity, a training loss is calculated, and the initial prediction model is iteratively trained based on the training loss to obtain the displacement prediction model.

8. The multi-source trajectory data fusion prediction method according to claim 7, characterized in that, The calculation of training loss based on the predicted location sample, the predicted velocity sample, the actual location, and the actual velocity includes: Based on the predicted location samples and the corresponding actual locations at the given time, the regression loss is calculated; Based on the predicted velocity samples and the actual velocities at the corresponding times, calculate the velocity consistency loss; Calculate the standard deviation of each of the true positions within the sliding window, and based on the standard deviation, weight the difference between the predicted position sample and the true position at the corresponding time to obtain the weighted distance loss; The difference between the predicted location sample and the actual location at the corresponding time is converted into the actual geographical distance, and the actual geographical distance is used as the geographical distance loss. Based on the predicted velocity samples at each time point within the sliding window, the velocity change rate at each time point within the sliding window is solved, and based on the velocity change rate at each time point within the sliding window, the acceleration penalty loss is calculated. The training loss is calculated based on the regression loss, the velocity consistency loss, the weighted distance loss, the geographic distance loss, and the acceleration penalty loss.

9. The multi-source track data fusion prediction method according to any one of claims 1-5, characterized in that, The step of determining the extrapolated position of the target object at the next moment based on the predicted velocity and the trajectory observation data corresponding to the current moment includes: If the actual position of the target object at the current moment is empty, then the predicted position corresponding to the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position; If the actual position of the target object at the current moment is not empty, then the actual position of the target object at the current moment is used as the anchor position, and extrapolation is performed based on the predicted velocity to obtain the extrapolated position.

10. A multi-source trajectory data fusion and prediction device, characterized in that, include: The data acquisition module is used to acquire track observation data of the target object collected by multiple sensors in the current time window; The current time window includes the current moment and multiple historical moments; The time unification module is used to merge the track observation data into a sparse observation data sequence on a unified time axis based on the detection principle of each sensor and the timestamp of each track observation data. The displacement prediction module is used to perform feature encoding on the observation data at each time step within the sparse observation data sequence to obtain a word embedding vector sequence, and based on the word embedding vector sequence, apply the displacement prediction model to determine the predicted position and predicted velocity of the target object at the next time step. The position prediction module is used to determine the extrapolated position of the target object at the next moment based on the predicted velocity and the track observation data corresponding to the current moment, and to determine the final predicted position of the target object based on the predicted position and the extrapolated position.

Images