Method, device and equipment for distinguishing moving target trajectory based on heterogeneous feature fusion

Abstract

This invention provides a method, apparatus, and device for distinguishing moving target trajectories based on heterogeneous feature fusion, comprising: acquiring continuous motion trajectories of different types of moving targets; extracting discriminative temporal motion features and spatial morphological features from the continuous motion trajectories; and utilizing the complementarity between the spatial morphological features and temporal motion features to achieve deep fusion of heterogeneous information. To bridge the semantic gap between heterogeneous features, a cross-modal fusion network is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism to obtain enhanced features after fusion. Training samples are constructed based on the enhanced features and the corresponding moving target type labels to train a moving target trajectory discrimination model. For the continuous motion trajectory of the moving target to be judged, the trained moving target trajectory discrimination model is used to classify the moving target type and output the classification result, thereby achieving deep fusion of heterogeneous information and collaborative decision-making.

Description

Technical Field

[0001] This invention relates to the field of target trajectory analysis technology, specifically to a method, apparatus, and device for distinguishing the trajectory of a moving target based on heterogeneous feature fusion. Background Technology

[0002] In wide-area infrared search systems, infrared imaging equipment needs to continuously monitor low-altitude areas under complex backgrounds and interference from various moving targets. Because small drones and birds exhibit highly similar pixel areas, grayscale intensity distributions, and morphological features in single-frame infrared images, traditional single-frame detection methods relying solely on static appearance information struggle to effectively distinguish between real and non-threat targets, leading to numerous false alarms. These false alarms not only increase the system's analytical burden but can also negatively impact early warning and response decisions. Therefore, under passive infrared detection conditions, mining target motion patterns through multi-frame observations to differentiate the trajectories of different dynamic targets has become a crucial means of improving the reliability of infrared moving target identification.

[0003] In existing research, multi-frame correlation and trajectory analysis methods are mostly applied in the radar field, typically relying on directly obtainable target state measurements such as range, velocity, and angle. Target trajectories are estimated and classified using methods such as Kalman filtering, joint probabilistic data correlation, and multi-hypothesis tracking. However, infrared search systems lack active range measurement capabilities, acquiring only the projected points of the target on a two-dimensional image plane, reducing the target's true three-dimensional motion state to two-dimensional positional changes. Furthermore, the step-gaze scanning method used to achieve wide field-of-view coverage results in a low refresh rate, leading to large target displacement spans and complex motion patterns between adjacent frames, making traditional trajectory analysis methods based on deterministic motion models difficult to apply. Moreover, in wide-area infrared search scenarios, the number of false alarm targets is significantly higher than that of real targets, and trajectory differentiation strategies relying solely on two-dimensional displacement features face a high risk of confusion.

[0004] To improve trajectory-level recognition capabilities, some studies have attempted to introduce two-dimensional or multi-dimensional features for trajectory classification, including simple temporal features such as velocity estimation, grayscale changes, and area changes. However, these features have limited dimensions and cannot fully express the differences in spatial morphology and dynamic laws of targets. On the other hand, heterogeneous feature fusion methods in deep learning have shown good performance in multimodal recognition tasks, but existing research on small infrared targets mostly focuses on multi-scale fusion of features from the same source image, and has not yet fully explored the complementarity of target trajectories in temporal dynamics and spatial geometry, nor does it have a suitable cross-modal alignment mechanism for infrared point sequence.

[0005] In infrared search environments characterized by low frame rates, numerous false alarms, and high background interference, relying solely on single-modal features for trajectory differentiation is insufficient to meet the demands of UAV identification and threat confirmation. A key technical challenge in this field is how to simultaneously utilize target temporal motion attributes such as grayscale variations, area, displacement vectors, and first- and second-order motion parameters, along with the spatial geometric features of local trajectories, and employ an effective cross-modal fusion mechanism to bridge the semantic differences between these two types of features, thereby achieving highly reliable differentiation between real targets and false alarm trajectories. Summary of the Invention

[0006] To address the problems existing in the prior art, the present invention provides a method, apparatus and device for distinguishing the trajectory of a moving target based on heterogeneous feature fusion.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] On the one hand, a method for distinguishing the trajectory of moving targets based on heterogeneous feature fusion is provided, including the following steps:

[0009] To acquire the continuous motion trajectory of different types of moving targets;

[0010] From each continuous motion trajectory, a feature extraction network is used to extract trajectory features used to distinguish target types, including temporal motion features and spatial morphological features;

[0011] A cross-modal fusion network is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism to obtain enhanced features after fusion;

[0012] Based on the enhanced features and the corresponding moving target type labels, training samples are constructed, and the parameters of the feature extraction network and the cross-modal fusion network are jointly optimized using the loss function to obtain a trained moving target trajectory discrimination model.

[0013] For the continuous motion trajectory of the moving target to be identified, the trained moving target trajectory discrimination model is used to classify the moving target type and output the classification result.

[0014] In the above scheme, the continuous motion trajectory of the moving target is a sequence of dots output by the infrared search and track system.

[0015] On the other hand, a moving target trajectory differentiation device based on heterogeneous feature fusion is provided, comprising:

[0016] The first module is used to acquire the continuous motion trajectory of different types of moving targets;

[0017] The second module is used to extract trajectory features, including temporal motion features and spatial morphological features, from each continuous motion trajectory using a feature extraction network to distinguish target types.

[0018] The third module is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism using a cross-modal fusion network to obtain the fused enhanced features;

[0019] The fourth module is used to construct training samples based on the enhanced features and the corresponding moving target type labels, and to jointly optimize the parameters of the feature extraction network and the cross-modal fusion network using a loss function to obtain a trained moving target trajectory discrimination model.

[0020] The fifth module is used to classify the type of moving target by using a trained moving target trajectory discrimination model for the continuous motion trajectory of the moving target to be judged, and output the classification result.

[0021] On the other hand, the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-mentioned method for distinguishing moving target trajectories based on heterogeneous feature fusion.

[0022] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for distinguishing moving target trajectories based on heterogeneous feature fusion.

[0023] On the other hand, the present invention provides a computer program product stored on a computer-readable storage medium and including computer instructions that, when executed by a processor, cause an electronic device to implement the steps of the above-described method for distinguishing moving target trajectories based on heterogeneous feature fusion.

[0024] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0025] This invention proposes a method for distinguishing moving target trajectories based on heterogeneous feature fusion. It acquires continuous motion trajectories of different types of moving targets, extracts temporal motion features and spatial morphological features from these trajectories, and leverages the complementarity of these features to achieve deep fusion of heterogeneous information and intelligent decision-making. Specifically, temporal motion features capture the motion patterns of the target, while spatial morphological features are described by the triangular shape of the trajectory formed by three consecutive points, from which two-dimensional morphological characteristics are extracted. To bridge the semantic gap between heterogeneous features, a cross-modal fusion network is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism, resulting in enhanced features. Based on these enhanced features and corresponding moving target type labels, training samples are constructed. A loss function is used to jointly optimize the parameters of the feature extraction network and the cross-modal fusion network, resulting in a trained moving target trajectory distinguishing model. For the continuous motion trajectory of the target to be distinguished, the trained model is used to classify the target type and output the classification results, achieving deep fusion of heterogeneous information and collaborative decision-making. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a method for distinguishing moving target trajectories based on heterogeneous feature fusion in one embodiment;

[0028] Figure 2 The histogram of variance distribution of the triangles obtained from the statistical analysis of drone and bird trajectory segments is shown. Figure 2 (a) is the histogram of variance distribution of the triangle of the UAV trajectory segment. Figure 2 (b) is the histogram of variance distribution of the triangle segment of the bird's trajectory;

[0029] Figure 3 This is a histogram showing the maximum value distribution of triangles representing drone and bird trajectory segments obtained through statistical analysis. Figure 3 (a) is a histogram of the maximum value distribution of the triangle of the UAV trajectory segment. Figure 3 (b) is a histogram of the maximum value distribution of the triangle segment of the bird's trajectory;

[0030] Figure 4 This is a framework diagram of an embodiment of a moving target trajectory differentiation model;

[0031] Figure 5This is a block diagram of an electronic device in one embodiment. Detailed Implementation

[0032] The technical solution of the present invention will now be clearly and completely described through specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0033] In one embodiment, reference is made to Figure 1 This paper provides a method for distinguishing the trajectory of moving targets based on heterogeneous feature fusion, including the following steps:

[0034] To acquire the continuous motion trajectory of different types of moving targets;

[0035] From each continuous motion trajectory, a feature extraction network is used to extract trajectory features used to distinguish target types, including temporal motion features and spatial morphological features;

[0036] A cross-modal fusion network is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism to obtain enhanced features after fusion;

[0037] Based on the enhanced features and the corresponding moving target type labels, training samples are constructed, and the parameters of the feature extraction network and the cross-modal fusion network are jointly optimized using the loss function to obtain a trained moving target trajectory discrimination model.

[0038] For the continuous motion trajectory of the moving target to be identified, the trained moving target trajectory discrimination model is used to classify the moving target type and output the classification result.

[0039] In the above embodiments, the continuous motion trajectory of the moving target is a sequence of points output by the infrared search and track system. The moving target is not limited, and may include birds and drones. To distinguish trajectories, the above embodiments first extract discriminative features from the continuous motion trajectory of the moving target. Therefore, a trajectory feature representation capable of effectively distinguishing different moving targets (especially birds and drones) is constructed from both temporal motion features and spatial morphological features. The temporal motion features are based on continuous measurements of the target's position, appearance, velocity, and acceleration, while the spatial morphological features are derived from the shape of the target's trajectory in the azimuth-pitch plane.

[0040] Furthermore, in the above embodiments, extracting trajectory features from continuous motion trajectories to distinguish target types includes:

[0041] Group every 3 consecutive points in the continuous motion trajectory;

[0042] Extract the temporal motion features and spatial morphological features corresponding to each set of points.

[0043] Different moving targets exhibit significant differences in their motion characteristics, which are reflected in velocity stability, acceleration variation, and positional changes. For example, birds and drones differ significantly. Drones are typically controlled by navigation systems, resulting in relatively stable flight trajectories, while birds' movements are influenced by biological behavior and environmental factors, exhibiting stronger randomness and nonlinearity. Specifically, the temporal motion characteristics are constructed based on at least one of the following continuous point measurement information of the target: azimuth angle, pitch angle, intensity value, pixel area, local signal-to-noise ratio, angular velocity, angular acceleration, and correlation distance. The azimuth angle refers to the deviation angle of the target's center point relative to true north; the pitch angle refers to the deviation angle of the target's center point relative to the horizontal plane; the intensity value refers to the target amplitude in the infrared data of the original moving target's continuous motion trajectory; the pixel area refers to the number of pixels in the segmented connected region of the target; the local signal-to-noise ratio refers to the intensity ratio of the target signal relative to the background noise within a local window; the angular velocity refers to the target's velocity in the azimuth and pitch planes; the angular acceleration refers to the target's acceleration in the azimuth and pitch planes; and the correlation distance refers to the angular distance between adjacent points in the target trajectory.

[0044] In one embodiment, three consecutive points in a continuous motion trajectory are grouped together, and the temporal motion features and spatial morphological features corresponding to each group of points are extracted. The extraction of the temporal motion features corresponding to each group of points includes:

[0045] (1.1) Calculate the 11-dimensional trajectory motion feature vector for each point in each group of points, expressed as:

[0046] ;

[0047] in and These are the azimuth and elevation angles corresponding to the target, respectively. For local signal-to-noise ratio, The average intensity This represents the maximum intensity. For pixel area, The distance between points in the trajectory. , , and These are the azimuth angular velocity, pitch angular velocity, azimuth acceleration, and pitch acceleration of the dot, respectively.

[0048] (1.2) The 11-dimensional trajectory motion feature vectors corresponding to the three consecutive points in each group of points form an input feature matrix of shape 3×11, where the row dimension 3 corresponds to the number of consecutive points and the column dimension 11 corresponds to the feature degree of each point.

[0049] (1.3) Input the input feature matrix of each set of points into a one-dimensional convolutional neural network for feature extraction to obtain the temporal motion features corresponding to each set of points.

[0050] To more effectively extract motion features from continuous motion trajectories, a feature extraction module based on a one-dimensional convolutional neural network was designed. The input feature matrix of the one-dimensional convolutional neural network is shown below. The shape is 3×11, where the row dimension of 3 corresponds to the number of consecutive dots, i.e., the number of channels for the one-dimensional feature, and the column dimension of 11 corresponds to the feature intensity of each dot, i.e., the length of the one-dimensional feature. The one-dimensional convolutional neural network consists of a one-dimensional convolutional layer, a one-dimensional batch normalization layer, a ReLU activation function, and a one-dimensional max pooling layer. The one-dimensional convolutional layer uses one-dimensional convolution with a kernel size of 3 and a stride of 1, and zero-padding at both ends. The length of the input one-dimensional feature remains unchanged before and after convolution. The output of the one-dimensional convolutional layer passes through a one-dimensional batch normalization layer and a ReLU activation function. Then, a one-dimensional max pooling with a stride of 3 is applied. The temporal motion features extracted by the feature extraction module based on the one-dimensional convolutional neural network are... Its channel count is expanded to 8, and its length is shortened to 9. The above process can be represented as:

[0051] ;

[0052] In the formula This represents the activation function. This represents a one-dimensional batch normalization layer. This represents a one-dimensional convolutional layer.

[0053] Regarding trajectory patterns, let's take drones and birds as examples: Drone trajectories typically appear as smooth straight lines, arcs, or multi-segment broken lines, with fewer high-frequency sharp turns or jitters, exhibiting high stability. In contrast, bird flight trajectories are more irregular on a short timescale, potentially exhibiting random turns, sudden ascents, or descents.

[0054] On a two-dimensional azimuth-pitch plane, three consecutive points from the candidate trajectory are marked, connected sequentially, and then joined end-to-end to form a triangle. Bird flight paths often exhibit significant randomness, influenced by wind, wing movements, and biological behavior, resulting in large fluctuations and irregularities in the area of ​​the trajectory triangle. In contrast, drones typically move along stable flight paths, and even when they change direction, their trajectory changes are relatively smooth; therefore, the area of ​​their triangles is usually smaller, and the variation is smaller over multiple cycles. Furthermore, regarding the shape of the triangle, drones, constrained by their navigation control systems, typically exhibit gentler changes in direction, resulting in triangles that are usually extremely flat and obtuse-angled. Moving targets such as birds may suddenly turn or jerk, causing triangles in different cycles to potentially exhibit acute, obtuse, or extremely flat shapes.

[0055] Figure 2 The histogram of variance distribution of the triangles obtained from the statistical analysis of drone and bird trajectory segments is shown. Figure 2 (a) is the histogram of variance distribution of the triangle of the UAV trajectory segment. Figure 2 (b) is a histogram of the variance distribution of the triangle segment of the bird's flight path. From Figure 2 It can be observed that the angular variance of the triangles formed by the drone trajectory segments is concentrated in the higher value region, indicating that the angular variation range is large and the triangles are mainly obtuse. Figure 3 This is a histogram showing the maximum value distribution of triangles representing drone and bird trajectory segments obtained through statistical analysis. Figure 3 (a) is a histogram of the maximum value distribution of the triangle of the UAV trajectory segment. Figure 3 (b) is a histogram of the maximum value distribution of triangles representing bird trajectory segments. The maximum angle of the UAV is close to 180 degrees, indicating that the triangles formed by the detection points tend to have a linear distribution, reflecting the smoothness of the UAV's trajectory. In contrast, the angular variance and maximum angle distribution of birds are more dispersed, with a large number of samples in different types of triangles, showing diversity and revealing that bird movement is often accompanied by frequent turns and abrupt changes.

[0056] Based on this, in one embodiment, a method for extracting the spatial morphological features corresponding to each set of dots is proposed, including the following steps:

[0057] (2.1) For each set of points, connect and close the three consecutive points on the azimuth-elevation plane in sequence to form the triangular trajectory shape corresponding to each set of points, and obtain the azimuth-elevation plane map with the triangular trajectory shape marked on each set of points.

[0058] (2.2) Input the azimuth-elevation plane map corresponding to each set of points and labeled with the shape of the triangle trajectory into the ResNet network and extract the output features of the global pooling layer;

[0059] (2.3) Adjust the channels of the output features of the global pooling layer to obtain the spatial morphological features corresponding to each group of points. The number of channels of the spatial morphological features corresponding to each group of points is the same as the number of channels of the temporal motion features corresponding to each group of points.

[0060] In one embodiment, to extract the spatial morphological features corresponding to each set of points, the input to the ResNet network is an azimuth-elevation plane map labeled with the triangular trajectory shape corresponding to each set of points. Where 1 represents the number of image channels, and 224×224 represents the input image size. Extract the output features of the global pooling layer. One-dimensional convolution is used to increase the number of channels to 8 as spatial morphological features. The above process can be represented as:

[0061] ;

[0062] ;

[0063] In the formula This represents the activation function. Indicates the batch normalization layer. This represents a one-dimensional convolutional layer.

[0064] One-dimensional temporal motion features and two-dimensional spatial morphological features differ significantly in data structure, statistical properties, and physical meaning. Temporal motion features primarily characterize the motion pattern of a target over time, such as velocity and acceleration, exhibiting strict time dependence. Spatial morphological features, on the other hand, describe the shape of the target's motion in space, such as curvature and smoothness, emphasizing spatial geometric relationships. For trajectory discrimination tasks, the reasonable fusion of temporal motion features and spatial morphological features is crucial for improving the accuracy and robustness of target recognition. However, traditional feature fusion methods, such as channel splicing or element-wise addition, struggle to fully model the deep correlation between the two heterogeneous features, potentially leading to information redundancy or modal conflicts, thus affecting the model's discriminative ability. To address these issues, one embodiment proposes a heterogeneous feature fusion method. This method fuses the temporal motion features and spatial morphological features corresponding to each set of points using a cross-modal self-attention mechanism, resulting in enhanced fused features for each set of points. The core idea of ​​this method is to use the transformed features as weights to correct the features to be fused. A self-attention mechanism is employed for feature transformation, enabling the model to learn the correlation between temporal motion features and spatial morphological features. Specifically, the temporal motion features and spatial morphological features corresponding to each set of points are fused through a cross-modal self-attention mechanism to obtain the fused enhanced features for each set of points, including the following steps:

[0065] (3.1) Obtain the enhanced temporal motion features corresponding to each set of points through the following steps:

[0066] (3.1.1) For each set of points, its temporal motion features are mapped to a query vector through learnable parameters, and its spatial morphological features are mapped to a key vector and a value vector through learnable parameters, respectively.

[0067] (3.1.2) Calculate the similarity between the query vector and the transpose of the key vector, and then normalize the result to obtain the similarity matrix;

[0068] (3.1.3) Multiplying the similarity matrix with the value vector yields the temporal motion features enhanced by spatial morphological features;

[0069] (3.1.4) Input the temporal motion features enhanced by spatial morphology into the feedforward network of the residual structure, and output the final enhanced temporal motion features;

[0070] (3.2) Obtain the enhanced spatial morphological features corresponding to each set of dots through the following steps;

[0071] (3.2.1) For each set of points, its spatial morphological features are mapped to a query vector through learnable parameters, and its temporal motion features are mapped to a key vector and a value vector through learnable parameters, respectively.

[0072] (3.2.2) Calculate the similarity between the query vector and the transpose of the key vector, and then normalize the result to obtain the similarity matrix;

[0073] (3.2.3) Multiplying the similarity matrix with the value vector yields the spatial morphological features enhanced by temporal motion characteristics;

[0074] (3.2.4) Input the spatial morphological features enhanced by temporal motion features into the feedforward network of the residual structure, and output the final enhanced spatial morphological features;

[0075] (3.3) The enhanced temporal motion features and spatial morphological features corresponding to each set of points are fused to obtain the fused enhanced features corresponding to each set of points.

[0076] Temporal motion characteristics and spatial morphological features A common input feature enhancement module learns parameters that map to query vectors, key vectors, and value vectors. Unlike standard self-attention mechanisms, a cross-modal strategy is introduced. This utilizes temporal motion features. For example, temporal motion characteristics Learnable parameters are mapped to query vectors Spatial morphological characteristics Learnable parameters are mapped to key vectors. Sum value vector Next, cross-modal enhancement is performed according to the following formula:

[0077] Calculate query vector transpose of the key vector The similarity between them was determined and normalized to obtain a similarity matrix. ;

[0078] ;

[0079] in, The number of channels for the feature. Similarity matrix. Reflects the characteristics of temporal motion The degree of correlation between each component and morphological features.

[0080] Multiplying the similarity matrix by the value vector yields the temporal motion features enhanced by spatial morphological characteristics. ;

[0081] ;

[0082] Temporal motion characteristics enhanced by spatial morphology The input residual structure is fed forward network FFN, which outputs the final enhanced temporal motion features. ,in This represents the features after passing through the feedforward network FFN.

[0083] Multiplying the similarity matrix by the value vector highlights highly correlated components, guiding the model to learn modality-independent features and features with high consistency across different modalities. Subsequently, similar to a regular Transformer block, the cross-modal enhanced temporal features are fed into the feedforward network of the residual structure to obtain the final result. Similarly, based on the same method steps described above, cross-modal enhanced spatial morphological features can be obtained.

[0084] Reference Figure 4 This diagram illustrates the framework of a moving target trajectory discrimination model. Motion feature vectors are input into a feature extraction module of a one-dimensional convolutional neural network (CNN). The CNN sequentially comprises a one-dimensional convolutional layer, a one-dimensional batch normalization layer, a ReLU activation function, and a one-dimensional max-pooling layer. The temporal motion features extracted by the feature extraction module based on the CNN are... Each set of points corresponds to an azimuth-elevation planar image of the described triangular trajectory shape, which is input into a ResNet network to output features. Output features After passing through a one-dimensional convolutional layer, a batch normalization layer, and a ReLU activation function, the output spatial morphological features are obtained. Temporal motion characteristics Learnable parameters are mapped to query vectors , , Spatial morphological features Learnable parameters are mapped to key vectors and value vectors respectively. , , Enhanced temporal motion features are obtained by feature fusion through a cross-modal self-attention mechanism. and enhanced spatial morphological features The heterogeneous feature fusion operation does not change the shape of the features. To ensure consistency, the enhanced temporal motion features are first... and enhanced spatial morphological features The features are reorganized into a form with a length of 8. Then, a one-dimensional convolutional layer with a kernel size of 1 is used to reduce the number of channels to 1, followed by a batch normalization layer and a ReLU activation function. Finally, the two types of features are fused using element-wise addition to obtain the fused enhanced features. Expressed by the following formula:

[0085] ;

[0086] Enhanced features after fusion The input is fed into the fully connected layer, and the output logic value is then processed... softmax Function mapping to probability This is used to determine the type of target corresponding to the trajectory segment, such as whether it belongs to a drone or a bird.

[0087] ;

[0088] In the formula This indicates a fully connected layer.

[0089] The loss function used during model training can be the cross-entropy loss function, as follows:

[0090] ;

[0091] In the formula This indicates the actual target category label.

[0092] In actual testing, the probability value output by the method determines whether to issue an alarm for the point: if the probability of the drone target is greater than the probability of the false target, it is judged as a real target; otherwise, the point is considered a false target.

[0093] To demonstrate the effectiveness of the proposed method for distinguishing moving target trajectories based on heterogeneous feature fusion, a specific experiment is provided below:

[0094] The method was validated using measured wide-area infrared search data, comprising 33 flights. Of these, 11 flights lacked targets, and 22 flights contained targets. From the output trajectories of these 33 flights, every three consecutive points formed a training sample, consisting of a motion feature (.csv) file and a trajectory map (.jpg), totaling 882 samples. A five-fold cross-validation method was employed, with 20% of the samples used for testing and the remaining 80% used for training in each iteration. Five experiments were conducted, and the performance of different methods was compared based on the mean.

[0095] The network used for extracting spatial morphological features was a ResNet18 pre-trained on ImageNet. Motion features and trajectory maps were standardized before being input into the network. The training batch size was 8, the learning rate was 0.01, the optimizer used the SAM method, and the network was trained for 20 epochs. The network parameters of the epoch with the best performance were selected for testing.

[0096] To evaluate the performance of the method, accuracy is used as the metric:

[0097] ;

[0098] In the formula The number of correctly identified trajectories. This represents the total number of trajectories.

[0099] To verify the effectiveness of the proposed heterogeneous feature fusion-based moving target trajectory differentiation method (CMoSA), comparative experiments were conducted, including single-modal methods (1DConv and ResNet) and multi-modal fusion methods (MulT, GAFN, CrossModalFusion, CMDNet, UniModal, and CIGAM). The results are shown in Table 1.

[0100] Two baseline experiments were conducted under a single-modal setting. In the one-dimensional convolution experiment based on motion features, the classification accuracy of five-fold cross-validation fluctuated between 60.23% and 69.89%, with an average of 64.77%. This result indicates that relying solely on motion features is insufficient for effective trajectory classification, possibly because the motion patterns exhibit high similarity, resulting in limited discriminative information provided by the motion parameters themselves. In contrast, the fine-tuned ResNet model based on morphological features significantly outperformed the motion feature-based method, achieving an average accuracy of 95.45% in the five-fold experiment with minimal fluctuation, indicating that the morphological features of trajectory segments have stronger discriminative power.

[0101] Table 1. Comparison of accuracy rates of different trajectory differentiation methods

[0102]

[0103] All fusion methods achieved accuracies exceeding 80%, a significant improvement over using motion features alone. However, not all fusion strategies delivered further performance gains compared to the 95.45% accuracy achieved using morphological features alone. The early fusion method MulT employed a Transformer architecture for feature optimization after cross-modal channel concatenation, but its accuracy was only 82.84%, likely due to the channel concatenation operation. In this task, the feature dimensions of different modalities differed significantly, and simple concatenation easily introduced modal conflicts. A similar performance degradation was observed in CMDNet's channel concatenation method, with an accuracy of 95.23%, lower than the 95.45% achieved using morphological features alone.

[0104] To investigate the impact of the fusion direction, two sets of experiments were conducted based on GAFN. When motion features were used to enhance morphological features, the trajectory classification accuracy reached 95.57%, while the result of the reverse setting was only 95.23%. This asymmetry may stem from the difference in discriminative ability between the two types of features in the single-modal experiment, i.e., morphological features are significantly better than motion features. UniModal and CrossModalFusion based on the gating mechanism achieved accuracies of 94.43% and 95.11%, respectively, both lower than the proposed heterogeneous feature fusion-based moving target trajectory differentiation method (CMoSA) in this invention.

[0105] When CMDNet employs modality alignment and fuses heterogeneous features through mean summation, its accuracy improves to 95.57%, surpassing the two baseline methods. This result highlights the importance of dimensional alignment in heterogeneous feature fusion, an operation that helps alleviate the problem of imbalanced feature representations.

[0106] In contrast, CIGAM, originally designed to fuse hyperspectral and lidar data from remote sensing, achieved an accuracy of only 94.66% in this task. Fundamentally, both modalities reflect the morphological information of the scene, and their semantic differences are smaller than the differences between motion and morphological features in this task. Therefore, the performance improvement in this research scenario is not significant.

[0107] The proposed heterogeneous feature fusion-based moving target trajectory differentiation method (CMoSA) introduces a bidirectional self-attention module before element-by-element addition, enhancing cross-modal interaction between motion and morphological features, thereby optimizing the fusion process. The average accuracy reaches 96.02%, verifying that the attention-based adaptive alignment strategy can effectively alleviate semantic conflicts between modalities and improve the compatibility of heterogeneous feature fusion.

[0108] To further analyze the role of heterogeneous feature interaction enhancement in the proposed heterogeneous feature fusion-based moving target trajectory differentiation method (CMoSA), an ablation experiment was conducted, and the results are shown in Table 2.

[0109] Table 2. Comparison of accuracy before and after enhancement for different structural features.

[0110]

[0111] After enhancing the motion morphology features, while maintaining the same feature dimensions, the average accuracy increased from 64.77% to 95.45%, significantly improving trajectory discrimination performance. This indicates that the cross-modal interaction mechanism effectively strengthens the category judgment information in motion features. After enhancing the morphology features, the average accuracy increased by 0.12 percentage points, from 95.45% to 95.57%, further validating the complementarity between motion and morphology features. After fusing the enhanced heterogeneous features, the average accuracy across five experiments further improved to 96.02%, outperforming single-feature enhancement and achieving the highest accuracy in four experiments. These experimental results demonstrate that the proposed Heterogeneous Feature Fusion-Based Moving Target Trajectory Discrimination Method (CMoSA) can overcome the performance bottleneck of single-modality methods through the collaborative optimization of heterogeneous features, further enhancing method performance and improving trajectory discrimination effectiveness.

[0112] This invention constructs a heterogeneous feature cross-modal fusion framework for infrared moving targets, fully leveraging the complementarity between target motion parameters and trajectory spatial morphology. This achieves highly reliable differentiation between real and false target trajectories under conditions of low frame rate, strong background interference, and high false alarm rate. Compared to existing methods relying solely on single-modal features, this invention utilizes temporal motion features to characterize the target's dynamics, represents the motion morphology with a three-point trajectory structure, and establishes semantic alignment between the two types of features through an adaptive cross-modal attention mechanism, effectively improving the discriminativeness and robustness of feature representation. Experimental results show that this invention significantly enhances trajectory-level recognition capabilities: while maintaining the feature dimension, cross-modal enhancement greatly improves the classification performance of motion features, and the fused heterogeneous features further break through the performance limit of single-modal features, achieving a trajectory differentiation accuracy of 96.02%, superior to many existing single-modal and multi-modal fusion methods. This technology effectively reduces the false alarm rate in UAV monitoring scenarios, improves the reliability of threat confirmation, and provides stable, accurate, and practically valuable technical support for the identification of small UAVs in wide-area infrared search systems.

[0113] On the other hand, a moving target trajectory differentiation device based on heterogeneous feature fusion is provided, comprising:

[0114] The first module is used to acquire the continuous motion trajectory of different types of moving targets;

[0115] The second module is used to extract trajectory features, including temporal motion features and spatial morphological features, from each continuous motion trajectory using a feature extraction network to distinguish target types.

[0116] The third module is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism using a cross-modal fusion network to obtain the fused enhanced features;

[0117] The fourth module is used to construct training samples based on the enhanced features and the corresponding moving target type labels, and to jointly optimize the parameters of the feature extraction network and the cross-modal fusion network using a loss function to obtain a trained moving target trajectory discrimination model.

[0118] The fifth module is used to classify the type of moving target by using a trained moving target trajectory discrimination model for the continuous motion trajectory of the moving target to be judged, and output the classification result.

[0119] Figure 5 The figure shows a block diagram of an electronic device according to an embodiment, such as Figure 5 As shown, the electronic device includes one or more processors and a memory. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the moving target trajectory differentiation method based on heterogeneous feature fusion provided in any of the above embodiments. The processor may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device to perform desired functions. The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc.

[0120] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0121] Of course, for the sake of simplicity, Figure 5 Only some of the components of the electronic device relevant to this application are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0122] Embodiments of the present invention may also be computer-readable storage media storing a computer program thereon. When executed by a processor, the computer program implements the steps of the moving target trajectory differentiation method based on heterogeneous feature fusion provided in any of the above embodiments. The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0123] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not restrict the application from being implemented using the specific details described above.

[0124] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0125] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0126] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application should not be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0127] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for distinguishing moving target trajectories based on heterogeneous feature fusion, characterized in that, Includes the following steps: To acquire the continuous motion trajectory of different types of moving targets; From each continuous motion trajectory, a feature extraction network is used to extract trajectory features for distinguishing target types. This includes: grouping every three consecutive points in the continuous motion trajectory into a group, and extracting the temporal motion features and spatial morphological features corresponding to each group of points. The extraction of spatial morphological features corresponding to each group of points includes: (2.1) For each set of points, connect and close the three consecutive points on the azimuth-elevation plane in sequence to form the triangular trajectory shape corresponding to each set of points, and obtain the azimuth-elevation plane map with the triangular trajectory shape marked on each set of points. (2.2) Input the azimuth-elevation plane map corresponding to each set of points and labeled with the shape of the triangle trajectory into the ResNet network and extract the output features of the global pooling layer; (2.3) Adjust the channels of the output features of the global pooling layer to obtain the spatial morphological features corresponding to each group of points. The number of channels of the spatial morphological features corresponding to each group of points is the same as the number of channels of the temporal motion features corresponding to each group of points. A cross-modal fusion network is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism to obtain enhanced features after fusion; Based on the enhanced features and the corresponding moving target type labels, training samples are constructed, and the parameters of the feature extraction network and the cross-modal fusion network are jointly optimized using the loss function to obtain a trained moving target trajectory discrimination model. For the continuous motion trajectory of the moving target to be identified, the trained moving target trajectory discrimination model is used to classify the moving target type and output the classification result.

2. The method for distinguishing moving target trajectories based on heterogeneous feature fusion according to claim 1, characterized in that, The continuous motion trajectory of the moving target is a sequence of dots output by the infrared search and track system, and the moving target includes birds and drones.

3. The method for distinguishing moving target trajectories based on heterogeneous feature fusion according to claim 1 or 2, characterized in that, The temporal motion features are constructed based on at least one of the following continuous target trace measurement information: azimuth angle, pitch angle, intensity value, pixel area, local signal-to-noise ratio, angular velocity, angular acceleration, and correlation distance.

4. The method for distinguishing moving target trajectories based on heterogeneous feature fusion according to claim 3, characterized in that, Extract the temporal motion features corresponding to each set of points, including: (1.1) Calculate the 11-dimensional trajectory motion feature vector for each point in each group of points, expressed as: in and These are the azimuth and elevation angles corresponding to the target, respectively. For local signal-to-noise ratio, The average intensity This represents the maximum intensity. For pixel area, The distance between points in the trajectory. , , and These are the azimuth angular velocity, pitch angular velocity, azimuth acceleration, and pitch acceleration of the dot, respectively. (1.2) The 11-dimensional trajectory motion feature vectors corresponding to the three consecutive points in each group of points form an input feature matrix of shape 3×11, where the row dimension 3 corresponds to the number of consecutive points and the column dimension 11 corresponds to the feature degree of each point. (1.3) Input the input feature matrix of each set of points into a one-dimensional convolutional neural network for feature extraction to obtain the temporal motion features corresponding to each set of points.

5. The method for distinguishing moving target trajectories based on heterogeneous feature fusion according to claim 4, characterized in that, The temporal motion features and spatial morphological features corresponding to each set of points are fused through a cross-modal self-attention mechanism to obtain the fused enhanced features corresponding to each set of points, including: (3.1) Obtain the enhanced temporal motion features corresponding to each set of points through the following steps: (3.1.1) For each set of points, its temporal motion features are mapped to a query vector through learnable parameters, and its spatial morphological features are mapped to a key vector and a value vector through learnable parameters, respectively. (3.1.2) Calculate the similarity between the query vector and the transpose of the key vector, and then normalize the result to obtain the similarity matrix; (3.1.3) Multiplying the similarity matrix with the value vector yields the temporal motion features enhanced by spatial morphological features; (3.1.4) Input the temporal motion features enhanced by spatial morphology into the feedforward network of the residual structure, and output the final enhanced temporal motion features; (3.2) Obtain the enhanced spatial morphological features corresponding to each set of dots through the following steps; (3.2.1) For each set of points, its spatial morphological features are mapped to a query vector through learnable parameters, and its temporal motion features are mapped to a key vector and a value vector through learnable parameters, respectively. (3.2.2) Calculate the similarity between the query vector and the transpose of the key vector, and then normalize the result to obtain the similarity matrix; (3.2.3) Multiplying the similarity matrix with the value vector yields the spatial morphological features enhanced by temporal motion characteristics; (3.2.4) Input the spatial morphological features enhanced by temporal motion features into the feedforward network of the residual structure, and output the final enhanced spatial morphological features; (3.3) The enhanced temporal motion features and spatial morphological features corresponding to each set of points are fused to obtain the fused enhanced features corresponding to each set of points.

6. A moving target trajectory differentiation device based on heterogeneous feature fusion, used to implement the moving target trajectory differentiation method based on heterogeneous feature fusion as described in claim 1, characterized in that, include: The first module is used to acquire the continuous motion trajectory of different types of moving targets; The second module is used to extract trajectory features, including temporal motion features and spatial morphological features, from each continuous motion trajectory using a feature extraction network to distinguish target types. The third module is used to fuse temporal motion features and spatial morphological features through a cross-modal self-attention mechanism using a cross-modal fusion network to obtain the fused enhanced features; The fourth module is used to construct training samples based on the enhanced features and the corresponding moving target type labels, and to jointly optimize the parameters of the feature extraction network and the cross-modal fusion network using a loss function to obtain a trained moving target trajectory discrimination model. The fifth module is used to classify the type of moving target by using a trained moving target trajectory discrimination model for the continuous motion trajectory of the moving target to be judged, and output the classification result.

7. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, they implement the steps of the moving target trajectory differentiation method based on heterogeneous feature fusion as described in claim 1.

8. A computer storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by the processor, they implement the steps of the moving target trajectory differentiation method based on heterogeneous feature fusion as described in claim 1.

Images