A method for basic aircraft motion recognition based on selective feature fusion model

Abstract

The present application discloses a method for recognizing basic aircraft actions based on a selective feature fusion model, which includes inputting original time series data into an original time series feature construction module, obtaining features related to aircraft actions and amplifying them into the original time series data to obtain new time series data, and obtaining input features of the selective feature fusion model and corresponding aircraft basic action category labels based on the new time series data; constructing a selective feature fusion model; training the selective feature fusion model through a total loss function to obtain a trained selective feature fusion model and its parameters; simply inputting the original data into the original time series feature construction module to convert it into the input feature format required by the selective feature fusion model, and loading the trained selective feature fusion model parameters to perform end-to-end inference to obtain the recognition results of basic aircraft actions. The present application improves the performance of aircraft basic action recognition.

Description

Technical Field

[0001] The present application relates to the technical field of situational awareness, and in particular to a method for recognizing basic aircraft actions based on a selective feature fusion model. Background Art

[0002] Aircraft basic movements are the smallest units that constitute the aircraft's complex maneuvers and are one of the foundations for further high-level situational awareness such as aircraft target intention recognition.

[0003] In terms of related research background, most current scholars have skipped over basic aircraft motion recognition and directly addressed complex maneuvers (such as loops, turns, and rolls), or have mixed basic and complex maneuvers for simulation research. Their research data primarily relies on continuous trajectory segments output by simulation platforms, but most fail to consider the high dynamics and adversarial nature of non-cooperative aircraft targets, nor the time constraints of onboard sensors on friendly airborne platforms. For example, in scenarios where friendly radars actively detect and track other aircraft targets, on the one hand, when a target detects being tracked, it will actively maneuver to escape, hindering the ability to capture its continuous motion characteristics. On the other hand, friendly active radars also operate intermittently, reducing the probability of detection and are typically not powered on for extended periods. This results in discontinuous and short detected aircraft target trajectories. This means that it is theoretically difficult to identify whether a target is performing a long maneuver (such as a turn, which typically takes a long time). Furthermore, trajectory data obtained through active detection cannot infer whether the target is performing maneuvers such as rolls, and the accuracy limitations of detection and tracking also make it difficult to identify maneuvers such as loops performed by targets in a small space. Therefore, this type of research method is mainly suitable for applications such as flight quality assessment of one's own aircraft and aircraft fault detection, and is not suitable for the field of situational awareness.

[0004] In terms of related technologies, traditional methods for identifying basic aerial aircraft motions primarily include methods based on expert knowledge base reasoning, temporal similarity methods such as dynamic time warping, and traditional machine learning methods such as support vector machines or Bayesian networks. Methods based on expert knowledge base reasoning first require the establishment of a motion feature library for various aircraft targets using expert knowledge. This requires high accuracy in describing aircraft motion features. However, the features of detected and tracked aircraft targets, which typically require online recognition, often exhibit high ambiguity, reducing recognition accuracy. Balancing feature accuracy and ambiguity relies entirely on the subjective judgment of experts, requiring frequent knowledge base updates and high maintenance costs. Temporal similarity methods, while not requiring an expert knowledge base, have difficulty identifying similar flight maneuvers. Furthermore, due to their sensitivity to noise and highly dynamic trajectory mutations, threshold parameters are difficult to determine, resulting in low recognition accuracy. Methods based on Bayesian theory rely on expert experience in network structure design and parameter assignment, such as prior probabilities, resulting in a high design threshold. Traditional machine learning methods such as support vector machines generally only learn shallow features, leaving room for improvement in classification and recognition accuracy. Overall, all of these methods suffer from various challenges, including low recognition accuracy and insufficient generalization.

[0005] Precisely because of the limitations of traditional algorithms, scholars have been exploring new technical approaches. With the gradual penetration of artificial intelligence technologies such as deep learning into various technical fields, deep learning has attracted significant attention from researchers in the field due to its powerful feature extraction and pattern recognition capabilities, its lack of complex domain theory formula derivation and expert experience, and its data-driven end-to-end recognition model. Consequently, a growing number of deep learning methods have been applied to the recognition of basic aircraft target motions. Scholars have transformed basic aircraft motion recognition into a classification problem in pattern recognition. By building neural network classification models for training and learning, they map aircraft detection and tracking data to basic motion classification labels, thereby achieving automatic recognition of basic aircraft target motions. Related research primarily designs basic aircraft target motion recognition methods based on architectures such as convolutional neural networks and temporal neural networks. Convolutional neural network-based methods typically construct selective feature fusion models by stacking convolutional layers, pooling layers, or fully connected layers, such as fully convolutional neural networks and temporal convolutional neural networks. These models are used to capture complex local patterns and structures within action unit data. However, for aircraft action recognition time series data, the receptive field of the convolutional layer is typically fixed and cannot be dynamically adjusted. Furthermore, they focus too much on local feature patterns and are less capable of capturing long-term dependency feature patterns. While temporal neural network-based methods (such as recurrent neural networks, long short-term memory networks, gated recurrent units, and their improved algorithms) have strong capabilities for capturing long-term dependency patterns and are particularly suitable for deep feature mining of data with clear temporal relationships, these methods are insufficient in filtering redundant information from input data and may experience training instability. Although some researchers have incorporated attention mechanisms into these network architectures to capture key features and increase the robustness of the models in certain scenarios, they have not fundamentally overcome the limitations of these network architectures. In addition, when designing classification loss functions for aircraft target action recognition, most scholars directly use the cross-entropy loss function, which only considers the degree of difference in the probability distribution of the classification problem, and does not consider the feature compactness between the features of samples with the same label and the separability between the features of samples with different labels. This leads to a strong dependence of the training model on the training samples, which is not conducive to improving the model's generalization ability. Therefore, further realizing the selective extraction of important information from global features and local feature patterns in time series data, filtering redundant feature information, and taking into account the similarities and differences of features of samples with different labels in the classification task will have a positive effect on improving the performance of the selective feature fusion model for aircraft target basic action recognition based on deep learning. Summary of the Invention

[0006] In response to the limitations of existing deep learning-based basic aircraft motion recognition methods, this application provides a basic aircraft motion recognition method based on a selective feature fusion model, which is suitable for basic motion recognition tasks such as active detection and tracking of airborne platform radars and multi-source detection and fusion tracking of aircraft targets. The conversion equation between hidden states and input features is constructed by approximating the second-order discretization of the state space model of the random system, and a long-distance dependent global feature pattern extractor of the temporal features of the basic aircraft motions is designed to enhance the ability of the selective feature fusion model to selectively focus on important information and ignore irrelevant information on the input features. On this basis, a local feature pattern extractor is designed through multi-scale causal convolution and small-scale feature fusion to increase the ability to capture local features in different time windows, and perform global and local feature fusion enhancement to improve recognition performance. At the same time, a new classification loss function is designed by combining the cluster center distance loss of sample features with the same label and the exponential distance loss between sample features with different labels, thereby improving the fitting degree of the selective feature fusion model and improving classification and recognition performance.

[0007] This application discloses a method for recognizing basic aircraft actions based on a selective feature fusion model, which includes:

[0008] Step 1: Input the original time series data into the original time series feature construction module, obtain the features related to the aircraft movement and amplify them into the original time series data to obtain new time series data. Based on the new time series data, obtain the input features of the selective feature fusion model. and its corresponding aircraft basic action category label Y; features related to aircraft actions include aircraft velocity scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle; aircraft basic action category label Y is used to indicate the aircraft basic action category; aircraft basic action categories include straight, turning, climbing, and diving;

[0009] Step 2: Construct a selective feature fusion model, which consists of a temporal feature nonlinear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the temporal feature nonlinear enhancement module is connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier;

[0010] Step 3: Train the selective feature fusion model by constructing a total loss function to obtain the trained selective feature fusion model and its parameters;

[0011] Step 4: Simply input the original data into the original time series feature construction module to convert it into the input feature format required by the selective feature fusion model, load the parameters of the trained selective feature fusion model, and perform end-to-end inference to obtain the recognition results of the basic aircraft movements; the basic aircraft movement recognition results are one of the basic aircraft movement categories.

[0012] Furthermore, the step 1 includes:

[0013] Assumptions The original time series data at the moment is ,in, Represents longitude, Represents latitude, Represents height, represents the eastward speed, represents the northbound speed, Represents the celestial speed, represents the eastward acceleration, represents the northward acceleration, represents the celestial acceleration;

[0014] The original time series data Input into the original time series data construction module to get the aircraft Velocity scalar at time for:

[0015]

[0016] Get the plane Acceleration scalar at time for:

[0017]

[0018] Get the plane Track angle at time for:

[0019]

[0020] The range of track deviation angle is , the north direction is 0, and the north-east direction is positive;

[0021] airplane Aircraft track pitch angle at time for:

[0022]

[0023] The range of track pitch angle is , the aircraft nose is pointing up is positive, and the nose is pointing down is negative;

[0024] Add the aircraft's velocity scalar, acceleration scalar, aircraft track angle, and aircraft track pitch angle to the original time series data , get new time series data :

[0025]

[0026] The new time series data After normalization, we get:

[0027]

[0028] in, After normalization Characteristic data at the moment; and are functions for finding minimum and maximum values, respectively. is the time step;

[0029] Will The feature vector of the time step Accumulate to form the input features of the selective feature fusion model :

[0030]

[0031] The time step The basic action category labels of the aircraft are quantified to obtain the basic action category labels of the aircraft. :

[0032]

[0033] Among them, 0 means The corresponding aircraft basic action category is "straight"; 1 means The corresponding basic action category of the aircraft is "turn"; 2 means The corresponding aircraft basic action category is "climb"; 3 means The corresponding basic aircraft action category is "dive".

[0034] Furthermore, in step 2, the global feature extraction module includes two branches, a linear projection layer, and a global average pooling layer, and the outputs of the two branches are connected to the global average pooling layer through the linear projection layer;

[0035] One of the two branches will input the feature And its corresponding aircraft basic action category label Y is sequentially processed by linear projection, layer normalization, and second-order discrete state space model. The other branch processes the input feature The corresponding aircraft basic action category label Y is linearly projected and SiLU activated in turn, and then the features output by the two branches are multiplied, and the multiplication result is linearly projected to restore it to the input feature The dimension of , and finally obtain the final global features through global average pooling; input features is the output of the temporal feature nonlinear enhancement module.

[0036] Furthermore, the second-order discrete state space model is processed as the feature output after the layer normalization processing by the second-order discrete state space model Process and get output :

[0037] The process of constructing the second-order discrete state space model is:

[0038] The state space model of the random system is introduced. The state space model includes the state equation and the observation equation:

[0039] (1)

[0040] (2)

[0041] in, is the state vector at time t, for The differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transfer matrix, which is used to control the change of state over time; is the input matrix, which is used to control the impact of the input on the state; is the output matrix, which represents the output generated based on the current state; is the feedforward matrix, which represents the direct impact of input on output; Used to indicate that the random system is a continuous-time system; Formula (1) is the state equation, and Formula (2) is the observation equation;

[0042] Formulas (1) and (2) are approximated by discretization and converted into:

[0043] (3)

[0044] (4)

[0045] in, is the kth time step hidden state of the second-order discrete state space model, is the k-1th time step hidden state of the second-order discrete state space model, The deterministic input at the kth time step, and Represents a discrete interval index, i.e., a time step index; is the output of the kth time step of the second-order discrete state space model; 、 In order to form a new state transfer matrix and input matrix by discretization, the matrix is ​​obtained by solving the differential equations of formulas (3) and (4) through Taylor series discretization expansion. 、 ;

[0046] Expand the state equation of formula (3) through Taylor series, retain the quadratic term approximation, and the new state transfer matrix and the new input matrix The form is:

[0047] (5)

[0048] (6)

[0049] Among them, the parameters Parameters introduced for discretization; is a unit array; and are the state transfer matrix and input matrix, respectively, both of which are parameters that require network initialization; is a high-order term;

[0050] The following formula and Modeling as Features Function:

[0051] (7)

[0052] (8)

[0053] in, is a softplus function, to ensure Provide nonlinear control capabilities; , , , is the linearization parameter;

[0054] In order to make the output of the second-order discrete state space model follow the characteristics Control state flow, the output matrix in the observation equation of formula (4) Modeling as Features Function:

[0055] (9)

[0056] in, and are all linearization parameters;

[0057] Substitute formula (5) to formula (8) into formula (3), and change the characteristic As a deterministic input, substitute formula (9) into formula (4) and introduce the noise vector of the state equation and the noise vector of the observation equation , the second-order discrete state space model is obtained as:

[0058] (10)

[0059] (11)

[0060] Among them, the newly added parameters and Error in discretizing the equilibrium state-space model.

[0061] Furthermore, the multi-scale local feature extraction module includes three feature extraction branches, a feature splicing layer, and a convolution layer; each feature extraction branch in the three-way feature extraction branch includes a linear projection layer, a causal convolution layer, a batch normalization layer, a ReLU activation layer, and a global average pooling layer connected in sequence; the global average pooling layer of each feature extraction branch in the three-way feature extraction branch is connected to the convolution layer through the feature splicing layer; the convolution kernel size used in the causal convolution layer of each feature extraction branch in the three-way feature extraction branch is different;

[0062] Each feature extraction branch uses a linear projection layer to transform the input features Nonlinear feature enhancement is performed, and then each feature extraction branch uses convolution kernels of different sizes to extract local feature patterns within different receptive fields, and then batch normalization is performed. The batch-normalized features are residually connected with the output of the linear projection layer and then ReLU activation is performed. Global average pooling is used to average the features of different channels, compress the feature information, and only retain the most critical local feature patterns. Finally, the outputs of all global average pooling in the three feature extraction branches are feature spliced ​​and multi-scale feature fusion is performed. The convolution layer is used to adjust the feature dimension after multi-scale feature fusion to the dimension output by the global feature extraction module.

[0063] Furthermore, the global and local feature fusion module obtains selective fusion features by splicing the global features output by the global feature extraction module and the local features output by the multi-scale local feature extraction module, and fusing them through two linear projection scalings.

[0064] Furthermore, the feature classifier maps the selective fusion features output by the global and local feature fusion module to the probability representation of the aircraft's basic action categories, and the index of the maximum probability represents the aircraft's basic action category.

[0065] Furthermore, the step 3 includes:

[0066] During the training phase, the original data is used as training samples, and the total loss function is used to train the optional feature fusion model, and the balance coefficient of the total loss function is set. The original data includes time series data generated by simulation on various simulation environment platforms, original detection time series data of aircraft targets obtained by real sensors, and multi-source fusion time series data of aircraft targets obtained by multi-source fusion.

[0067] Set the batch size, number of learning rounds, and initial learning rate. Use the segmented learning rate update strategy to update the learning rate corresponding to a certain number of rounds during training, and set the segmented interval for learning rate update. Use the Adam optimizer to optimize the selective feature fusion model, and use early stopping judgment. That is, when the validation set loss does not decrease for multiple consecutive rounds, stop training and obtain the parameters of the trained selective feature fusion model.

[0068] Furthermore, the total loss function constructed is:

[0069] (12)

[0070] in, is the total loss function; Cross entropy loss for classification of basic aircraft action categories; The cluster center distance loss for training samples whose labels are predicted to be of the same category; An exponential distance loss between selectively fused features for predicting training samples with labels of different classes; and As a compensation term for the categorical cross entropy loss function; 、 、 for 、 and The balance coefficient between them; the optional fusion feature is the feature output by the global and local feature fusion module.

[0071] Furthermore, 、 and The expressions are:

[0072] (13)

[0073] (14)

[0074] (15)

[0075] in, is the number of aircraft target time series data samples, and the aircraft target time series data samples are original data; The number of basic aircraft action categories; For the training samples are basic action categories The true label of For the training samples are predicted as basic action categories The predicted probability of For the Selective fusion features of training samples; For the Selective fusion features of training samples; Indicates that the label prediction is the basic action category The number of training samples, basic action categories The label Y belongs to the basic action category of the aircraft, and its value is 0, 1, 2 or 3; Indicates that the label prediction is the basic action category In the training sample Selective fusion features of training samples; For all predicted categories The clustering center of the selective fusion features of the training samples; Represents the exponential function.

[0076] Due to the adoption of the above technical solution, this application has the following advantages:

[0077] 1. By constructing a new feature fusion model for global and local features of target (aircraft) time series data, the phenomenon of insufficient expression of single-type features is alleviated, the feature expression capability is improved, and the subsequent classification performance is promoted;

[0078] 2. By performing second-order discrete optimization and noise vector balance on the state space model, the approximation accuracy of the second-order discrete state space model is improved. The coefficient matrix is ​​modeled as a function of the input feature vector, which promotes the selective feature fusion model to adaptively extract key information from the input features and filter out redundant information, thus realizing the automatic selection and extraction of global feature patterns.

[0079] 3. By designing a multi-scale causal convolution and residual connection network module (multi-scale local feature extraction module), the system avoids future information leakage caused by traditional convolutional networks, better conforms to the temporal continuity characteristics of basic aircraft motion data, and further improves the local feature expression capability through small-scale multi-scale feature fusion.

[0080] 4. By adding cluster center distance loss and exponential distance loss to the classification feature samples on top of the classification cross entropy loss, the feature distances between samples with similar labels can be shortened and the feature distances between samples with different labels can be widened. This improves the model's fit and matching of the fused features, further enhancing the model's adaptability while still taking into account the task of recognizing and classifying the basic aircraft action categories.

[0081] 5. Overall, by optimizing the global feature extraction model, we achieve higher-precision selective feature extraction. Furthermore, by taking into account the characteristics of time series data, we enhance local features through multi-scale local feature extraction and fusion, mining more local feature patterns. Furthermore, by designing a fusion-enhanced network that combines global and local features, we filter out redundant information, retain key features that are beneficial for subsequent classification of basic aircraft maneuvers, and improve classification performance. Furthermore, the classification loss function takes into account the differences between the features of samples with different labels, and specifically designs the loss function. This aims to not only consider the impact of label predictions during training, but also the enhanced expressiveness of fused features, thereby further enhancing recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0083] Figure 1 This is a block diagram corresponding to a method for recognizing basic aircraft actions based on a selective feature fusion model according to an embodiment of the present application;

[0084] Figure 2 This is a schematic diagram of the application flow of another method for basic aircraft motion recognition based on a selective feature fusion model according to an embodiment of the present application. DETAILED DESCRIPTION

[0085] The present application is further described with reference to the accompanying drawings and embodiments. The embodiments described are only a part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field should fall within the scope of protection of the embodiments of the present application.

[0086] See also Figure 1 The present application provides an embodiment of a method for identifying basic aircraft actions based on a selective feature fusion model, which constructs a conversion equation between hidden states and input features using a second-order discretization approximation of a random system state space model. Specifically, the time series features of the target (aircraft) are substituted into the state space model as deterministic inputs, and the state equation is discretized by performing a Taylor series expansion, discarding high-order terms and retaining second-order terms, thereby making the state equation coefficient matrix more accurate. At the same time, the coefficient matrix of the state equation is modeled as a function of the input features, so that during network training, the model parameters automatically focus on the key information therein and filter out redundant components according to the input features. At the same time, the discretization error is balanced by adding a noise vector, thereby maximizing the approximate accuracy of the state space model and improving the accuracy of the state space model. The extraction performance of long-distance dependent global feature patterns; on this basis, relying on the advantages of convolutional networks in local feature extraction, considering the progressive relationship of time series feature data in the time dimension, through multi-scale causal convolution and feature splicing fusion, overcome the leakage problem of traditional convolution on future information, improve the extraction performance of local feature patterns, and finally further enhance the feature expression ability of the entire selective feature fusion model through the fusion of global and local feature patterns; in addition, this application considers combining the cluster center distance loss of the same label sample features and the exponential distance loss between different label sample features in the classification loss function to improve the fitting degree of the selective feature fusion model, reduce the underfitting risk of the selective feature fusion model during convergence, and improve the automatic recognition performance of the selective feature fusion model.

[0087] The technical solutions of this embodiment include:

[0088] Step 1: Input the original time series data into the original time series feature construction module, obtain the features related to the aircraft movement and amplify them into the original time series data to obtain new time series data. Based on the new time series data, obtain the input features of the selective feature fusion model. and its corresponding aircraft basic action category label Y; features related to aircraft actions include aircraft velocity scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle; aircraft basic action category label Y is used to indicate the aircraft basic action category; aircraft basic action categories include straight, turning, climbing, and diving;

[0089] Step 2: Construct a selective feature fusion model, which consists of a temporal feature nonlinear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the temporal feature nonlinear enhancement module is connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier;

[0090] Step 3: Train the selective feature fusion model by constructing a total loss function to obtain the trained selective feature fusion model and its parameters;

[0091] Step 4: Simply input the original data into the original time series feature construction module to convert it into the input feature format required by the selective feature fusion model, load the parameters of the trained selective feature fusion model, and perform end-to-end inference to obtain the recognition results of the basic aircraft movements; the basic aircraft movement recognition results are one of the basic aircraft movement categories.

[0092] For ease of explanation, we first introduce the concept of the state-space model of a stochastic system and its discretization. The state-space model of a stochastic system is derived from the Kalman filter and is a method for describing the dynamic behavior of a system over time. It is widely used in control theory. It usually includes the following two equations:

[0093] (1)

[0094] (2)

[0095] in, is the state vector at time t, for The differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transfer matrix, which is used to control the change of state over time; is the input matrix, which is used to control the impact of the input on the state; is the output matrix, which represents the output generated based on the current state; is the feedforward matrix, which represents the direct impact of input on output; Used to indicate that the random system is a continuous-time system; Formula (1) is the state equation, and Formula (2) is the observation equation;

[0096] Since the above state space model is a continuous time system, there is a state vector The differential form of , it needs to be approximated by discretization and converted into the following form:

[0097] (3)

[0098] (4)

[0099] Among them, the subscript and Represents a discrete interval index, i.e., a time step index; is the kth time step hidden state of the second-order discrete state space model, is the k-1th time step hidden state of the second-order discrete state space model, The deterministic input at the kth time step, and Represents a discrete interval index, i.e., a time step index; is the output of the kth time step of the second-order discrete state space model; 、 The new state transfer matrix and input matrix formed by discretization are obtained mainly by solving the differential equations of formulas (3) and (4) through Taylor series discretization expansion. 、 ;

[0100] refer to Figure 1 The global feature extraction module in the CNN consists of two branches, a linear projection layer, and a global average pooling layer. The outputs of the two branches are connected to the global average pooling layer through the linear projection layer. One branch of the two branches takes the input feature And its corresponding aircraft basic action category label Y is sequentially processed by linear projection, layer normalization, and second-order discrete state space model. The other branch processes the input feature The corresponding aircraft basic action category label Y is linearly projected and SiLU activated in turn, and then the features output by the two branches are multiplied, and the multiplication result is linearly projected to restore it to the input feature The dimension of , and finally obtain the final global features through global average pooling; input features is the output of the temporal feature nonlinear enhancement module.

[0101] Figure 1 The second-order discrete state space model in the process is processed as the feature output after the layer normalization processing by the second-order discrete state space model Process and get output :

[0102] Expand the state equation of formula (3) through Taylor series, retain the quadratic term approximation, and the new state transfer matrix and the new input matrix The form is:

[0103] (5)

[0104] (6)

[0105] Among them, the parameters Parameters introduced for discretization; is a unit array; and are the state transfer matrix and input matrix, respectively, both of which are parameters that require network initialization; is a higher-order term.

[0106] In order to enable the second-order discrete state space model to filter out irrelevant information and compress the context of the second-order discrete state space model into a valid state, so that the second-order discrete state space model can more finely control the propagation of the second-order discrete state space model input and state, , Modeled as input features Function:

[0107] (7)

[0108] (8)

[0109] in, is a softplus function, to ensure Provide nonlinear control capabilities; , , , is the linearization parameter.

[0110] Then, in order to make the output of the second-order discrete state space model follow the characteristics Control state flow, the output matrix in the observation equation of formula (4) Also modeled as input features Function:

[0111] (9)

[0112] in, , is the linearization parameter.

[0113] Substitute formula (5), formula (6), formula (7), and formula (8) into formula (3) and change the characteristic As a deterministic input, substitute formula (9) into formula (4) and introduce the noise vector of the state equation and the noise vector of the observation equation , the constructed second-order discrete state space model is obtained as:

[0114] (10)

[0115] (11)

[0116] in, is the hidden state of the second-order discrete state space model; is the output of the second-order discrete state space model. New parameters , The error used to discretize the equilibrium state space model can be regarded as a regularization term and is generally initialized to a small parameter.

[0117] The local feature extraction module structure based on multi-scale causal convolution and residual network connection constructed in this application is shown in Figure 1 The multi-scale local feature extraction module in the CNN consists of three feature extraction branches, a feature splicing layer, and a convolution layer. Each of the three feature extraction branches uses a linear projection layer for nonlinear feature enhancement, and then the three branches use , , The causal convolution of different convolution kernel sizes extracts local feature patterns within different receptive fields, and then performs batch normalization. It is then connected to the output of the previous linear projection layer for residual connection and ReLU activation, which not only enables the model to fuse features from different layers, but also alleviates the gradient disappearance or explosion phenomenon. Global average pooling is then used to average the features of different channels, compress feature information, and retain only the most critical local feature patterns. Finally, the outputs of all global average pooling in the three-way feature extraction branch are spliced ​​for multi-scale feature fusion, and the Convolution adjusts the feature dimension to the dimension size output by the global feature extraction module to facilitate further fusion of global and local features.

[0118] This application proposes a combination of cluster center distance loss of sample features with the same label and exponential distance loss between sample features with different labels. Since different aircraft have different flight speeds and maneuverability, the deep features of the same basic action type of different aircraft may be quite different. Therefore, if only the cross entropy commonly used in classification tasks is used as the loss function, the model may not fit the training data well enough, which is manifested in that there are still relatively large differences in the features between samples with the same category label when the model converges, or there are certain similarities in the features between samples with different category labels, which is not conducive to the subsequent reasoning test application of the model. Therefore, in order to improve the performance of the model, the differences between the classification features of samples with different types of labels can be combined as a compensation term of the loss function, that is, considering the classification task itself, while taking into account the more compact expression of the classification sample feature space. The total loss function constructed in this application is shown in formula (12).

[0119] The total loss function constructed is:

[0120] (12)

[0121] in, is the total loss function; Cross entropy loss for classification of basic aircraft action categories; The cluster center distance loss for training samples whose labels are predicted to be of the same category; An exponential distance loss between selectively fused features for predicting training samples with labels of different classes; and As a compensation term for the categorical cross entropy loss function; 、 、 for 、 and The balance coefficient between 、 、 . The selective fusion features are the features output after the training samples are processed by the temporal feature nonlinear enhancement module, the global feature extraction module, the multi-scale local feature extraction module, and the global and local feature fusion module. Since the main task of this application is to perform basic aircraft action recognition and classification, the first term occupies a dominant position in the total loss function of formula (12); secondly, in order to achieve the best classification effect, it is also necessary to ensure that the distance between sample features of different label categories is far enough, and the distance between sample features of the same label is close enough, so the last two terms are of secondary importance. The detailed definitions of these three loss functions are:

[0122] (13)

[0123] (14)

[0124] (15)

[0125] in, is the number of aircraft target time series data samples, and the aircraft target time series data samples are original data; The number of basic aircraft action categories; For the training samples are basic action categories The true label of For the training samples are predicted as basic action categories The predicted probability of For the Selective fusion features of training samples; For the Selective fusion features of training samples; Indicates that the label prediction is the basic action category The number of training samples, basic action categories The label Y belongs to the basic action category of the aircraft, and its value is 0, 1, 2 or 3; Indicates that the label prediction is the basic action category In the training sample Selective fusion features of training samples; For all predicted categories The clustering center of the selective fusion features of the training samples; Represents the exponential function.

[0126] The last two loss functions adopt a bounded loss design, the purpose of which is to ensure that the model is more inclined to the first cross-entropy loss function in the early stage of training, that is, the difference between the sample prediction probability and the true label probability distribution in the classification task; but in the stage close to the convergence of the model, the last two losses can be "close to or approaching" the value of the first cross-entropy loss in terms of value. At this time, the balance coefficient is used to better achieve a balance with the value range of the cross-entropy loss function, so as to fully consider the similarities and differences of the classification features in the deep space, thereby avoiding the huge difference between the value range and the value range of the classification cross-entropy function caused by directly using loss functions such as Euclidean distance, and thus avoiding abnormal derivative values ​​​​in the early backward propagation process of the selective feature fusion model training stage, which mistakenly causes the model to overfit the last two losses.

[0127] This application utilizes the advantages of convolutional neural networks and temporal neural networks in extracting local features and global feature patterns respectively, aiming to improve the deep feature mining capabilities of aircraft target time series data through a selective feature fusion model, and to compact the feature space after convergence of the selective feature fusion model through loss function design, thereby improving the robustness of the algorithm.

[0128] For ease of understanding, this application provides a more specific embodiment:

[0129] See Figure 1 and Figure 2 The present application provides a method for recognizing basic aircraft actions based on a selective feature fusion model, comprising the following steps:

[0130] (1) Original data input: The original data input of this application is mainly the original detection time series data or multi-source fusion time series data of aircraft targets obtained by our detection and tracking equipment. The original data at the moment is ,in, Represents longitude, Represents latitude, Represents height, represents the eastward speed, represents the northbound speed, Represents the celestial speed, represents the eastward acceleration, represents the northward acceleration, Represents the celestial acceleration, a total of 9 dimensions. Time has come time( The original data of the time step is accumulated to form a continuous time series data segment with the dimension of During the training phase, the original data is divided into time steps , time interval Generate time series data segments as training samples and The ratio is divided into training set and validation set.

[0131] (2) Original time series feature construction: The original data is input into the original time series data construction module to construct the domain knowledge of aircraft action recognition, and further obtain the features related to the aircraft action. Figure 1 and Figure 2 The original time series feature construction module in . It mainly constructs the aircraft's velocity scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle, and expands them into the original time series data.

[0132] airplane Velocity scalar at time The construction method is:

[0133]

[0134] airplane Acceleration scalar at time The construction method is:

[0135]

[0136] airplane Track angle at time The construction paradigm is:

[0137]

[0138] The range of track deviation angle is , the north direction is 0, and the north-east direction is positive.

[0139] airplane Aircraft track pitch angle at time The construction paradigm is:

[0140]

[0141] The range of track pitch angle is , the aircraft is positive when its nose is pointing up, and negative when its nose is pointing down.

[0142] The reason for constructing the above four eigenvalues ​​is that the aircraft's speed, acceleration, track angle change rate (i.e., the ratio of the continuous track angle difference to the time interval), and track pitch angle change rate (i.e., the ratio of the continuous track pitch angle difference to the time interval) can reflect the maneuverability of the real aircraft. If these features are added to the original time series data, the subsequent selective feature fusion model can capture these feature patterns, which is beneficial to the classification of the basic aircraft movements. Adding the above four features to the original time series data, a new feature vector is obtained. :

[0143]

[0144] In order to facilitate the training of the selective feature fusion model, it is also necessary to normalize the feature vector, considering the time step In the time range of , the maximum and minimum normalization principle is used for processing, then The eigenvector after normalization is:

[0145]

[0146] in, After normalization Characteristic data at the moment; and are functions for finding minimum and maximum values ​​respectively.

[0147] Finally, The feature vector of the time step Added together, they form the input features of the selective feature fusion model:

[0148]

[0149] In addition, the original time series feature construction also includes the time step The basic action categories of the aircraft are quantified to form quantitative labels :

[0150]

[0151] Among them, 0 means Corresponding to the basic action "straight forward"; 1 means Corresponding to the basic action "turn"; 2 means Corresponding to the basic action "climbing"; 3 means Corresponding to the basic action "dive";

[0152] (3) Construction of the selective feature fusion model: Pytorch is used to build the selective feature fusion model. Its network structure can be found in Figure 1, the selective feature fusion model constructs the output of the module with the original temporal features and quantitative labels As the network input. Specifically, the network structure consists of five parts: temporal feature nonlinear enhancement module, global feature extraction module, multi-scale local feature extraction module, global and local feature fusion module and feature classifier. The temporal feature nonlinear enhancement module is to transform the original input into Expand to Dimension, where Indicates the batch size; the global feature extraction module takes the output features of the temporal feature nonlinear enhancement module as input features, deeply extracts the global feature patterns related to the input features in all time steps, and compresses the features to , thereby automatically screening and obtaining important information; the multi-scale feature extraction module also uses the output features of the temporal feature nonlinear enhancement module as input features, deeply extracts local feature patterns in different scale spaces and performs multi-scale feature splicing and fusion in a small range, through Convolution adjusts the feature dimension to , thereby obtaining enhanced detail feature information; the global and local feature fusion module is to The global characteristics and The local features of the , further enhancing the feature expression capability; the feature classifier maps the deeply fused features to the probability representation of the aircraft's basic action categories, and the index of the maximum probability represents the category.

[0153] Among them, the second-order discrete state space model processing in the global feature extraction module mainly relies on formula (7), formula (8), formula (9), formula (10), and formula (11) for processing.

[0154] In formula (7), (at this time is the result of global feature extraction module after layer normalization) Depend on Mapped to , are model parameters.

[0155] In formula (8), Will Depend on Mapped to , are model parameters.

[0156] In formula (9), Will Depend on Mapped to , is the model parameter. To maintain dimensional consistency, let , collectively referred to as the dimension of the second-order discrete state space.

[0157] In formula (10) and formula (11), The hidden layer state is called the second-order discrete state space, and its dimension is , is a variable that needs to be parameterized and can be initialized to 0; , , , These are all model initialization parameters, which can be parameterized using the Parameter function in Pytorch. , The noise vectors representing the state equation and observation equation respectively can be initialized using a normal distribution with a mean of 0 and a variance of 1e-6.

[0158] When processing the second-order discrete state space model, the features are calculated by formula (10) and formula (11). Processing is done time-step by time-step, where is the time step index, output of all time steps Stacked together, we get the output of the second-order discrete state space model processing, whose output dimension is .

[0159] (4) Training phase: The total loss function constructed by formula (12), formula (13), formula (14) and formula (15) is used in the training phase. The balance coefficients of the loss function are , , The raw data in the training phase can be selected from the simulated time series data generated by various simulation environment platforms for aircraft targets, as well as a small amount of original detection time series data of aircraft targets or multi-source fusion time series data, to enrich the sample space in the training phase. The learning rate is set to 256, the number of learning rounds is 100, and the initial learning rate is set to 1e-2. The learning rate corresponding to a certain round number in the training process is updated using a piecewise learning rate update strategy, and the learning rate update segmentation interval is [15:30:60]. The Adam optimizer is used, and early stopping judgment is adopted. When the validation set loss does not decrease for 5 consecutive rounds, training is stopped.

[0160] (5) Reasoning stage: In the reasoning stage, the raw data is simply input into the original time series feature construction module to convert it into the input feature format required by the network structure, and the trained optional feature fusion model parameters are loaded to perform end-to-end reasoning to obtain the classification and recognition results of the basic aircraft action categories. Figure 2 The classification and recognition results of the basic aircraft action categories obtained by inference (i.e. the predicted labels) and the quantitative labels By comparing the real labels, we can calculate the recognition accuracy and other evaluation indicators.

[0161] (6) Aircraft basic action recognition test: The simulated time series data of various types of aircraft detected and tracked by a simulation platform simulated radar sensor are used as a data set. The method proposed in this application is used for preliminary aircraft basic action recognition tests. The average accuracy of all basic action recognition types is used for comparison. The results are shown in Table 1. In Table 1, TCN represents the temporal convolutional neural network method, FCN1D represents the one-dimensional full convolutional neural network method, LSTM represents the long short-term memory network method, and Transformer represents the transformer method; A1 represents the global feature extraction module of this application adopts the conventional zero-order hold state space model discretization method, and the loss function also only adopts the ordinary cross entropy loss function; A2 represents the use of only the global feature extraction module of this application, and the loss function only adopts the ordinary cross entropy loss function; A3 represents the full use of the global feature extraction module and multi-scale local feature extraction module of this application, and the loss function still adopts the ordinary cross entropy loss function; A4 represents the complete method of this application. Preliminary experimental results show that compared with conventional single convolutional neural network methods (TCN, FCN1D) or temporal neural network methods (LSTM), and even advanced Transformer methods, the average recognition accuracy of the method of this application has been improved to a certain extent, verifying the effectiveness of this application.

[0162] Table 1 Aircraft basic motion recognition test

[0163]

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit them. Although the present application has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present application can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present application should be included in the scope of protection of the claims of the present application.

Claims

1. A method for recognizing basic aircraft actions based on a selective feature fusion model, characterized in that: include: Step 1: Input the original time series data into the original time series feature construction module, obtain the features related to the aircraft movement and amplify them into the original time series data to obtain new time series data. Based on the new time series data, obtain the input features of the selective feature fusion model. and its corresponding aircraft basic action category label Y; the features related to the aircraft action include the aircraft's velocity scalar, acceleration scalar, aircraft track angle and aircraft track pitch angle; The aircraft basic action category label Y is used to indicate the aircraft basic action category; aircraft basic action categories include straight, turning, climbing, and diving; the original time series data includes time series data generated by simulation on various simulation environment platforms, original aircraft target detection time series data obtained through real sensor detection, and aircraft target multi-source fusion time series data obtained through multi-source fusion. Step 2: Construct a selective feature fusion model, which consists of a temporal feature nonlinear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the temporal feature nonlinear enhancement module is connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier; The global feature extraction module includes second-order discrete state space model processing; Step 3: Train the selective feature fusion model by constructing a total loss function to obtain the trained selective feature fusion model and its parameters; Step 4: Simply input the raw data into the original time series feature construction module to convert it into the input feature format required by the selective feature fusion model, load the trained selective feature fusion model and its parameters, and perform end-to-end inference to obtain the recognition results of the aircraft's basic movements. The aircraft basic action recognition result is a type of aircraft basic action category; the original data includes time series data generated by simulation on various simulation environment platforms, original detection time series data of aircraft targets obtained through real sensor detection, and multi-source fusion time series data of aircraft targets obtained through multi-source fusion means.

2. The method according to claim 1, characterized in that The step 1 comprises: Assumptions The original time series data at the moment is ,in, Represents longitude, Represents latitude, Represents height, represents the eastward speed, represents the northbound speed, Represents the celestial speed, represents the eastward acceleration, represents the northward acceleration, represents the celestial acceleration; The original time series data Input into the original time series data construction module to get the aircraft Velocity scalar at time for: Get the plane Acceleration scalar at time for: Get the plane Track angle at time for: The range of track deviation angle is , the north direction is 0, and the north-east direction is positive; airplane Aircraft track pitch angle at time for: The range of track pitch angle is , the aircraft nose is pointing up is positive, and the nose is pointing down is negative; Add the aircraft's velocity scalar, acceleration scalar, aircraft track angle, and aircraft track pitch angle to the original time series data , get new time series data : The new time series data After normalization, we get: in, After normalization Characteristic data at the moment; and are functions for finding minimum and maximum values, respectively. is the time step; Will The feature vector of the time step Accumulate to form the input features of the selective feature fusion model : The time step The basic action category labels of the aircraft are quantified to obtain the basic action category labels of the aircraft. : Among them, 0 means The corresponding aircraft basic action category is "straight"; 1 means The corresponding basic action category of the aircraft is "turn"; 2 means The corresponding aircraft basic action category is "climb"; 3 means The corresponding basic aircraft action category is "dive".

3. The method according to claim 1, characterized in that In step 2, the global feature extraction module includes two branches, a linear projection layer, and a global average pooling layer, and the outputs of the two branches are connected to the global average pooling layer through the linear projection layer; One of the two branches will input the feature And its corresponding aircraft basic action category label Y is sequentially processed by linear projection, layer normalization, and second-order discrete state space model. The other branch processes the input feature The corresponding aircraft basic action category label Y is linearly projected and SiLU activated in turn, and then the features output by the two branches are multiplied, and the multiplication result is linearly projected to restore it to the input feature The dimension of , and finally the final global features are obtained through global average pooling; Input features is the output of the temporal feature nonlinear enhancement module.

4. The method according to claim 3, characterized in that The second-order discrete state space model is processed as the feature output after the layer normalization processing by the second-order discrete state space model Process and get output : The process of constructing the second-order discrete state space model is: The state space model of the random system is introduced. The state space model includes the state equation and the observation equation: (1) (2) in, is the state vector at time t, for The differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transfer matrix, which is used to control the change of state over time; is the input matrix, which is used to control the impact of the input on the state; is the output matrix, which represents the output generated based on the current state; is the feedforward matrix, which represents the direct impact of input on output; Used to indicate that the random system is a continuous-time system; Formula (1) is the state equation, and Formula (2) is the observation equation; Formulas (1) and (2) are approximated by discretization and converted into: (3) (4) in, is the kth time step hidden state of the second-order discrete state space model, is the k-1th time step hidden state of the second-order discrete state space model, The deterministic input at the kth time step, and Represents a discrete interval index, i.e., a time step index; is the output of the kth time step of the second-order discrete state space model; 、 In order to form a new state transfer matrix and input matrix by discretization, the matrix is ​​obtained by solving the differential equations of formulas (3) and (4) through Taylor series discretization expansion. 、 ; Expand the state equation of formula (3) through Taylor series, retain the quadratic term approximation, and the new state transfer matrix and the new input matrix The form is: (5) (6) Among them, the parameters Parameters introduced for discretization; is a unit array; and are the state transfer matrix and input matrix, respectively, both of which are parameters that require network initialization; is a high-order term; The following formula and Modeling as Features Function: (7) (8) in, is a softplus function, to ensure Provide nonlinear control capabilities; , , , is the linearization parameter; In order to make the output of the second-order discrete state space model follow the characteristics Control state flow, the output matrix in the observation equation of formula (4) Modeling as Features Function: (9) in, and are all linearization parameters; Substitute formula (5) to formula (8) into formula (3), and change the characteristic As a deterministic input, substitute formula (9) into formula (4) and introduce the noise vector of the state equation and the noise vector of the observation equation , the second-order discrete state space model is obtained as: (10) (11) Among them, the newly added parameters and Error in discretizing the equilibrium state-space model.

5. The method according to claim 1, wherein The multi-scale local feature extraction module includes three feature extraction branches, a feature splicing layer, and a convolution layer. Each feature extraction branch in the three-way feature extraction branch includes a linear projection layer, a causal convolution layer, a batch normalization layer, a ReLU activation layer, and a global average pooling layer connected in sequence. The global average pooling layer of each feature extraction branch in the three-way feature extraction branch is connected to the convolution layer through the feature splicing layer. The convolution kernel size used in the causal convolution layer of each feature extraction branch in the three-way feature extraction branch is different. Each feature extraction branch uses a linear projection layer to transform the input features Nonlinear feature enhancement is performed, and then each feature extraction branch uses convolution kernels of different sizes to extract local feature patterns within different receptive fields, and then batch normalization is performed. The batch-normalized features are residually connected with the output of the linear projection layer and then ReLU activation is performed. Global average pooling is used to average the features of different channels, compress the feature information, and only retain the most critical local feature patterns. Finally, the outputs of all global average pooling in the three feature extraction branches are feature spliced ​​and multi-scale feature fusion is performed. The convolution layer is used to adjust the feature dimension after multi-scale feature fusion to the dimension output by the global feature extraction module.

6. The method according to claim 1, characterized in that The global and local feature fusion module obtains selective fusion features by splicing the global features output by the global feature extraction module and the local features output by the multi-scale local feature extraction module, and fusing them through two linear projection scalings.

7. The method according to claim 1, characterized in that The feature classifier maps the selective fusion features output by the global and local feature fusion modules to the probability representation of the aircraft's basic action categories, and the index of the maximum probability represents the aircraft's basic action category.

8. The method according to claim 1, characterized in that The step 3 includes: During the training phase, the original data is used as training samples, and the total loss function is used to train the selective feature fusion model, and the balance coefficient of the total loss function is set; the batch size, number of learning rounds, and initial learning rate are set, and the segmented learning rate update strategy is used to update the learning rate corresponding to a certain number of rounds in the training process, and the segmented interval of the learning rate update is set; the Adam optimizer is used to optimize the selective feature fusion model, and early stopping judgment is used, that is, when the validation set loss does not decrease for multiple consecutive rounds, the training is stopped to obtain the parameters of the trained selective feature fusion model.

9. The method according to claim 8, characterized in that The total loss function constructed is: (12) in, is the total loss function; Cross entropy loss for classification of basic aircraft action categories; The cluster center distance loss for training samples whose labels are predicted to be of the same category; An exponential distance loss between selectively fused features for predicting training samples with labels of different classes; and As a compensation term for the categorical cross entropy loss function; 、 、 for 、 and The balance coefficient between them; the optional fusion feature is the feature output by the global and local feature fusion module.

10. The method according to claim 9, characterized in that 、 and The expressions are: (13) (14) (15) in, is the number of aircraft target time series data samples, and the aircraft target time series data samples are original data; The number of basic aircraft action categories; For the training samples are basic action categories The true label of For the training samples are predicted as basic action categories The predicted probability of For the Selective fusion features of training samples; For the Selective fusion features of training samples; Indicates that the label prediction is the basic action category The number of training samples, basic action categories The label Y belongs to the basic action category of the aircraft, and its value is 0, 1, 2 or 3; Indicates that the label prediction is the basic action category In the training sample Selective fusion features of training samples; For all predicted categories The clustering center of the selective fusion features of the training samples; Represents the exponential function.

Images