Aircraft basic action recognition method based on selectable feature fusion model

Abstract

The invention discloses an aircraft basic action recognition method based on a selectable feature fusion model, and the method comprises the steps: inputting original time sequence data into an original time sequence feature construction module, obtaining features related to aircraft actions, amplifying the features into the original time sequence data, obtaining new time sequence data, and carrying out the recognition of the aircraft basic actions according to the new time sequence data. Input features of the selectable feature fusion model and corresponding aircraft basic action category labels are obtained; constructing a selective feature fusion model; training the selectable feature fusion model through the total loss function to obtain a trained selectable feature fusion model and parameters thereof; the method comprises the following steps: inputting original data into an original time sequence feature construction module, converting the original data into an input feature format required by a selectable feature fusion model, loading trained selectable feature fusion model parameters, and carrying out end-to-end reasoning to obtain an identification result of basic actions of an aircraft; according to the invention, the performance of aircraft basic action recognition is improved.

Description

Technical Field

[0001] This application relates to the field of situation awareness technology, and particularly to an aircraft basic motion recognition method based on a selective feature fusion model. Background Art

[0002] The basic motion of an aircraft is the smallest unit that constitutes the complex maneuvering motions of the aircraft, and is one of the bases for further high-level situation awareness such as aircraft target intention recognition.

[0003] In terms of the relevant research background, currently most scholars directly skip the recognition of the basic motions of aircraft and directly conduct the recognition of complex maneuvering motions (such as loops, rolls, etc.), or conduct simulation research by mixing basic motions and complex maneuvering motions together. The research data basis mainly relies on the continuous trajectory segments output by the simulation platform. However, most of them neither consider the high dynamics and antagonism of non-cooperative aircraft targets, nor consider the usage time limit of the sensors of the own airborne platform. For example, in the scenario where the own radar actively detects and tracks other aircraft targets, on the one hand, when the target discovers that it is being detected and tracked, it will actively maneuver to get rid of it, which is not conducive to our side to master its continuous motion characteristics. On the other hand, the own active radar also works intermittently, reducing the probability of being detected. Usually, it will not be turned on for a long time, resulting in discontinuous and short-time aircraft target trajectories detected, which means that it is theoretically difficult to identify whether the target is performing long-term maneuvering motions (such as rolls, which generally require a long time). In addition, the trajectory data obtained through active detection cannot infer that the target is performing maneuvers such as rolls, and it is also difficult to identify maneuvers such as loops performed by the target in a small space due to the accuracy limitation of detection and tracking. Therefore, such research methods are mainly applicable to applications such as flight quality assessment and aircraft fault detection of the own aircraft, and are not suitable for the field of situation awareness.

[0004] In the related technologies, the basic methods for identifying the actions of traditional aerial aircraft targets mainly include methods based on expert knowledge base reasoning, time series similarity methods such as dynamic time warping, and traditional machine learning methods such as support vector machines or Bayesian networks. The method based on expert knowledge base reasoning first needs to establish an action feature library for various aircraft targets through expert knowledge, which requires a high accuracy in describing the action features of aircraft. However, the features of the detected and tracked aircraft targets that usually need to be identified online often show a high degree of ambiguity, reducing the recognition accuracy. The problem of how to balance the accuracy and ambiguity of feature description completely depends on the subjective consciousness of experts. Therefore, it is often necessary to frequently update the knowledge base, resulting in high maintenance costs. Although the time series similarity method does not require an expert knowledge base, it is difficult to identify similar flight actions. Moreover, due to being very sensitive to noise and high dynamic mutations of trajectories, it is difficult to determine the threshold parameters, resulting in low recognition accuracy. The method based on Bayesian theory depends on expert experience in aspects such as network structure design and parameter assignment of prior probabilities, with a relatively high design threshold. And traditional machine learning methods such as support vector machines generally can only learn shallow features, and the accuracy of classification and recognition needs to be improved. Generally speaking, the above methods all have problems such as low recognition accuracy and insufficient generalization ability to varying degrees.

[0005] Due to the limitations of traditional algorithms, scholars have been exploring new technical approaches. With the gradual penetration of artificial intelligence technologies such as deep learning into various technical fields, its powerful feature extraction and pattern recognition capabilities, without the need for complex domain theory formula derivation and expert experience, as well as the data-driven end-to-end recognition mode and other advantages, have attracted the high attention of relevant scholars in this field. More and more deep learning methods have gradually been applied to the research of aircraft target basic action recognition. Scholars convert the aircraft basic action recognition into a classification problem in pattern recognition, and through constructing a neural network classification model for learning and training, obtain the mapping relationship between aircraft detection and tracking data and basic action classification labels, so as to realize the automatic recognition of aircraft target basic actions. Relevant research mainly designs aircraft target basic action recognition methods based on architectures such as convolutional neural networks and temporal neural networks. The method based on convolutional neural networks usually constructs an optional feature fusion model, such as a fully convolutional neural network, a temporal convolutional neural network, etc., by stacking convolutional layers, pooling layers or fully connected layers, etc., to capture complex local patterns and structures in action unit data. However, for aircraft action recognition time series data, the receptive field of the convolutional layer is usually of a fixed size and cannot be dynamically adjusted, and it overly focuses on local feature patterns, and has a weak ability to capture long-term dependence relationship feature patterns. The method based on temporal neural networks (such as recurrent neural networks, long short-term memory networks, gated recurrent units and their improved algorithms, etc.) has a powerful ability to capture long-term dependence patterns, especially suitable for in-depth feature mining of data with clear temporal relationships. However, the performance of this type of method in filtering redundant information in input data is insufficient, and it may experience unstable training. Although some scholars also combine the attention mechanism on the basis of the above network architectures to capture some key features and increase the robustness of the model in some specific scenarios, essentially it does not break through the limitations of the above network architectures. In addition, when scholars design the classification loss function for aircraft target action recognition, most of them directly adopt the cross-entropy loss function, only considering the difference degree of the probability distribution of the classification problem, without considering the feature compactness between the same label samples and the separability between different label samples, resulting in a strong dependence of the training model on training samples, which is not conducive to the model to improve the generalization ability. Therefore, further realizing the selective extraction of important information in global features and local feature patterns in time series data, filtering redundant feature information, and taking into account the similarity and difference of features of different label samples in the classification task have a positive effect on improving the performance of the selective feature fusion model for aircraft target basic action recognition based on deep learning. Summary of the Invention

[0006] Aiming at the limitations of existing aircraft basic motion recognition methods based on deep learning, this application provides an aircraft basic motion recognition method based on a selective feature fusion model, which is applicable to the basic motion recognition tasks of airborne platform radar active detection and tracking, and multi-source detection and fusion tracking of aircraft targets. The conversion equation between the hidden state and the input features is constructed by the second-order discretization approximation of the stochastic system state space model, and a long-distance dependent global feature pattern extractor for the temporal features of aircraft basic motions is designed to enhance the ability of the selective feature fusion model to selectively focus on important information and ignore irrelevant information in the input features. On this basis, a local feature pattern extractor is designed through multi-scale causal convolution and small-range feature fusion to increase the ability to capture local features in different time windows, and global and local feature fusion enhancement is performed to improve the recognition performance. At the same time, by combining the clustering center distance loss of the same label sample features and the exponential distance loss between different label sample features, a new classification loss function is designed to improve the fitting degree of the selective feature fusion model and enhance the classification recognition performance.

[0007] This application discloses an aircraft basic motion recognition method based on a selective feature fusion model, which includes: Step 1: Input the original temporal data into the original temporal feature construction module to obtain the features related to the aircraft motion and amplify them into the original temporal data to obtain new temporal data. According to the new temporal data, the input features of the selective feature fusion model are obtained. And its corresponding aircraft basic motion category label Y; the features related to the aircraft motion include the aircraft speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle; the aircraft basic motion category label Y is used to indicate the aircraft basic motion category; the aircraft basic motion categories include straight flight, turning, climbing, and diving. Step 2: Construct a selective feature fusion model, which consists of a temporal feature non-linear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the temporal feature non-linear enhancement module is respectively connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier. Step 3: Train the selective feature fusion model by constructing a total loss function to obtain the trained selective feature fusion model and its parameters. Step 4: Just input the original data into the original temporal feature construction module to convert it into the input feature format required by the selective feature fusion model, load the parameters of the trained selective feature fusion model, and perform end-to-end inference to obtain the recognition result of the aircraft basic motion; the recognition result of the aircraft basic motion is one of the aircraft basic motion categories.

[0008] Furthermore, step 1 includes: Assume the original time-series data at the moment is , where represents longitude, represents latitude, represents altitude, represents eastward velocity, represents northward velocity, represents upward velocity, represents eastward acceleration, represents northward acceleration, represents upward acceleration; Input the original time-series data into the original time-series data construction module to obtain the speed scalar of the aircraft at the moment as:

[0009] Obtain the acceleration scalar of the aircraft at the moment as:

[0010] Obtain the track deviation angle of the aircraft at the moment as:

[0011] The value range of the track deviation angle is , with the due north direction being 0 and north by east being positive; The aircraft moment's aircraft track pitch angle is:

[0012] The value range of the track pitch angle is , with the aircraft nose up being positive and the head down being negative; Add the aircraft's speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle to the original time-series data to obtain the new time-series data :

[0013] Normalize the new time-series data to obtain:

[0014] Among them, represents after normalization Feature data at a moment; and are functions for finding the minimum and maximum values respectively, is the time step; Sum the feature vectors of time steps to form the input features of the optional feature fusion model :

[0015] Quantify the labels of the basic aircraft motion categories at the time step to obtain the basic aircraft motion category labels :

[0016] Among them, 0 means the corresponding basic aircraft motion category is "going straight"; 1 means the corresponding basic aircraft motion category is "turning"; 2 means the corresponding basic aircraft motion category is "climbing"; 3 means the corresponding basic aircraft motion category is "diving".

[0017] Furthermore, in step 2, the global feature extraction module includes two branches, a linear projection layer, and a global average pooling layer. The outputs of the two branches are connected through the linear projection layer and the global average pooling layer; One branch of the two branches linearly projects, layer normalizes, and processes the input features and their corresponding basic aircraft motion category labels Y in sequence through a second-order discrete state space model. The other branch linearly projects and activates the input features and their corresponding basic aircraft motion category labels Y through SiLU in sequence, then multiplies the features output by the two branches, and linearly projects the multiplied result to restore to the dimension of the input features , and finally obtains the final global features through global average pooling; the input features are the outputs of the temporal feature non-linear enhancement module.

[0018] Furthermore, the second-order discrete state space model processing is to process the features output after layer normalization through a second-order discrete state space model to obtain the output : The process of constructing the second-order discrete state space model is as follows: Introduce the state space model of a stochastic system. The state space model includes a state equation and an observation equation: (1) (2) Among them, is the state vector at time t, is the differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transition matrix, used to control the change of the state over time; is the input matrix, used to control the influence of the input on the state; is the output matrix, representing the output generated based on the current state; is the feedforward matrix, representing the direct influence of the input on the output; is used to indicate that the stochastic system is a continuous-time system; Equation (1) is the state equation, and Equation (2) is the observation equation; Approximate Equations (1) and (2) through discretization and convert them to: (3) (4) Among them, is the hidden state at the k-th time step of the second-order discrete state space model, is the hidden state at the (k - 1)-th time step of the second-order discrete state space model, the deterministic input at the k-th time step, and represent the discrete interval index, i.e., the time step index; is the output at the k-th time step of the second-order discrete state space model; , are the new state transition matrix and input matrix formed through discretization, mainly obtained by discretizing the differential equations of Equations (3) and (4) through Taylor series expansion to obtain the matrices , ; Expand the state equation of Equation (3) through Taylor series, retain the quadratic term approximation, and the new state transition matrix and the new input matrix are in the form of: (5) (6) Among them, the parameter is the parameter introduced by discretization; is the identity matrix; and They are the state transition matrix and the input matrix respectively, both of which are parameters that need to be initialized by the network; is the high-order term; Model and as functions of the feature by the following formula: (7) (8) where is the softplus function, to ensure that provides non-linear control ability; , , , are linearization parameters; To enable the output of the second-order discrete state space model to control the state flow along with the feature , model the output matrix in the observation equation of formula (4) as a function of the feature : (9) where and are both linearization parameters; Substitute formulas (5) to (8) into formula (3), regard the feature as a deterministic input, and at the same time substitute formula (9) into formula (4), and introduce the noise vector of the state equation and the noise vector of the observation equation, to obtain the second-order discrete state space model as: (10) (11) where the newly added parameters and are used to balance the error of the discretization of the state space model.

[0019] Furthermore, the multi-scale local feature extraction module includes three feature extraction branches, a feature splicing layer, and a convolutional layer; each of the three feature extraction branches includes a linear projection layer, a causal convolutional layer, a batch normalization layer, a ReLU activation layer, and a global average pooling layer connected in sequence; the global average pooling layer of each of the three feature extraction branches is connected to the convolutional layer through the feature splicing layer; the causal convolutional layers of each of the three feature extraction branches use different convolutional kernel sizes; Each feature extraction branch uses a linear projection layer to project the input feature Nonlinear feature enhancement is performed. Then, each feature extraction branch uses convolutional kernels of different sizes to extract local feature patterns within different receptive fields, followed by batch normalization. After batch normalization, the features are subjected to residual connection with the output of the linear projection layer and then ReLU activation. Global average pooling is used to average the features of different channels, compressing the feature information and retaining only the most critical local feature patterns. Finally, the outputs of all global average pooling in the three feature extraction branches are concatenated for feature fusion at multiple scales, and a convolutional layer is used to adjust the feature dimension after multi-scale feature fusion to the dimension output by the global feature extraction module.

[0020] Further, the global and local feature fusion module fuses the global features output by the global feature extraction module with the local features output by the multi-scale local feature extraction module by concatenation and through two linear projection scalings to obtain selectively fused features.

[0021] Further, the feature classifier maps the selectively fused features output by the global and local feature fusion module to the probability representation of the basic aircraft action categories, and the index of the maximum probability represents the basic aircraft action category.

[0022] Further, step 3 includes: In the training stage, the original data is used as training samples, and the selectively feature fusion model is trained using the total loss function, and the balance coefficient of the total loss function is set; the original data includes time series data simulated and generated by various simulation environment platforms, the original detection time series data of aircraft targets obtained through real sensors, and the multi-source fusion time series data of aircraft targets obtained through multi-source fusion means. The batch size, number of training epochs, and initial learning rate are set. The piecewise learning rate update strategy is used to update the learning rate corresponding to a certain epoch during training, and the piecewise intervals for learning rate update are set. The Adam optimizer is used to optimize the selectively feature fusion model, and early stopping judgment is adopted, that is, when the validation set loss does not decrease for multiple consecutive epochs, the training is stopped to obtain the parameters of the trained selectively feature fusion model.

[0023] Further, the constructed total loss function is: (12) where is the total loss function; is the cross-entropy loss for the classification of basic aircraft action categories; is the distance loss of the training sample clustering center with the same predicted label; is the exponential distance loss between the selectively fused features of training samples with different predicted labels; and As a compensation term for the categorical cross-entropy loss function; , , is , and the balance coefficient between; The selectively fused feature is the feature output by the global and local feature fusion module.

[0024] Furthermore, , and The expressions of are respectively: (13) (14) (15) Among them, is the number of aircraft target time-series data samples, and the aircraft target time-series data sample is the original data; is the number of aircraft basic action categories; is the th training sample is the true label of the basic action category ; is the predicted probability that the th training sample is predicted as the basic action category ; is the selectively fused feature of the th training sample; is the selectively fused feature of the th training sample; represents the number of training samples whose label is predicted as the basic action category , and the basic action category belongs to the aircraft basic action category label Y, and its value is 0, 1, 2, or 3; represents the selectively fused feature of the th training sample among the training samples whose label is predicted as the basic action category ; is the clustering center of the selectively fused features of all training samples whose predicted category is ; represents the exponential function. Due to the adoption of the above technical solutions, the present application has the following advantages: 1. By constructing a new feature fusion mode of global and local features of the target (aircraft) time-series data, the phenomenon of insufficient expression of a single type of feature is alleviated, the feature expression ability is improved, and the subsequent classification performance is promoted; 2. By performing second-order discrete optimization and noise vector balance on the state space model, the approximation accuracy of the second-order discrete state space model is improved, and the coefficient matrix is ​​modeled as a function of the input feature vector, which promotes the selective feature fusion model to adaptively extract key information from the input features and filter out redundant information, thus realizing the automatic selection and extraction of global feature patterns; 3. By designing a multi-scale causal convolution and residual connection network module (multi-scale local feature extraction module), the future information leakage generated by the traditional convolutional network is avoided, which is more in line with the characteristics of the time continuity of the basic action data of the aircraft. At the same time, through the fusion of small-scale multi-scale features, the local feature expression ability is further improved; 4. By adding the cluster center distance loss and exponential distance loss of the classification feature samples on the basis of the classification cross entropy loss, the feature distance of the predicted samples with the same label can be shortened, and the feature distance between the predicted samples with different labels can be widened. In this way, the model's fitting degree of the fusion features can be increased while taking into account the classification task of the basic aircraft action category recognition, and the adaptability of the model can be further improved. 5. In general, on the one hand, the global feature extraction mode is optimized to achieve higher-precision selective feature extraction. On the other hand, considering the characteristics of time series data, local feature enhancement is performed through multi-scale local feature extraction and fusion, more local feature patterns are mined, and the fusion enhancement network of global and local features is designed to filter redundant information, retain key features that are beneficial to the subsequent classification of basic aircraft actions, and improve classification performance. In addition, the differences between the features of samples with different labels are considered in the classification loss function, and the loss function is designed in a targeted manner. Its purpose is to not only consider the influence of label prediction results during the training process, but also take into account the improvement of the expression ability of fusion features, thereby further improving the recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0026] Figure 1 A block diagram corresponding to a method for recognizing basic aircraft actions based on a selective feature fusion model according to an embodiment of the present application; Figure 2 This is a schematic diagram of the application flow of another method for basic aircraft motion recognition based on a selective feature fusion model according to an embodiment of the present application. DETAILED DESCRIPTION

[0027] The present application will be further described in conjunction with the accompanying drawings and embodiments. The described embodiments are only a part of the embodiments of the present application, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art shall fall within the scope of protection of the embodiments of the present application.

[0028] Referring to Figure 1 , an embodiment of a method for identifying basic aircraft maneuvers based on a selective feature fusion model is provided in the present application. A transformation equation between hidden states and input features is constructed by approximating the second-order discretization of a stochastic system state space model. Specifically, the temporal features of the target (aircraft) are used as deterministic inputs and substituted into the state space model. The state equation is discretized by performing a Taylor series expansion on the state equation, and high-order terms are discarded and only the second-order terms are retained, so that the coefficient matrix of the state equation is more accurate. At the same time, the coefficient matrix of the state equation is modeled as a function of the input features, so that in network training, the model parameters automatically focus on the key information therein and filter out redundant components according to the input features. At the same time, the discretization error is balanced by adding a noise vector, and the approximation accuracy of the state space model is improved as much as possible, and the extraction performance of the global feature pattern with long-distance dependence is improved; on this basis, relying on the advantage of the convolutional network in local feature extraction, considering the progressive relationship of the temporal feature data in the time dimension, through multi-scale causal convolution and feature splicing fusion, the problem of leakage of future information by traditional convolution is overcome, and the extraction performance of local feature patterns is improved. Finally, the feature expression ability of the entire selective feature fusion model is further enhanced through the fusion of global and local feature patterns; in addition, the present application considers combining the clustering center distance loss of the same-label sample features and the exponential distance loss between different-label sample features in the classification loss function to improve the fitting degree of the selective feature fusion model, slow down the underfitting risk when the selective feature fusion model converges, and improve the automatic recognition performance of the selective feature fusion model.

[0029] The technical solution of this embodiment includes: Step 1: Input the original temporal data into the original temporal feature construction module to obtain features related to the aircraft maneuver and amplify them into the original temporal data to obtain new temporal data, and obtain the input features of the selective feature fusion model according to the new temporal data and its corresponding basic aircraft maneuver category label Y; the features related to the aircraft maneuver include the speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle of the aircraft; the basic aircraft maneuver category label Y is used to indicate the basic aircraft maneuver category; the basic aircraft maneuver categories include straight flight, turning, climbing, and diving; Step 2: Construct an optional feature fusion model, which consists of a temporal feature non-linear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the temporal feature non-linear enhancement module is respectively connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier; Step 3: Train the optional feature fusion model by constructing a total loss function to obtain the trained optional feature fusion model and its parameters; Step 4: Just input the original data into the original temporal feature construction module to convert it into the input feature format required by the optional feature fusion model, load the parameters of the trained optional feature fusion model, and perform end-to-end inference to obtain the recognition result of the basic aircraft actions; the recognition result of the basic aircraft actions is one of the basic aircraft action categories.

[0030] For the sake of convenience of explanation, first introduce the state space model of the stochastic system and the concept of its discretization. The state space model of the stochastic system is derived from the Kalman filter and is a description of the dynamic behavior of the system over time, which is widely used in control theory. It usually includes the following two equations: (1) (2) where, is the state vector at time t, is the differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transition matrix, which is used to control the change of the state over time; is the input matrix, which is used to control the influence of the input on the state; is the output matrix, which represents the output generated based on the current state; is the feedforward matrix, which represents the direct influence of the input on the output; is used to indicate that the stochastic system is a continuous-time system; Equation (1) is the state equation, and Equation (2) is the observation equation; Since the above state space model is a continuous-time system, there is a differential form of the state vector , and it needs to be approximated by discretization and converted into the following form: (3) (4) where the subscript and represents a discrete interval index, i.e., a time step index; is the hidden state at the k-th time step of the second-order discrete state space model, is the hidden state at the (k - 1)-th time step of the second-order discrete state space model, the deterministic input at the k-th time step, and represents a discrete interval index, i.e., a time step index; is the output at the k-th time step of the second-order discrete state space model; 、 are the new state transition matrix and input matrix formed by discretization, mainly obtained by discretizing the differential equations in formulas (3) and (4) through Taylor series expansion to solve for the matrices 、 ; Refer to Figure 1 the global feature extraction module in, which includes two branches, a linear projection layer, and a global average pooling layer. The outputs of the two branches are connected through the linear projection layer and the global average pooling layer; one of the two branches linearly projects the input feature and its corresponding aircraft basic action category label Y in sequence, performs layer normalization, and processes them through a second-order discrete state space model. The other branch linearly projects and applies SiLU activation to the input feature and its corresponding aircraft basic action category label Y in sequence, then multiplies the features output by the two branches, linearly projects the result of the multiplication to restore to the dimension of the input feature , and finally obtains the final global feature through global average pooling; the input feature is the output of the temporal feature non-linear enhancement module.

[0031] Figure 1 The processing by the second-order discrete state space model in is to process the feature output after layer normalization through the second-order discrete state space model to obtain the output : Expand the state equation of formula (3) through Taylor series, retain the quadratic term approximation, and the new state transition matrix and the new input matrix are in the form of: (5) (6) where the parameter is the parameter introduced by discretization; is the identity matrix; and are the state transition matrix and the input matrix respectively, both of which are parameters that need to be initialized by the network; is the high-order term.

[0032] In order to enable the second-order discrete state space model to filter out irrelevant information and compress the context of the second-order discrete state space model into an effective state, so that the second-order discrete state space model can more finely control the propagation of the input and state of the second-order discrete state space model, , is modeled as a function of the input feature : (7) (8) where, is the softplus function, which provides non-linear control ability to ensure ; , , , are linearization parameters.

[0033] Then, in order to enable the output of the second-order discrete state space model to control the state flow along with the feature , the output matrix in the observation equation of formula (4) is also modeled as a function of the input feature : (9) where, , are linearization parameters.

[0034] Substitute formula (5), formula (6), formula (7), and formula (8) into formula (3), regard the feature as a deterministic input, and at the same time substitute formula (9) into formula (4), and introduce the noise vector of the state equation and the noise vector of the observation equation, and the constructed second-order discrete state space model is obtained as: (10) (11) where, is the hidden state of the second-order discrete state space model; is the output of the second-order discrete state space model. The newly added parameters , are used to balance the error of the discretization of the state space model and can be regarded as a regularization term, which is generally initialized as a small parameter.

[0035] The local feature extraction module structure based on multi-scale causal convolution and residual network connection constructed in this application is shown in Figure 1 The multi-scale local feature extraction module, which includes three feature extraction branches, a feature splicing layer, and a convolutional layer; each of the three feature extraction branches uses a linear projection layer for non-linear feature enhancement, and then the three branches respectively use , , Causal convolution with the size of the convolutional kernel extracts local feature patterns within different receptive fields, and then performs batch normalization. Then, it is connected with the output of the previous linear projection layer through a residual connection and activated by ReLU, which not only enables the model to fuse features of different layers, but also can alleviate the phenomenon of gradient disappearance or explosion; then, global average pooling is used to average the features of different channels, compress the feature information, and only retain the most critical local feature patterns; finally, the outputs of all global average pooling in the three feature extraction branches are spliced for multi-scale feature fusion, and Convolution is used to adjust the feature dimension to the dimension size of the output of the global feature extraction module, so as to facilitate further global and local feature fusion in the subsequent process.

[0036] The clustering center distance loss that combines the features of samples with the same label and the exponential distance loss between the features of samples with different labels proposed in this application. Since the flight speeds and maneuvering capabilities of different aircraft are different, the deep features of the same basic action type of different aircraft may be significantly different. Therefore, if only the commonly used cross-entropy in the classification task is used as the loss function, it may lead to insufficient fitting of the model to the training data, manifested as there are still relatively large differences in the features between samples with the same category label when the model converges, or there is a certain similarity in the features between samples with different category labels, which is not conducive to the subsequent inference and testing applications of the model. Therefore, to improve the performance of the model, the differences between the classification features of samples with different types of labels can be combined as a compensation term for the loss function, that is, considering the classification task itself and also taking into account the more compact expression of the feature space of the classification samples. The total loss function constructed in this application is shown in formula (12).

[0037] The constructed total loss function is: (12) Among them, is the total loss function; is the cross-entropy loss for the classification of aircraft basic action categories; is the clustering center distance loss of training samples with the same predicted label; is the exponential distance loss between the selectable fusion features of training samples with different predicted labels; and serve as compensation terms for the cross-entropy loss function; , , is , and The balance coefficient between can take , , . The selectively fused feature is the feature output after the training samples are processed by the temporal feature non - linear enhancement module, the global feature extraction module, the multi - scale local feature extraction module, and the global - local feature fusion module. Since the main task of this application is to perform aircraft basic action recognition and classification, among the total loss functions in formula (12), the first item occupies the main position; secondly, while achieving the best classification effect, it is also necessary to ensure that the distances between the sample features of different label categories are far enough, and the distances between the sample features of the same label are close enough. Therefore, the latter two items are in a secondary position. The detailed definitions of these three loss functions are as follows: (13) (14) (15) Among them, is the number of aircraft target temporal data samples, and the aircraft target temporal data sample is the original data; is the number of aircraft basic action categories; is the th training sample, and is the true label of the basic action category is the prediction probability that the th training sample is predicted as the basic action category ; is the selectively fused feature of the th training sample; is the selectively fused feature of the th training sample; represents the number of training samples whose labels are predicted as the basic action category , and the basic action category belongs to the aircraft basic action category label Y, and its value is 0, 1, 2, or 3; represents the selectively fused feature of the th training sample among the training samples whose labels are predicted as the basic action category ; is the clustering center of the selectively fused features of all training samples whose predicted categories are ; represents the exponential function. The latter two loss functions adopt a bounded loss design, aiming to ensure that the model in the early stage of the training phase is more inclined to the first cross-entropy loss function, that is, the difference between the sample prediction probability and the true label probability distribution in the classification task; however, when approaching the convergence stage of the model, the latter two losses can be "close to or approaching" the value of the first cross-entropy loss numerically. At this time, the balance coefficient is used to better achieve the balance with the value range of the cross-entropy loss function, so as to fully consider the similarity and difference of classification features in the deep space, thereby avoiding the huge difference between the value range of directly using loss functions such as Euclidean distance and the value range of the classification cross-entropy function, and further avoiding the abnormal derivative values generated during the backpropagation process in the early stage of the training phase of the selective feature fusion model, which may wrongly cause the model to overfit the latter two losses.

[0038] This application utilizes the advantages of convolutional neural networks and temporal neural networks in local feature and global feature pattern extraction respectively, aiming to improve the deep feature mining ability of aircraft target temporal data through a selective feature fusion model, and design a loss function to compact the feature space after the convergence of the selective feature fusion model, thereby improving the robustness of the algorithm.

[0039] For the convenience of understanding, this application provides a more specific embodiment: Refer to Figure 1 and Figure 2 A method for recognizing basic aircraft actions based on a selective feature fusion model provided by this application includes the following steps: (1) Original data input: The original data input of this application is mainly the original detection temporal data of aircraft targets obtained by our detection and tracking equipment or multi-source fusion temporal data. Assume The original data at time is representing longitude, representing latitude, representing altitude, representing eastward velocity, representing northward velocity, representing upward velocity, representing eastward acceleration, representing northward acceleration, representing upward acceleration, with a total of 9-dimensional features. Through the accumulation of the original data from time to time ( is called the time step), a continuous temporal data segment is formed, and its dimension is . In the training stage, the original data is used to generate temporal data segments as training samples according to the time step and the time interval , and can be Divide into a training set and a validation set according to the ratio.

[0040] (2) Construction of original time-series features: Input the original data into the original time-series data construction module for knowledge construction in the field of aircraft motion recognition, and further obtain features related to aircraft motion. See Figure 1 and Figure 2 for the original time-series feature construction module. Mainly construct the speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle of the aircraft, and augment them into the original time-series data.

[0041] Aircraft Speed scalar at time The construction method is:

[0042] Aircraft Acceleration scalar at time The construction method is:

[0043] Aircraft Track deviation angle at time The construction paradigm is:

[0044] The value range of the track deviation angle is , with the due north direction being 0 and north by east being positive.

[0045] Aircraft Aircraft track pitch angle at time The construction paradigm is:

[0046] The value range of the track pitch angle is , with the aircraft nose up being positive and the nose down being negative.

[0047] The reason for constructing the above four eigenvalue features is that the speed, acceleration, track deviation angle change rate (i.e., the ratio of the continuous track deviation angle difference to the time interval), and track pitch angle change rate (i.e., the ratio of the continuous track pitch angle difference to the time interval) of the aircraft can reflect the maneuverability of a real aircraft. If these features are added to the original time-series data, the subsequent selective feature fusion model can capture these feature patterns, which is conducive to the classification of basic aircraft actions. Add the above four features to the original time-series data to obtain a new feature vector :

[0048] To facilitate the training of the selective feature fusion model, it is also necessary to normalize this feature vector. Considering within the time range of the time step and using the min-max normalization principle for processing, then the normalized feature vector at time

[0049] is: where represents the feature data at time after normalization; and

[0050] are functions for finding the minimum and maximum values respectively. Finally, the feature vectors of time steps are

[0051] accumulated to form the input features of the selective feature fusion model: In addition, the construction of the original time series features also includes quantifying the basic action categories of the aircraft at the time step to form the quantization label

[0052] where 0 represents corresponding to the basic action "going straight"; 1 represents corresponding to the basic action "turning"; 2 represents corresponding to the basic action "climbing"; 3 represents corresponding to the basic action "diving"; (3) Construction of the selective feature fusion model: Use Pytorch to build the selective feature fusion model, and its network structure is shown in Figure 1 This selective feature fusion model takes the output of the original time series feature construction module and the quantization label as the network input. Specifically, the network structure consists of five parts: a time series feature non-linear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier. The time series feature non-linear enhancement module expands the original input from to dimensions, where represents the batch size; the global feature extraction module takes the output features of the time series feature non-linear enhancement module as input features, deeply extracts the global feature patterns related to the input features within all time steps, and compresses the features to , thus automatically screening and obtaining the important information therein; the multi-scale feature extraction module also takes the output features of the temporal feature non-linear enhancement module as input features, deeply extracts the local feature patterns in different scale spaces, and performs multi-scale feature splicing and fusion within a small range. Through convolution to adjust the feature dimension to , thus obtaining enhanced detailed feature information; the global and local feature fusion module is to splice the global features of with the local features of , and through two linear projection scalings for deep fusion to , further enhancing the feature expression ability; the feature classifier maps the deeply fused features to the probability representation of the basic aircraft action categories, and the index of the maximum probability represents the category.

[0053] Among them, the processing of the second-order discrete state space model in the global feature extraction module mainly relies on formulas (7), (8), (9), (10), and (11) for processing.

[0054] In formula (7), (at this time is the result after layer normalization of the global feature extraction module) maps from to , is the model parameter.

[0055] In formula (8), maps from to , is the model parameter.

[0056] In formula (9), maps from to , is the model parameter. To maintain dimensional consistency, let , collectively referred to as the dimension of the second-order discrete state space.

[0057] In formulas (10) and (11), is called the hidden layer state of the second-order discrete state space, and its dimension is , which is a variable that needs to be parameterized and can be initialized to 0; , , , are all model initialization parameters and can be parameterized using the Parameter function in Pytorch. Since , represent the noise vectors of the state equation and the observation equation respectively, and the parameters can be initialized by a normal distribution with a mean of 0 and a variance of 1e-6.

[0058] When processing the second-order discrete state space model, the features are processed step by step in time through formulas (10) and (11), where is the time step index. Stacking the outputs of all time steps together gives the output of the second-order discrete state space model processing, and its output dimension is . .

[0059] (4) Training stage: In the training stage, the total loss function constructed by formulas (12), (13), (14) and (15) is adopted. The balance coefficients of the loss function are , , . The original data in the training stage can select the simulation time series data generated by various simulation environment platforms for aircraft targets, as well as a small amount of original detection time series data or multi-source fusion time series data of aircraft targets to enrich the sample space of the training stage. The batch size is set to 256, the number of learning rounds is 100, and the initial learning rate is set to 1e-2; the piecewise learning rate update strategy is adopted to update the learning rate corresponding to a certain round number in the training process, and the piecewise interval of the learning rate update is [15:30:60]; the Adam optimizer is adopted, and early stopping judgment is used. When the validation set loss does not decrease for 5 consecutive rounds, the training is stopped.

[0060] (5) Inference stage: In the inference stage, only the original data needs to be input into the original time series feature construction module to convert it into the input feature format required by the network structure, and the trained selectable feature fusion model parameters are loaded for end-to-end inference, and the classification and recognition results of the basic actions of the aircraft can be obtained. See Figure 2 . By comparing the classification and recognition results (i.e., the predicted labels) of the basic actions of the aircraft obtained by inference with the quantization labels (true labels), evaluation indicators such as recognition accuracy can be calculated.

[0061] (6) Aircraft basic motion recognition experiment: Using the simulation time-series data of various types of aircraft obtained by a certain simulation platform's simulated radar sensor detection and tracking as the dataset, a preliminary experiment on aircraft basic motion recognition was conducted on the method proposed in this application. The average accuracy of all basic motion recognition types was used for comparison, and the results are shown in Table 1. In Table 1, TCN represents the temporal convolutional neural network method, FCN1D represents the one-dimensional fully convolutional neural network method, LSTM represents the long short-term memory network method, and Transformer represents the Transformer method; A1 means that the global feature extraction module of this application uses the conventional zero-order hold state-space model discretization method, and the loss function only uses the ordinary cross-entropy loss function; A2 means that only the global feature extraction module of this application is used, and the loss function only uses the ordinary cross-entropy loss function; A3 means that the global feature extraction module and the multi-scale local feature extraction module of this application are fully used, and the loss function still uses the ordinary cross-entropy loss function; A4 means the complete method of this application. The preliminary experimental results show that, compared with the conventional single convolutional neural network method (TCN, FCN1D) or the temporal neural network method (LSTM), and even the advanced Transformer method, the average recognition accuracy of the method of this application has been improved to a certain extent, verifying the effectiveness of this application.

[0062] Table 1 Aircraft basic motion recognition experiment

[0063] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and not to limit them. Although this application has been described in detail with reference to the above embodiments, those of ordinary skill in the art should understand that: it is still possible to modify the specific implementation manners of this application or make equivalent replacements, and any modification or equivalent replacement that does not depart from the spirit and scope of this application shall be covered by the protection scope of the claims of this application.

Claims

1. An aircraft basic motion recognition method based on a selective feature fusion model, characterized in that Including: Step 1: Input the original time-series data into the original time-series feature construction module, obtain the features related to the aircraft actions and augment them into the original time-series data to get new time-series data, and based on the new time-series data, obtain the input features of the selective feature fusion model and its corresponding aircraft basic action category label Y; the features related to the aircraft actions include the aircraft speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle; The aircraft basic motion category label Y is used to indicate the aircraft basic motion category; the aircraft basic motion category includes straight flight, turning, climbing, and diving; the original time-series data includes time-series data simulated and generated by various simulation environment platforms, the original detection time-series data of the aircraft target detected by real sensors, and the multi-source fusion time-series data of the aircraft target obtained by multi-source fusion means; Step 2: Construct a selective feature fusion model, which is composed of a time-series feature non-linear enhancement module, a global feature extraction module, a multi-scale local feature extraction module, a global and local feature fusion module, and a feature classifier; the time-series feature non-linear enhancement module is respectively connected to the global and local feature fusion module through the global feature extraction module and the multi-scale local feature extraction module, and the global and local feature fusion module is connected to the feature classifier; Step 3: Train the selective feature fusion model by constructing a total loss function to obtain the trained selective feature fusion model and its parameters; Step 4: Just input the original data into the original time-series feature construction module to convert it into the input feature format required by the selective feature fusion model, load the trained selective feature fusion model and its parameters, and perform end-to-end inference to obtain the recognition result of the aircraft basic motion; The recognition result of the aircraft basic motion is one of the aircraft basic motion categories; the original data includes time-series data simulated and generated by various simulation environment platforms, the original detection time-series data of the aircraft target detected by real sensors, and the multi-source fusion time-series data of the aircraft target obtained by multi-source fusion means.

2. The method according to claim 1, wherein The said Step 1 includes: Hypothesis The original time-series data at a moment is , where represents longitude,[[]] represents latitude,[[]] represents altitude,[[]] represents eastward velocity,[[]] represents northward velocity,[[]] represents upward velocity,[[]] represents eastward acceleration,[[]] represents northward acceleration,[[]] represents upward acceleration; Input the original time series data into the original time series data construction module to obtain the scalar velocity at the moment of the aircraft as follows: Obtain the aircraft The scalar acceleration at a moment is as follows: Obtain the aircraft Track deviation angle at a certain moment is as follows: The range of the track deflection angle is , with the due north direction being 0 and the direction east of north being positive; Aircraft Pitch angle of the aircraft flight path at a certain moment is as follows: The value range of the track pitch angle is , with the aircraft nose up being positive and the nose down being negative; Add the aircraft's speed scalar, acceleration scalar, aircraft track deviation angle, and aircraft track pitch angle to the original time-series data , to obtain new time-series data : Normalize the new time series data to obtain: Among them, represents the feature data at the normalized moment; and are functions for finding the minimum value and the maximum value respectively, is the time step; Accumulate the feature vectors of each time step to form the input features of the selective feature fusion model : Quantify the basic motion categories of the aircraft with a time step to obtain the basic motion category labels of the aircraft as follows: Among them, 0 means The corresponding basic aircraft movement category is "going straight"; 1 means The corresponding basic aircraft movement category is "turning"; 2 means The corresponding basic aircraft movement category is "climbing"; 3 means The corresponding basic aircraft movement category is "diving".

3. The method according to claim 1, wherein In the said Step 2, the global feature extraction module includes two branches, a linear projection layer, and a global average pooling layer, and the outputs of the two branches are connected to the global average pooling layer through the linear projection layer; One of the two branches linearly projects the input features and their corresponding aircraft basic action category labels Y in sequence, performs layer normalization, and processes them with a second-order discrete state space model. The other branch linearly projects the input features and their corresponding aircraft basic action category labels Y in sequence, performs a SiLU activation, then multiplies the features output by the two branches, and linearly projects the multiplication result to restore it to the dimension of the input features Finally, the final global feature is obtained through global average pooling; Input feature It is the output of the time series feature non-linear enhancement module.

4. The method according to claim 3, characterized in that The second-order discrete state space model is processed to obtain an output by processing the features output after layer normalization through the second-order discrete state space model for processing to obtain an output : The process of constructing the second-order discrete state space model is: Introduce the state space model of the stochastic system, and the state space model includes a state equation and an observation equation: (1) (2) Among them, is the state vector at time t, is the differential form of the current state; is the deterministic input at time t; is the output at time t; is the state transition matrix, used to control the change of the state over time; is the input matrix, used to control the influence of the input on the state; is the output matrix, representing the output generated based on the current state; is the feedforward matrix, representing the direct influence of the input on the output; is used to indicate that the stochastic system is a continuous-time system; Equation (1) is the state equation and Equation (2) is the observation equation; Approximate formulas (1) and (2) by discretization and convert them into: (3) (4) Among them, is the hidden state at the k-th time step of the second-order discrete state space model, is the hidden state at the (k - 1)-th time step of the second-order discrete state space model, the deterministic input at the k-th time step, and represents the discrete interval index, i.e., the time step index; is the output at the k-th time step of the second-order discrete state space model; 、 are the new state transition matrix and input matrix formed by discretization, mainly obtained by discretizing the differential equations of formulas (3) and (4) through Taylor series expansion to obtain the matrices 、 ; Expand the state equation of formula (3) by Taylor series, retain the quadratic term approximation, the new state transition matrix and the new input matrix are in the form of: (5) (6) Among them, the parameter is the parameter introduced by discretization; is the identity matrix; and are the state transition matrix and the input matrix respectively, both of which are parameters that need to be initialized by the network; is the high-order term; Model and as a function of feature using the following formula: (7) (8) Among them, is the softplus function, to ensure provides non-linear control ability; , , , are linearization parameters; To enable the output of the second-order discrete state space model to follow the flow of the feature control state, the output matrix in the observation equation of Equation (4) is modeled as a function of the feature: (9) Among them, and are both linearization parameters; Substitute Formulas (5) to (8) into Formula (3), regard the feature as a deterministic input, and at the same time substitute Formula (9) into Formula (4), and introduce the noise vector of the state equation and the noise vector of the observation equation, then the second-order discrete state space model is obtained as follows: (10) (11) Among them, the newly added parameters and are used to balance the errors in the discretization of the state space model.

5. The method according to claim 1, wherein The multi-scale local feature extraction module includes three feature extraction branches, a feature splicing layer, and a convolutional layer; each feature extraction branch in the three feature extraction branches includes a linear projection layer, a causal convolutional layer, a batch normalization layer, a ReLU activation layer, and a global average pooling layer connected in sequence; the global average pooling layer of each feature extraction branch in the three feature extraction branches is connected to the convolutional layer through the feature splicing layer; the convolutional kernel sizes used in the causal convolutional layers of each feature extraction branch in the three feature extraction branches are different; Each feature extraction branch uses a linear projection layer for the input features to perform non-linear feature enhancement. Then, each feature extraction branch uses convolution kernels of different sizes to extract local feature patterns within different receptive fields, followed by batch normalization. After batch normalization, the features are connected with the output of the linear projection layer through residual connection and then ReLU activation is performed. Global average pooling is used to average the features of different channels, compressing the feature information and only retaining the most critical local feature patterns. Finally, the outputs of all global average pooling in the three feature extraction branches are concatenated for feature fusion at multiple scales, and a convolutional layer is used to adjust the dimension of the features after multi-scale feature fusion to the dimension output by the global feature extraction module.

6. The method according to claim 1, wherein The said global and local feature fusion module splices the global feature output by the global feature extraction module and the local feature output by the multi-scale local feature extraction module, and performs fusion through two linear projection scalings to obtain the selective fusion feature.

7. The method according to claim 1, wherein The feature classifier maps the selective fusion feature output by the global and local feature fusion module to the probability representation of the aircraft basic motion category, and the index of the maximum probability represents the aircraft basic motion category.

8. The method according to claim 1, characterized in that, The said Step 3 includes: In the training stage, the original data is used as training samples, and the selective feature fusion model is trained using the total loss function, and the balance coefficient of the total loss function is set; the batch size, the number of learning epochs, and the initial learning rate are set, and the piecewise learning rate update strategy is adopted to update the learning rate corresponding to a certain number of epochs in the training process, and the piecewise interval for learning rate update is set; the Adam optimizer is used to optimize the selective feature fusion model, and early stopping judgment is adopted, that is, when the validation set loss does not decrease for multiple consecutive epochs, the training is stopped, and the parameters of the trained selective feature fusion model are obtained.

9. The method according to claim 8, characterized in that The constructed total loss function is: (12) Among them, is the total loss function; is the cross-entropy loss for aircraft basic action category classification; is the loss of the distance between the clustering centers of training samples with the same predicted label; is the exponential distance loss between the selectable fusion features of training samples with different predicted labels; and serve as compensation terms for the cross-entropy loss function; , , are , and are the balance coefficients between; the selectable fusion features are the features output by the global and local feature fusion module.

10. The method according to claim 9, characterized in that, , and The expressions of are as follows: (13) (14) (15) Among them, is the number of aircraft target time-series data samples, and the aircraft target time-series data samples are the original data; is the number of types of basic aircraft actions; is the th training sample, and is the true label of the basic action category is the th training sample predicted as the basic action category and is the prediction probability; is the selectable fusion feature of the th training sample; is the selectable fusion feature of the th training sample; represents the number of training samples with the label predicted as the basic action category , and the basic action category belongs to the aircraft basic action category label Y, and its value is 0, 1, 2, or 3; represents the selectable fusion feature of the th training sample among the training samples with the label predicted as the basic action category ; is the clustering center of the selectable fusion features of all training samples with the predicted category represents the exponential function.

Images