A flight action recognition method and device based on bidirectional recurrent neural network

Abstract

The present invention discloses a flight action recognition method and device based on a bidirectional recurrent neural network, which belongs to the field of flight data intelligent processing technology, including: collecting real-time operation data in flight training, and performing maneuvering action segmentation preprocessing to obtain a real-time operation data set; bringing the real-time operation data set into the best bidirectional recurrent neural network model obtained by training, and using the best bidirectional recurrent neural network model to perform basic flight action recognition modeling. The present application can correctly identify basic and complex actions without relying on knowledge, which is consistent with the era of "big data + big computing power + big model" that artificial intelligence has entered.

Description

Technical Field

[0001] The invention discloses a flight action recognition method and device based on a bidirectional recurrent neural network, belonging to the technical field of flight data intelligent processing. Background Art

[0002] During flight training, completing the flight maneuvers assigned by the instructor is a very important training subject for pilots. The quality of completion will directly affect the evaluation of the pilot's flight control quality. Therefore, it is necessary to identify the flight maneuvers of the aircraft in order to evaluate the pilot's flight control quality. In the prior art, flight training maneuver recognition can be divided into two methods: knowledge rule and data driven, depending on the different recognition methods. Among them:

[0003] Flight action recognition based on knowledge rules refers to a domain expert system constructed based on artificial prior knowledge. It extracts flight data features and knowledge into a series of production rules, constructs a flight action recognition knowledge base, and then uses data mining technology or reasoning methods to complete the comparison between the original flight data and the rules, thereby realizing the recognition of flight actions. However, this method uses a rule base of up to 1,200 items, and the rule design is complex and difficult to verify. It cannot be fully applicable to all aircraft models. Common flight action deviations lead to recognition errors, and the overlapping and cross-transition parts of flight actions require additional processing.

[0004] In recent years, with the rapid development of artificial intelligence, many data-driven action recognition methods have emerged. Support vector machines can realize the recognition of flight actions and avoid the influence of subjective factors, but the calculation speed is slow in the case of large samples. The recognition method based on hidden Markov model has high accuracy, but the method is complex in calculation and takes a long time to recognize. The neural network-based method uses labeled data to train and recognize the network model, with high accuracy, but the structure and parameters of the neural network model have a great influence on the recognition results, so it is necessary to design an appropriate network model structure and suitable network model parameters. Summary of the invention

[0005] In view of the defects of the prior art, the present invention proposes a flight action recognition method and device based on a bidirectional recurrent neural network to solve the problem that the structure and parameters of the current neural network model have a significant impact on the recognition results.

[0006] The technical solution of the present invention is as follows:

[0007] According to a first aspect of an embodiment of the present invention, a flight action recognition method based on a bidirectional recurrent neural network is provided, comprising:

[0008] Collect real-time operation data in flight training and perform maneuver segmentation preprocessing to obtain real-time operation data set;

[0009] The real-time running data set is introduced into the best bidirectional recurrent neural network model obtained through training, and the best bidirectional recurrent neural network model is used to perform basic flight action recognition modeling.

[0010] Preferably, before collecting the real-time operation data in the flight training and performing maneuver segmentation preprocessing on the real-time operation data set to obtain the real-time operation data set, the method further includes:

[0011] Obtain historical operation data collected during flight training, and perform preprocessing and action annotation on it to obtain a flight training action dataset;

[0012] A bidirectional recurrent neural network model for flight action recognition is constructed, and the bidirectional recurrent neural network model is trained using the flight training action data set to obtain an optimal bidirectional recurrent neural network model.

[0013] Preferably, the acquiring of historical operation data collected during flight training, and preprocessing and action labeling thereof to obtain a flight training action data set includes:

[0014] Acquiring historical operation data collected during the flight training, wherein the historical operation data collected during the flight training includes at least: engine data, flight track data, flight attitude data, and air environment data;

[0015] The historical operation data collected during the flight training is subjected to a preprocessing operation of retaining valid information in the original data, compensating for defects in the data, and eliminating invalid data to obtain a highly available flight training data set;

[0016] The highly available flight training data set is annotated with flight actions according to expert interpretation rules to obtain a flight training action data set.

[0017] Preferably, the expert interpretation rules at least include: a bucket action, a half bucket action and a half roll inversion action, wherein:

[0018] The dogfighting maneuvers include: an entry speed greater than 620 kilometers per hour, a maximum pitch angle greater than 65 degrees, an altitude difference greater than 2,000 meters, an entry and exit altitude difference less than 800 meters, a heading angle change less than 40 degrees, and an entry speed greater than an exit speed;

[0019] The half-pound action includes: the entry speed is greater than the exit speed, the maximum pitch angle is greater than 65 degrees, the heading change is less than 160 degrees, the height difference is greater than 2000 meters, the sideslip angle is greater than 70 degrees, the minimum pitch angle is greater than 40 degrees and the duration is less than 30 seconds;

[0020] The half-roll inversion action includes: the difference between the exit altitude and the entry altitude is less than 500 meters, the minimum pitch angle is less than 70 degrees, the minimum pitch angle is less than 40 degrees, the change in heading angle from entry to exit is greater than 100 degrees and the duration is less than 30 seconds.

[0021] Preferably, the flight training action data set includes: all bucket data sets, all half bucket data sets, all half roll inversion data sets and other data sets.

[0022] Preferably, the collecting of real-time operation data in flight training and performing maneuver segmentation preprocessing on the real-time operation data set to obtain the real-time operation data set includes:

[0023] Collecting real-time operation data during the flight training;

[0024] Determine the width and step size of the sliding window, and flatten the real-time operation data in the flight training into real-time operation action sequence data in the flight training;

[0025] The sliding window is used to segment the real-time operation action sequence data in the flight training to obtain a plurality of initial flight training action subsequences, and a real-time operation data set is constructed according to the plurality of initial flight training action subsequences.

[0026] Preferably, determining the width and step size of the sliding window includes:

[0027] The width and step size of the sliding window are determined according to boundary conditions, wherein the boundary conditions include:

[0028] The absolute value of the pitch angle is greater than 20 for 10 consecutive times and the number is greater than 2, and the pitch angle is greater than 10, cutting starts; the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and the number is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and the number is greater than or equal to 4, cutting ends;

[0029] Cutting starts when the absolute value of the tilt angle is greater than 2 for 5 consecutive times and is greater than or equal to 4. Cutting ends when the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and is greater than or equal to 4.

[0030] According to a second aspect of an embodiment of the present invention, there is provided a flight action recognition device based on a bidirectional recurrent neural network, comprising:

[0031] The data segmentation module is used to collect real-time operation data in flight training and perform maneuver segmentation preprocessing on it to obtain a real-time operation data set;

[0032] The recognition modeling module is used to bring the real-time operation data set into the best bidirectional recurrent neural network model obtained by training, and use the best bidirectional recurrent neural network model to perform basic flight action recognition modeling.

[0033] According to a third aspect of an embodiment of the present invention, a terminal is provided, including:

[0034] one or more processors;

[0035] a memory for storing said one or more processor executable instructions;

[0036] Wherein, the one or more processors are configured to:

[0037] Execute the method described in the first aspect of the embodiment of the present invention.

[0038] According to a fourth aspect of an embodiment of the present invention, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the method described in the first aspect of the embodiment of the present invention.

[0039] According to a fifth aspect of the embodiments of the present invention, an application product is provided. When the application product is running on a terminal, the terminal executes the method described in the first aspect of the embodiments of the present invention.

[0040] The beneficial effects of the present invention are:

[0041] The present invention provides a flight action recognition method and device based on a bidirectional recurrent neural network. The method and device perform preprocessing and action annotation on historical operation data collected in flight training to construct a training data set, improve the reliability of the experiment, and construct a bidirectional recurrent neural network model for flight action recognition. The bidirectional recurrent neural network model is trained using the flight training action data set to obtain an optimal bidirectional recurrent neural network model. Real-time operation data in flight training is collected, and maneuvering action segmentation preprocessing is performed. The real-time operation data in flight training after the maneuvering action segmentation preprocessing is brought into the training to obtain the optimal bidirectional recurrent neural network model. The optimal neural network model is used to perform basic flight action recognition modeling. Basic actions and complex actions can be correctly identified without relying on knowledge, which is consistent with the opportunity of the "big data + big computing power + big model" era that artificial intelligence is entering.

[0042] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1The present invention is a flowchart of a method for identifying flight actions based on a bidirectional recurrent neural network according to an exemplary embodiment.

[0044] Figure 2 The present invention is a flowchart of a method for identifying flight actions based on a bidirectional recurrent neural network according to an exemplary embodiment.

[0045] Figure 3 It is a schematic block diagram of the structure of a flight action recognition device based on a bidirectional recurrent neural network according to an exemplary embodiment.

[0046] Figure 4 The present invention is a schematic block diagram of a terminal structure according to an exemplary embodiment. DETAILED DESCRIPTION

[0047] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0049] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0050] An embodiment of the present invention provides a flight action recognition method based on a bidirectional recurrent neural network. The method is implemented by a terminal. The terminal may be a desktop computer or a laptop computer, etc. The terminal at least includes a CPU, etc. Example 1

[0051] Figure 1The present invention is a flowchart of a method for identifying flight actions based on a bidirectional recurrent neural network according to an exemplary embodiment. The method is used in a terminal and includes the following steps:

[0052] Step 101, collecting real-time operation data in flight training, and performing maneuver segmentation preprocessing on the data to obtain a real-time operation data set;

[0053] Step 102: The real-time running data set is introduced into the best bidirectional recurrent neural network model obtained through training, and basic flight action recognition modeling is performed using the best bidirectional recurrent neural network model.

[0054] Preferably, before collecting the real-time operation data in the flight training and performing maneuver segmentation preprocessing on the real-time operation data set to obtain the real-time operation data set, the method further includes:

[0055] Obtain historical operation data collected during flight training, and perform preprocessing and action annotation on it to obtain a flight training action dataset;

[0056] A bidirectional recurrent neural network model for flight action recognition is constructed, and the bidirectional recurrent neural network model is trained using the flight training action data set to obtain an optimal bidirectional recurrent neural network model.

[0057] Preferably, the acquiring of historical operation data collected during flight training, and preprocessing and action labeling thereof to obtain a flight training action data set includes:

[0058] Acquiring historical operation data collected during the flight training, wherein the historical operation data collected during the flight training includes at least: engine data, flight track data, flight attitude data, and air environment data;

[0059] The historical operation data collected during the flight training is subjected to a preprocessing operation of retaining valid information in the original data, compensating for defects in the data, and eliminating invalid data to obtain a highly available flight training data set;

[0060] The highly available flight training data set is annotated with flight actions according to expert interpretation rules to obtain a flight training action data set.

[0061] Preferably, the expert interpretation rules at least include: a bucket action, a half bucket action and a half roll inversion action, wherein:

[0062] The dogfighting maneuvers include: an entry speed greater than 620 kilometers per hour, a maximum pitch angle greater than 65 degrees, an altitude difference greater than 2,000 meters, an entry and exit altitude difference less than 800 meters, a heading angle change less than 40 degrees, and an entry speed greater than an exit speed;

[0063] The half-pound action includes: the entry speed is greater than the exit speed, the maximum pitch angle is greater than 65 degrees, the heading change is less than 160 degrees, the height difference is greater than 2000 meters, the sideslip angle is greater than 70 degrees, the minimum pitch angle is greater than 40 degrees and the duration is less than 30 seconds;

[0064] The half-roll inversion action includes: the difference between the exit altitude and the entry altitude is less than 500 meters, the minimum pitch angle is less than 70 degrees, the minimum pitch angle is less than 40 degrees, the change in heading angle from entry to exit is greater than 100 degrees and the duration is less than 30 seconds.

[0065] Preferably, the flight training action data set includes: all bucket data sets, all half bucket data sets, all half roll inversion data sets and other data sets.

[0066] Preferably, the collecting of real-time operation data in flight training and performing maneuver segmentation preprocessing on the real-time operation data set to obtain the real-time operation data set includes:

[0067] Collecting real-time operation data during the flight training;

[0068] Determine the width and step size of the sliding window, and flatten the real-time operation data in the flight training into real-time operation action sequence data in the flight training;

[0069] The sliding window is used to segment the real-time operation action sequence data in the flight training to obtain a plurality of initial flight training action subsequences, and a real-time operation data set is constructed according to the plurality of initial flight training action subsequences.

[0070] Preferably, determining the width and step size of the sliding window includes:

[0071] The width and step size of the sliding window are determined according to boundary conditions, wherein the boundary conditions include:

[0072] The absolute value of the pitch angle is greater than 20 for 10 consecutive times and the number is greater than 2, and the pitch angle is greater than 10, cutting starts; the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and the number is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and the number is greater than or equal to 4, cutting ends;

[0073] Cutting starts when the absolute value of the tilt angle is greater than 2 for 5 consecutive times and is greater than or equal to 4. Cutting ends when the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and is greater than or equal to 4. Example 2

[0074] Figure 2The present invention is a flowchart of a method for identifying flight actions based on a bidirectional recurrent neural network according to an exemplary embodiment. The method is used in a terminal and includes the following steps:

[0075] Step 201, obtain the historical operation data collected in the flight training, and perform preprocessing and action annotation on it to obtain the flight training action data set, the specific content is as follows:

[0076] The historical operation data collected during flight training is obtained. The historical operation data collected during flight training includes at least: engine data, flight track data, flight attitude data and air environment data. However, the historical operation data collected during the original flight training often generates invalid data due to the instability of sensor instruments, sudden changes in the data generation environment and external interference. Such invalid data will seriously affect related experimental research. Therefore, the historical operation data collected during flight training must be preprocessed to retain the valid information in the original data, make up for the defects in the data and eliminate invalid data to obtain a highly available flight training data set.

[0077] The flight training action dataset is obtained by annotating the flight action of the highly available flight training dataset according to the expert interpretation rules. The flight training action dataset includes: all bucket datasets, all half bucket datasets, all half roll datasets and other datasets.

[0078] The rules for expert interpretation include at least the following: a cartwheel, a half cartwheel and a half roll, among which:

[0079] The dogfighting maneuvers include: an entry speed greater than 620 kilometers per hour, a maximum pitch angle greater than 65 degrees, an altitude difference greater than 2,000 meters, an entry and exit altitude difference less than 800 meters, a heading angle change less than 40 degrees, and an entry speed greater than an exit speed;

[0080] The half-pound action includes: the entry speed is greater than the exit speed, the maximum pitch angle is greater than 65 degrees, the heading change is less than 160 degrees, the height difference is greater than 2000 meters, the sideslip angle is greater than 70 degrees, the minimum pitch angle is greater than 40 degrees and the duration is less than 30 seconds;

[0081] The half-roll inversion action includes: the difference between the exit altitude and the entry altitude is less than 500 meters, the minimum pitch angle is less than 70 degrees, the minimum pitch angle is less than 40 degrees, the change in heading angle from entry to exit is greater than 100 degrees and the duration is less than 30 seconds.

[0082] Step 202, constructing a bidirectional recurrent neural network model for flight action recognition, using the flight training action data set to train the bidirectional recurrent neural network model to obtain an optimal bidirectional recurrent neural network model, the specific contents are as follows:

[0083] Construct a bidirectional recurrent neural network model for flight action recognition. The bidirectional recurrent neural network model contains an input layer, an output layer, and three LSTM layers. The model is constructed using fully connected layers. All nodes between each layer and the next layer are fully connected, and the output of each layer is the input feature of the next layer. The main feature of the bidirectional recurrent neural network is that two LSTM layers are connected to the output layer, which contains 6 shared weights. The model uses a loss function to measure the gap between the output and the predicted value, and uses this as a feedback signal to fine-tune the weights to reduce the loss value. The weights and biases are randomly generated at the beginning, and the loss value is high, but as the number of training and iterations increases, the weight values ​​are gradually fine-tuned in the right direction, and the loss is gradually decreasing. Therefore, some parameters in building the network need to be selected:

[0084] 1. Optimizer: It mainly solves the problem that the parameters in neural networks are not always convex functions, but complex functions. The gradient descent process will have a local minimum point. Therefore, the gradient descent function includes stochastic gradient descent, small batch stochastic gradient descent, momentum SGD, weighted average calculation of second-order momentum, and Adam, which is the culmination of multiple optimization ideas. This embodiment selects the Adam optimization algorithm, which integrates the advantages of SGD and RMSProp, and is also a method for different parameters to adapt to different learning rates.

[0085] 2. Activation function: In order to obtain a richer hypothesis space and thus show the advantages of multi-layer representation, it is necessary to use activation functions to introduce nonlinear factors to each neuron, so that the neural network can arbitrarily approximate any nonlinear function and form various nonlinear models with stronger fitting capabilities. Activation functions include sigmiod function, Tanh function, ReLU function and Softmax function. This model selects the ReLU activation function, which has a fast convergence speed for stochastic gradient descent and does not have the gradient vanishing phenomenon.

[0086] 3. Loss function selection: The loss function not only plays an important role in model optimization, but also belongs to the parameters of the entire neural network. The choice of loss function in the neural network depends on the type of problem. The guiding principles are as follows:

[0087] For the regression problem of continuous value vector, the mean square error loss function MSE is used; for the binary classification problem, the binary cross entropy loss function is used; for the multi-classification problem, if the output is one-hot encoded, the categorical cross entropy loss function is used, and if the output is an integer value, the sparse categorical cross entropy loss function is used. This embodiment belongs to multi-classification and the output is an integer value, so the sparse categorical cross entropy loss function is selected.

[0088] Selection of evaluation indicators: that is, to evaluate whether the network model is good or not. This indicator is somewhat similar to the loss function, both of which pursue the minimum error between the true value and the predicted value. The difference is that the loss function acts on the training set to train the machine and provide direction for gradient descent; while the evaluation indicator acts on the validation set and the test set to evaluate the model. For classification problems, the neural network uses precision as the evaluation indicator by default, which is the ratio of the number of samples with relatively accurate measurement to the total number of samples.

[0089] The bidirectional recurrent neural network model is trained using the flight training action dataset to obtain the optimal bidirectional recurrent neural network model.

[0090] Step 203, collect the real-time operation data in the flight training, and perform maneuver segmentation preprocessing on it to obtain the real-time operation data set, the specific content is as follows:

[0091] Real-time operation data during flight training is collected, and the width and step size of the sliding window are determined according to the boundary conditions. The boundary conditions include: the number of the absolute value of the pitch angle being greater than 20 for 10 consecutive values ​​is greater than 2, and the cutting starts when the pitch angle is greater than 10, and the number of the absolute value of the bank angle being less than 3.5 for 6 consecutive values ​​is greater than or equal to 4, and the cutting ends when the absolute value of the pitch angle is less than 6 for 6 consecutive values ​​is greater than or equal to 4; the cutting starts when the absolute value of the bank angle is greater than 2 for 5 consecutive values ​​is greater than or equal to 4, and the cutting ends when the absolute value of the bank angle is less than 3.5 for 6 consecutive values ​​is greater than or equal to 4.

[0092] The real-time operation data in flight training are flattened into a multi-dimensional real-time operation action sequence data in flight training in chronological order. The real-time operation action sequence data in flight training are segmented using a sliding window to obtain multiple initial flight training action subsequences, and a real-time operation data set is constructed based on the multiple initial flight training action subsequences.

[0093] Step 204: bring the real-time running data set into the best bidirectional recurrent neural network model obtained through training, and use the best bidirectional recurrent neural network model to perform basic flight action recognition modeling. Example 3

[0094] Figure 3 The present invention is a schematic block diagram of a flying action recognition device based on a bidirectional recurrent neural network according to an exemplary embodiment, wherein the device comprises:

[0095] The data segmentation module 310 is used to collect real-time operation data in flight training and perform maneuver segmentation preprocessing on the data to obtain a real-time operation data set;

[0096] The recognition modeling module 320 is used to bring the real-time operation data set into the trained optimal bidirectional recurrent neural network model, and use the optimal bidirectional recurrent neural network model to perform basic flight action recognition modeling.

[0097] This application uses historical operation data collected during flight training to perform preprocessing and action annotation to construct a training data set, improve the reliability of the experiment, construct a bidirectional recurrent neural network model for flight action recognition, use the flight training action data set to train the bidirectional recurrent neural network model to obtain the optimal bidirectional recurrent neural network model, collect real-time operation data during flight training, and perform maneuvering action segmentation preprocessing; the real-time operation data during flight training after the maneuvering action segmentation preprocessing is brought into the training to obtain the optimal bidirectional recurrent neural network model, and the optimal neural network model is used to perform basic flight action recognition modeling, which can correctly identify basic and complex actions without relying on knowledge, which is in line with the current opportunity of artificial intelligence entering the "big data + big computing power + big model" era. Example 4

[0098] Figure 4 1 is a block diagram of a terminal provided in an embodiment of the present application, and the terminal may be a terminal in the above embodiment. The terminal may be a portable mobile terminal, such as a smart phone or a tablet computer. The terminal may also be referred to as a user device, a portable terminal, or other names.

[0099] Typically, the terminal includes: a processor 401 and a memory 402 .

[0100] The processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 401 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 401 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0101] The memory 402 may include one or more computer-readable storage media, which may be tangible and non-transitory. The memory 402 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is used to be executed by the processor 401 to implement a flight action recognition method based on a bidirectional recurrent neural network provided in the present application.

[0102] In some embodiments, the terminal may further include: a peripheral device interface 403 and at least one peripheral device. Specifically, the peripheral device includes: at least one of a radio frequency circuit 404 , a display screen 405 , a camera component 406 , an audio circuit 407 , a positioning component 408 and a power supply 409 .

[0103] The peripheral device interface 403 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 401 and the memory 402. In some embodiments, the processor 401, the memory 402, and the peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 401, the memory 402, and the peripheral device interface 403 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0104] The radio frequency circuit 404 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 404 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 404 converts the electrical signal into an electromagnetic signal for transmission, or converts the received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 404 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 404 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0105] The display screen 405 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. The display screen 405 also has the ability to collect touch signals on the surface or above the surface of the display screen 405. The touch signal can be input to the processor 401 as a control signal for processing. The display screen 405 is used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 405 can be one, and the front panel of the terminal is set; in other embodiments, the display screen 405 can be at least two, which are respectively set on different surfaces of the terminal or in a folding design; in some other embodiments, the display screen 405 can be a flexible display screen, which is set on the curved surface or folding surface of the terminal. Even, the display screen 405 can also be set as a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 405 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0106] The camera assembly 406 is used to capture images or videos. Optionally, the camera assembly 406 includes a front camera and a rear camera. Typically, the front camera is used to realize video calls or selfies, and the rear camera is used to realize photo or video shooting. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, and a wide-angle camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, and the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting functions. In some embodiments, the camera assembly 406 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0107] The audio circuit 407 is used to provide an audio interface between the user and the terminal. The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 401 for processing, or input them into the radio frequency circuit 404 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 407 may also include a headphone jack.

[0108] The positioning component 408 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service).

[0109] The power supply 409 is used to power various components in the terminal. The power supply 409 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 409 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery charged through a wired line, and a wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0110] In some embodiments, the terminal further includes one or more sensors 410 , including but not limited to: an acceleration sensor 411 , a gyroscope sensor 412 , a pressure sensor 413 , a fingerprint sensor 414 , an optical sensor 415 , and a proximity sensor 416 .

[0111] The acceleration sensor 411 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 411 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 401 can control the display screen 405 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 411. The acceleration sensor 411 can also be used to collect game or user motion data.

[0112] The gyroscope sensor 412 can detect the body direction and rotation angle of the terminal, and the gyroscope sensor 412 can cooperate with the acceleration sensor 411 to collect the user's 3D (3 Dimensions) action on the terminal. The processor 401 can implement the following functions based on the data collected by the gyroscope sensor 412: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0113] The pressure sensor 413 can be set in the side frame of the terminal and / or the lower layer of the display screen 405. When the pressure sensor 413 is set in the side frame of the terminal, it can detect the user's holding signal of the terminal, and perform left and right hand recognition or shortcut operation according to the holding signal. When the pressure sensor 413 is set in the lower layer of the display screen 405, the operability control on the UI interface can be controlled according to the user's pressure operation on the display screen 405. The operability control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0114] The fingerprint sensor 414 is used to collect the user's fingerprint to identify the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 401 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 414 can be set on the front, back, or side of the terminal. When a physical button or manufacturer logo is set on the terminal, the fingerprint sensor 414 can be integrated with the physical button or manufacturer logo.

[0115] The optical sensor 415 is used to collect the ambient light intensity. In one embodiment, the processor 401 can control the display brightness of the display screen 405 according to the ambient light intensity collected by the optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the display screen 405 is increased; when the ambient light intensity is low, the display brightness of the display screen 405 is decreased. In another embodiment, the processor 401 can also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 415.

[0116] The proximity sensor 416, also known as a distance sensor, is usually arranged on the front of the terminal. The proximity sensor 416 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 401 controls the display screen 405 to switch from the screen-on state to the screen-off state; when the proximity sensor 416 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 401 controls the display screen 405 to switch from the screen-off state to the screen-on state.

[0117] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than those shown in the figure, or combine certain components, or adopt a different component arrangement. Example 5

[0118] In an exemplary embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by a processor, a flight action recognition method based on a bidirectional recurrent neural network as provided in all the inventive embodiments of the present application is implemented.

[0119] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0120] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0121] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0122] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider). Example 6

[0123] In an exemplary embodiment, an application product is also provided, including one or more instructions, which can be executed by the processor 401 of the above-mentioned device to complete the above-mentioned flight action recognition method based on a bidirectional recurrent neural network.

[0124] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A flight action recognition method based on a bidirectional recurrent neural network, characterized in that: include: Collect real-time operation data in flight training and perform maneuver segmentation preprocessing to obtain real-time operation data set; Bringing the real-time running data set into the best bidirectional recurrent neural network model obtained through training, and using the best bidirectional recurrent neural network model to perform basic flight action recognition modeling; The real-time operation data in the flight training is collected and preprocessed by maneuvering action segmentation to obtain a real-time operation data set, including: Collecting real-time operation data during the flight training; Determine the width and step size of the sliding window, and flatten the real-time operation data in the flight training into real-time operation action sequence data in the flight training; Using the sliding window to segment the real-time operation action sequence data in the flight training to obtain a plurality of initial flight training action subsequences, and constructing a real-time operation data set according to the plurality of initial flight training action subsequences; Determining the width and step size of the sliding window includes: The width and step size of the sliding window are determined according to boundary conditions, wherein the boundary conditions include: The number of times the absolute value of the pitch angle is greater than 20 for 10 consecutive times is greater than or equal to 2, and the pitch angle is greater than 10, cutting starts; the number of times the absolute value of the tilt angle is less than 3.5 for 6 consecutive times is greater than or equal to 4, and the number of times the absolute value of the pitch angle is less than 6 for 6 consecutive times is greater than or equal to 4, cutting ends; Cutting starts when the absolute value of the tilt angle is greater than 2 for 5 consecutive times and is greater than or equal to 4. Cutting ends when the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and is greater than or equal to 4.

2. The flight action recognition method based on a bidirectional recurrent neural network according to claim 1 is characterized in that: Before collecting the real-time operation data in the flight training and performing maneuvering action segmentation preprocessing on the real-time operation data set to obtain the real-time operation data set, the method further includes: Obtain historical operation data collected during flight training, and perform preprocessing and action annotation on it to obtain a flight training action dataset; A bidirectional recurrent neural network model for flight action recognition is constructed, and the bidirectional recurrent neural network model is trained using the flight training action data set to obtain an optimal bidirectional recurrent neural network model.

3. The flight action recognition method based on a bidirectional recurrent neural network according to claim 2 is characterized in that: The process of obtaining the historical operation data collected during the flight training, and preprocessing and annotating the data to obtain the flight training action data set includes: Acquiring historical operation data collected during the flight training, wherein the historical operation data collected during the flight training includes: engine data, flight track data, flight attitude data, and air environment data; The historical operation data collected during the flight training is subjected to a preprocessing operation of retaining valid information in the original data, compensating for defects in the data, and eliminating invalid data to obtain a highly available flight training data set; The highly available flight training data set is annotated with flight actions according to expert interpretation rules to obtain a flight training action data set.

4. The flight action recognition method based on a bidirectional recurrent neural network according to claim 3 is characterized in that: The expert interpretation rules include: bucket action, half bucket action and half roll inversion action, among which: The dogfighting maneuvers include: an entry speed greater than 620 kilometers per hour, a maximum pitch angle greater than 65 degrees, an altitude difference greater than 2,000 meters, an entry and exit altitude difference less than 800 meters, a heading angle change less than 40 degrees, and an entry speed greater than an exit speed; The half-pound action includes: the entry speed is greater than the recovery speed, the maximum pitch angle is greater than 65 degrees, the heading change is less than 160 degrees, the altitude difference is greater than 2000 meters, the sideslip angle is greater than 70 degrees, the minimum pitch angle is greater than 40 degrees and the duration is less than 30 seconds; The half-roll inversion action includes: the difference between the exit altitude and the entry altitude is less than 500 meters, the minimum pitch angle is less than 70 degrees, the change in heading angle from entry to exit is greater than 100 degrees and the duration is less than 30 seconds.

5. The flight action recognition method based on a bidirectional recurrent neural network according to claim 3 is characterized in that: The flight training action data set includes: all bucket data sets, all half bucket data sets and all half roll inversion data sets.

6. A flight action recognition device based on a bidirectional recurrent neural network, characterized in that: include: The data segmentation module is used to collect real-time operation data in flight training and perform maneuver segmentation preprocessing on it to obtain a real-time operation data set; A recognition modeling module, used for bringing the real-time operation data set into the best bidirectional recurrent neural network model obtained by training, and using the best bidirectional recurrent neural network model to perform basic flight action recognition modeling; The real-time operation data in the flight training is collected and preprocessed by maneuvering action segmentation to obtain a real-time operation data set, including: Collecting real-time operation data during the flight training; Determine the width and step size of the sliding window, and flatten the real-time operation data in the flight training into real-time operation action sequence data in the flight training; Using the sliding window to segment the real-time operation action sequence data in the flight training to obtain a plurality of initial flight training action subsequences, and constructing a real-time operation data set according to the plurality of initial flight training action subsequences; Determining the width and step size of the sliding window includes: The width and step size of the sliding window are determined according to boundary conditions, wherein the boundary conditions include: The number of times the absolute value of the pitch angle is greater than 20 for 10 consecutive times is greater than or equal to 2, and the pitch angle is greater than 10, cutting starts; the number of times the absolute value of the tilt angle is less than 3.5 for 6 consecutive times is greater than or equal to 4, and the number of times the absolute value of the pitch angle is less than 6 for 6 consecutive times is greater than or equal to 4, cutting ends; Cutting starts when the absolute value of the tilt angle is greater than 2 for 5 consecutive times and is greater than or equal to 4. Cutting ends when the absolute value of the tilt angle is less than 3.5 for 6 consecutive times and is greater than or equal to 4, and the absolute value of the pitch angle is less than 6 for 6 consecutive times and is greater than or equal to 4.

7. A terminal, characterized in that: include: one or more processors; a memory for storing said one or more processor executable instructions; Wherein, the one or more processors are configured to: Execute the flight action recognition method based on a bidirectional recurrent neural network as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the flight action recognition method based on a bidirectional recurrent neural network as described in any one of claims 1 to 5.

Images