Aircraft motion recognition method and system

By splitting the 3D convolution kernel into time and space convolution kernels, a flight action convolution neural network is established, and the problems of low aircraft action recognition accuracy and long training time in the existing technology are solved, and more efficient feature extraction and Recognition accuracy.

CN113255464BActive Publication Date: 2024-12-06BEIJING ZHONGDIAN LIANDA INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110481977.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-04-30
Publication Date
2024-12-06
Estimated Expiration
2041-04-30

AI Technical Summary

Technical Problem

When the existing action recognition method is classified and recognized by aircraft action, the classification accuracy is low, the training time is long, and it is difficult to effectively combine the space-time information in the video data.

Method used

By splitting the 3D convolution kernel into a time convolution kernel and a spatial convolution kernel, a flight action convolution neural network is established, and the two-dimensional image features of video frames are extracted using the spatial convolution module, and the time convolution module extracts inter-frame information. The two are spliced ​​into joint space-time features and input them to the fully connected layer and the Softmax layer for identification.

Benefits of technology

It significantly improves the recognition accuracy of flight actions, reduces training time and parameter amount, enhances the propagation and utilization of feature information, and improves the depth and accuracy of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113255464B_ABST
    Figure CN113255464B_ABST
Patent Text Reader

Abstract

The present invention provides an aircraft action recognition method and system, belonging to the field of target detection technology. The present invention provides an aircraft action recognition method, first pre-processing the flight sample video data to obtain continuous frame training samples, and inputting the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model, and finally inputting the real-time flight video data into the flight action recognition model to complete the aircraft action recognition. The present invention divides the 3D convolution kernel into two convolution kernels of time and space, and establishes a flight action convolutional neural network according to the time and space convolution kernels. The spatial convolution module can be used to extract two-dimensional image features for each video frame, and the temporal convolution module can extract inter-frame information for multiple video frames. The present invention strengthens the propagation of video features, so that the feature information is better propagated in the entire neural network, and a deeper recognition model can be trained, which significantly improves the recognition accuracy of flight actions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and more specifically, relates to an aircraft motion recognition method and system. Background Art

[0002] With the rapid development of electronic information technology and computer science, action recognition plays an increasingly important role in the field of computer vision. As an important branch of computer vision, the main goal of action recognition is to enable computers to correctly identify and classify a series of behaviors and actions. It has a wide range of applications in intelligent video surveillance, traffic flow prediction, military operations and other fields. Traditional action recognition methods use the method of manually extracting motion features for recognition, but in recent years, with the rise and development of deep learning method theory in the field of image recognition, this method theory has been applied to the field of action recognition, making the accuracy of action recognition methods higher and higher. Deep learning is a method of characterizing and learning data. It realizes the automation of motion feature extraction by combining low-level features to form more abstract high-level features or attributes. Among them, convolutional neural networks perform well in image feature extraction. However, feature extraction based on convolutional neural networks is a two-dimensional image convolution operation for a single image, while action recognition is a series of coherent actions, that is, the actions of adjacent video frames are interrelated, which makes the single 2D convolution operation ignore the information of the action in the video stream based on time changes, so it is difficult to achieve a high accuracy in the field of action recognition.

[0003] Depending on the type of data processed, action recognition methods can be divided into static image-based action recognition methods and video-based action recognition methods.

[0004] Action recognition methods based on static images need to consider the posture information of objects in the image and the interaction information between objects; action recognition methods based on videos need to consider not only the static spatial structure information of each frame of the video, but also the time-varying dynamic information between each frame in the video.

[0005] In recent years, researchers have proposed many methods for the problem of video action recognition. The first is a neural network system for 3D convolutional feature extraction, which extends the convolutional neural network to a three-dimensional convolutional neural network. The constructed 3D convolutional neural network can simultaneously extract time domain and spatial domain features to capture the motion information encoded between multiple adjacent video frames to identify and classify the current action of the object; secondly, there is a method that believes that a video action sequence can predict the action classification by inputting several key video frame images, and the remaining video frame images are redundant video frame image information. At the same time, a key frame mining model for action recognition is proposed to extract key frames, which reduces the amount of parameters while extracting more important video frame images to identify and classify object actions; finally, there is a method that proposes a feature fusion method that combines different types of classic descriptors and multi-channel 3D convolutional neural networks for action recognition and classification.

[0006] However, the above action recognition method still has the following problems: the traditional 2D convolution method can only perform convolution on the spatial dimension and cannot effectively combine the spatiotemporal information in the video data, which will cause low classification accuracy and inaccurate classification. When the 3D convolution method processes video stream data, the video stream data is too large and the number of parameters of the 3D convolution kernel is too large, which will result in long training time and difficulty in adjusting parameters. In addition, the video resource itself will be affected by many factors, such as the clutter of the video background, the occlusion of the target object, and the change of viewing angle, which makes the processing and analysis of the video very difficult. Summary of the invention

[0007] The purpose of the present invention is to provide an aircraft action recognition method and system, aiming to solve the problems of low classification accuracy and long training time when the existing action recognition method classifies and recognizes aircraft actions.

[0008] To achieve the above object, the technical solution adopted by the present invention is: an aircraft action recognition method, comprising the following steps:

[0009] Step 1: Obtain flight sample video data of the aircraft;

[0010] Step 2: preprocessing the flight sample video data to generate preprocessed flight sample video data;

[0011] Step 3: extracting each flight action of the aircraft in the preprocessed flight sample video data as a continuous frame training sample;

[0012] Step 4: Split the 3D convolution kernel into temporal convolution kernel and spatial convolution kernel;

[0013] Step 5: Establish a flight action convolutional neural network based on the temporal convolution kernel and the spatial convolution kernel;

[0014] Step 6: Input the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model; wherein the flight action recognition model is obtained by training with the continuous frame training samples as input and each aircraft action as output;

[0015] Step 7: Obtain real-time flight video data of the aircraft;

[0016] Step 8: Input the real-time flight video data into the flight action recognition model to perform aircraft action recognition.

[0017] Preferably, the step 2: preprocessing the flight sample video data to generate preprocessed flight sample video data includes:

[0018] Step 2.1: converting the flight sample video data into a flight sample picture sequence frame;

[0019] Step 2.2: Taking each pixel point of each frame in the flight sample image sequence frame as the center, taking a neighborhood, arranging the pixel values ​​in the neighborhood from small to large, and generating a pixel ascending sequence;

[0020] Step 2.3: Take the pixel value corresponding to the median of the pixel ascending sequence as the output of the central pixel to obtain a central pixel set;

[0021] Step 2.4: generating a preprocessed flight sample image sequence frame according to the central pixel set;

[0022] Step 2.5: Generate the preprocessed flight sample video data according to the preprocessed flight sample picture sequence frames.

[0023] Preferably, the step 4: splitting the 3D convolution kernel into a temporal convolution kernel and a spatial convolution kernel comprises:

[0024] The T×H×W 3D convolution kernel is split into a 1×H×W spatial convolution kernel and a T×1×1 temporal convolution kernel, where T represents the time dimension of the convolution kernel, H represents the height of the convolution kernel, and W represents the width of the convolution kernel.

[0025] Preferably, the step 8: inputting the real-time flight video data into the flight action recognition model to perform aircraft action recognition includes:

[0026] Step 8.1: preprocessing the real-time flight video data to generate preprocessed real-time flight video data;

[0027] Step 8.2: converting the preprocessed real-time flight video data into real-time flight picture sequence frames;

[0028] Step 8.3: Obtaining inter-frame information according to the real-time flight picture sequence frames;

[0029] Step 8.4: Using a spatial convolution layer to extract the real-time flight picture sequence frames to generate spatial feature information;

[0030] Step 8.5: extracting the inter-frame information using a temporal convolutional layer to generate temporal feature information;

[0031] Step 8.6: concatenate the spatial feature information and the temporal feature information to obtain a joint spatiotemporal feature;

[0032] Step 8.7: Input the joint spatiotemporal features into the first fully connected layer, the second fully connected layer and the Softmax layer in sequence to obtain the aircraft action.

[0033] Preferably, the step 8.3: obtaining inter-frame information according to the real-time flight picture sequence frames includes:

[0034] Step 8.3.1: converting the preprocessed real-time flight video data into real-time flight picture sequence frames;

[0035] Step 8.3.2: Use a differential formula to calculate the real-time flight picture sequence frame to obtain a differential picture sequence frame; wherein the differential formula is:

[0036]

[0037] Where D(x,y) represents the difference image between two consecutive frames, I(t) represents the image in the real-time flight image sequence frame at time t, I(t-1) represents the image in the real-time flight image sequence frame at time t-1, and T is the grayscale threshold value;

[0038] Step 8.3.3: Arrange the differential picture sequence frames in ascending order of time to obtain the inter-frame information.

[0039] Preferably, the step 8.4: using a spatial convolution layer to extract the real-time flight picture sequence frames to generate spatial feature information includes:

[0040] The real-time flight picture sequence frames are sequentially input into the first spatial convolution layer, the second spatial convolution layer, the pooling layer, the third spatial convolution layer, the fourth spatial convolution layer, the first spatial pooling layer and the spatial Flatten layer to generate spatial feature information; wherein the convolution kernels in the first spatial convolution layer, the second spatial convolution layer, the third spatial convolution layer and the fourth spatial convolution layer are all 1×3×3 spatial convolution kernels.

[0041] Preferably, the step 8.5: extracting the inter-frame information by using a temporal convolution layer to generate temporal feature information comprises:

[0042] The inter-frame information is sequentially input into the first temporal convolution layer, the second temporal convolution layer, the first temporal pooling layer, the third temporal convolution layer, the fourth temporal convolution layer, the second temporal pooling layer and the temporal Flatten layer to generate temporal feature information; wherein the convolution kernels in the first temporal convolution layer, the second temporal convolution layer, the third temporal convolution layer and the fourth temporal convolution layer are all 3×1×1 temporal convolution kernels.

[0043] The present invention also provides an aircraft action recognition system, comprising:

[0044] A flight sample video data acquisition module is used to acquire the flight sample video data of the aircraft;

[0045] A preprocessing module, used for preprocessing the flight sample video data to generate preprocessed flight sample video data;

[0046] A training sample extraction module, used to extract each flight action of the aircraft in the preprocessed flight sample video data as a continuous frame training sample;

[0047] Convolution kernel splitting module, used to split the 3D convolution kernel into temporal convolution kernel and spatial convolution kernel;

[0048] A flight action convolutional neural network establishment module, used to establish a flight action convolutional neural network according to the temporal convolution kernel and the spatial convolution kernel;

[0049] A flight action recognition model generation module is used to input the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model; wherein the flight action recognition model is obtained by training with the continuous frame training samples as input and each aircraft action as output;

[0050] A real-time flight video data acquisition module is used to acquire real-time flight video data of the aircraft;

[0051] The aircraft action recognition module is used to input the real-time flight video data into the flight action recognition model to perform aircraft action recognition.

[0052] The beneficial effect of an aircraft action recognition method and system provided by the present invention is that: compared with the prior art, an aircraft action recognition method of the present invention first pre-processes the flight sample video data to obtain continuous frame training samples, and inputs the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model, and finally inputs the real-time flight video data into the flight action recognition model to complete the aircraft action recognition. The present invention divides the 3D convolution kernel into two convolution kernels of time and space, and establishes a flight action convolutional neural network according to the time and space convolution kernels. The spatial convolution module can be used to extract two-dimensional image features for each video frame, and the temporal convolution module can extract inter-frame information for multiple video frames. The present invention strengthens the propagation of video features, so that the feature information is better maintained during the propagation process of the entire neural network, and a deeper recognition model can be trained, which significantly improves the recognition accuracy of flight actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A flow chart of an aircraft motion recognition method provided by an embodiment of the present invention;

[0055] Figure 2 Provide a schematic diagram of 3D convolution kernel splitting for an embodiment of the present invention;

[0056] Figure 3 A flowchart of the overall framework of a neural network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0058] The purpose of the present invention is to provide an aircraft action recognition method and system, aiming to solve the problems of low classification accuracy and long training time when the existing action recognition method classifies and recognizes aircraft actions.

[0059] See also Figure 1 To achieve the above purpose, the technical solution adopted by the present invention is: an aircraft action recognition method, comprising the following steps:

[0060] S1: Obtain flight sample video data of the aircraft;

[0061] In the present invention, a high-definition camera is used to shoot a video of an airplane in flight to obtain flight sample video data.

[0062] S2: preprocessing the flight sample video data to generate preprocessed flight sample video data;

[0063] S2 specifically includes:

[0064] S2.1: converting the flight sample video data into a flight sample picture sequence frame;

[0065] In the present invention, video editing software such as Video Editing can be used to convert the flight sample video data into a flight sample picture sequence frame.

[0066] S2.2: Take a neighborhood with each pixel point of each frame in the flight sample image sequence as the center, arrange the pixel values ​​in the neighborhood from small to large, and generate a pixel ascending sequence;

[0067] S2.3: Take the pixel value corresponding to the median of the ascending sequence of pixels as the output of the central pixel to obtain the central pixel set;

[0068] S2.4: Generate a pre-processed flight sample image sequence frame according to the central pixel set;

[0069] S2.5: Generate preprocessed flight sample video data according to the preprocessed flight sample image sequence frames.

[0070] The present invention can suppress noise on a target image while retaining the edge and detail features of the original image by preprocessing the image, so as to reduce the impact of different impulse noises on the image and improve the accuracy of subsequent model training.

[0071] S3: extracting various flight actions of the aircraft in the preprocessed flight sample video data as continuous frame training samples; wherein the flight actions include: horizontal flight, left turn, right turn, climb and descent.

[0072] S4: Split the 3D convolution kernel into temporal convolution kernel and spatial convolution kernel;

[0073] Using 3D convolution kernel to directly convolve video input data requires a large number of parameters and a large amount of calculation, which results in a long calculation time and makes parameter adjustment very difficult. In order to solve the above problems, the 3D convolution kernel is split into two parts: time convolution and space convolution. The time feature information and space feature information of the video image are extracted respectively, which reduces the number of parameters, improves the network calculation efficiency, and enhances the interaction between time information and space information, which can greatly reduce the number of parameters and the amount of calculation.

[0074] S4 specifically includes:

[0075] The T×H×W 3D convolution kernel is split into a 1×H×W spatial convolution kernel and a T×1×1 temporal convolution kernel, where T represents the time dimension of the convolution kernel, H represents the height of the convolution kernel, and W represents the width of the convolution kernel.

[0076] See also Figure 2 , this process is further explained below.

[0077] H and W are the height and width of the convolution kernel respectively, T is the time dimension of the convolution kernel, and the 3D convolution kernel of size T×H×W is split into two sizes of convolution kernels, 1×H×W and T×1×1, which perform convolution operations on the video stream data respectively. The 1×H×W convolution kernel extracts two-dimensional image features for each video frame, and the T×1×1 convolution kernel performs deep convolution on multiple video frames to extract inter-frame information, thereby capturing the information between motion frames.

[0078] S5: Establish a flight action convolutional neural network based on the temporal convolution kernel and the spatial convolution kernel;

[0079] S6: inputting the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model; wherein the flight action recognition model is obtained by training with the continuous frame training samples as input and each aircraft action as output;

[0080] S7: Acquire real-time flight video data of the aircraft;

[0081] S8: Input the real-time flight video data into the flight action recognition model to perform aircraft action recognition.

[0082] S8 specifically includes:

[0083] S8.1: Preprocess the real-time flight video data to generate preprocessed real-time flight video data; in the present invention, the principle of preprocessing the real-time flight video data to generate preprocessed real-time flight video data is the same as the principle of preprocessing the flight sample video data in S2, which will not be repeated.

[0084] S8.2: Convert the pre-processed real-time flight video data into real-time flight picture sequence frames; in the present invention, video editing software such as VideoStudio can be used to convert the flight sample video data into flight sample picture sequence frames.

[0085] S8.3: Obtain inter-frame information according to the real-time flight picture sequence frames;

[0086] S8.3 specifically includes:

[0087] S8.3.1: Convert the preprocessed real-time flight video data into real-time flight picture sequence frames;

[0088] S8.3.2: Use a differential formula to calculate the real-time flight image sequence frames to obtain differential image sequence frames; where the differential formula is:

[0089]

[0090] Where D(x,y) represents the difference image between two consecutive frames, I(t) represents the image in the real-time flight image sequence frame at time t, I(t-1) represents the image in the real-time flight image sequence frame at time t-1, and T is the grayscale threshold value;

[0091] S8.3.3: Arrange the frames of the differential image sequence in ascending order of time to obtain inter-frame information.

[0092] The present invention utilizes differential images as inter-frame information to well reflect changes in videos at different times, thereby enhancing the utilization efficiency of video information.

[0093] S8.4: Use spatial convolutional layers to extract real-time flight image sequence frames to generate spatial feature information;

[0094] S8.4 specifically includes:

[0095] The real-time flight picture sequence frames are input into the first spatial convolution layer, the second spatial convolution layer, the pooling layer, the third spatial convolution layer, the fourth spatial convolution layer, the first spatial pooling layer and the spatial Flatten layer in sequence to generate spatial feature information; wherein, the convolution kernels in the first spatial convolution layer, the second spatial convolution layer, the third spatial convolution layer and the fourth spatial convolution layer are all 1×3×3 spatial convolution kernels.

[0096] S8.5: Use the temporal convolution layer to extract inter-frame information and generate temporal feature information;

[0097] S8.5 specifically includes:

[0098] The inter-frame information is input into the first temporal convolution layer, the second temporal convolution layer, the first temporal pooling layer, the third temporal convolution layer, the fourth temporal convolution layer, the second temporal pooling layer and the temporal Flatten layer in sequence to generate temporal feature information; wherein, the convolution kernels in the first temporal convolution layer, the second temporal convolution layer, the third temporal convolution layer and the fourth temporal convolution layer are all 3×1×1 temporal convolution kernels.

[0099] S8.6: concatenate the spatial feature information and the temporal feature information to obtain a joint spatiotemporal feature;

[0100] S8.7: The joint spatiotemporal features are sequentially input into the first fully connected layer, the second fully connected layer, and the Softmax layer to obtain the aircraft action. After obtaining the temporal feature information and the spatial feature information, if the temporal feature information and the spatial feature information are processed separately, firstly, the amount of parameter calculation will increase, and secondly, the correlation information between the feature information is not extracted, which will reduce the calculation accuracy. To solve this problem, the temporal features and the spatial features are spliced ​​to obtain the joint spatiotemporal features. The probability of each action is calculated for the joint spatiotemporal features to increase the calculation accuracy.

[0101] The technical solution of the present invention is further described below in conjunction with specific embodiments.

[0102] Please refer to Figure 3 , input the video frame image, and select multiple consecutive frames as a group as the input of the neural network model. After the video is preprocessed, each input frame generates N channels. The two data streams are respectively subjected to 3D convolutions of size 1×3×3 and 3×1×1. In the spatial convolution module, a 1×3×3 convolution kernel is applied to process the image. This module passes through 2 convolution layers, 1 pooling layer, and then passes through two convolution layers and one pooling layer again. Finally, the image features are converted into one-dimensional vectors through the Flatten layer to obtain spatial features. In the temporal convolution module, a 3×1×1 convolution kernel is applied to process the inter-frame information. This module passes through 2 convolution layers, 1 pooling layer, and then passes through two convolution layers and one pooling layer again. Finally, the inter-frame information features are converted into one-dimensional vectors through the Flatten layer to obtain temporal features. The spatial features and temporal features are concatenated to obtain joint spatiotemporal features, which are then passed through two fully connected layers. After each fully connected layer, the neuron random inactivation method (Dropout) is used to prevent neuron overfitting. Finally, the probability of each action is output through the Softmax layer.

[0103] The present invention also provides an aircraft action recognition system, comprising:

[0104] A flight sample video data acquisition module is used to acquire the flight sample video data of the aircraft;

[0105] A preprocessing module, used for preprocessing the flight sample video data to generate preprocessed flight sample video data;

[0106] A training sample extraction module is used to extract each flight action of the aircraft from the preprocessed flight sample video data as a continuous frame training sample;

[0107] Convolution kernel splitting module, used to split the 3D convolution kernel into temporal convolution kernel and spatial convolution kernel;

[0108] A flight action convolutional neural network building module, used to build a flight action convolutional neural network based on a temporal convolution kernel and a spatial convolution kernel;

[0109] The flight action recognition model generation module is used to input the continuous frame training samples into the flight action convolutional neural network for training to obtain the flight action recognition model; wherein the flight action recognition model is obtained by training with the continuous frame training samples as input and each aircraft action as output;

[0110] A real-time flight video data acquisition module is used to acquire real-time flight video data of the aircraft;

[0111] The aircraft action recognition module is used to input real-time flight video data into the flight action recognition model for aircraft action recognition.

[0112] The beneficial effect of an aircraft action recognition method and system provided by the present invention is that: compared with the prior art, an aircraft action recognition method of the present invention first pre-processes the flight sample video data to obtain continuous frame training samples, and inputs the continuous frame training samples into the flight action convolution neural network for training to obtain a flight action recognition model, and finally inputs the real-time flight video data into the flight action recognition model to complete the aircraft action recognition. After the original 3D convolution is split into two modules of time and space, the present invention uses two convolution kernels of 1×3×3 and 3×1×1 to convolve the time and space modules of the video data, and connects the pooling layer after the convolution operation. The pooling method uses the average pooling (avg-pooling) method to effectively retain the texture information of the image, and the information of the two data streams is flattened and then spliced ​​and transmitted to the Softmax layer to calculate the probability of each action. The present invention integrates time and space information, increases the diversity of information, more effectively utilizes feature information, and strengthens feature propagation, so that the information is better maintained during the propagation process of the entire network, and significantly improves the recognition accuracy of flight actions.

[0113] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for aircraft motion recognition, characterized in that: The following steps are involved: Step 1: Obtain flight sample video data of the aircraft; Step 2: preprocessing the flight sample video data to generate preprocessed flight sample video data; Step 3: extracting each flight action of the aircraft in the preprocessed flight sample video data as a continuous frame training sample; Step 4: Split the 3D convolution kernel into a temporal convolution kernel and a spatial convolution kernel, and extract the temporal feature information and spatial feature information of the video image respectively, which reduces the number of parameters, improves the network calculation efficiency, and enhances the interaction between temporal information and spatial information, which can greatly reduce the number of parameters and the amount of calculation; Step 5: Establish a flight action convolutional neural network based on the temporal convolution kernel and the spatial convolution kernel; Step 6: Input the continuous frame training samples into the flight action convolutional neural network for training to obtain a flight action recognition model; wherein the flight action recognition model is obtained by training with the continuous frame training samples as input and each aircraft action as output; Step 7: Obtain real-time flight video data of the aircraft; Step 8: Inputting the real-time flight video data into the flight action recognition model to perform aircraft action recognition; The step 2: preprocessing the flight sample video data to generate preprocessed flight sample video data, comprises: Step 2.1: converting the flight sample video data into a flight sample picture sequence frame; Step 2.2: Taking each pixel point of each frame in the flight sample image sequence frame as the center, taking a neighborhood, arranging the pixel values ​​in the neighborhood from small to large, and generating a pixel ascending sequence; Step 2.3: Take the pixel value corresponding to the median of the pixel ascending sequence as the output of the central pixel to obtain a central pixel set; Step 2.4: generating a preprocessed flight sample image sequence frame according to the central pixel set; Step 2.5: generating the preprocessed flight sample video data according to the preprocessed flight sample picture sequence frames; The step 4: splitting the 3D convolution kernel into a temporal convolution kernel and a spatial convolution kernel, including: Split the T×H×W 3D convolution kernel into a 1×H×W spatial convolution kernel and a T×1×1 temporal convolution kernel, where T represents the time dimension of the convolution kernel, H represents the height of the convolution kernel, and W represents the width of the convolution kernel; The step 8: inputting the real-time flight video data into the flight action recognition model to perform aircraft action recognition, including: Step 8.1: preprocessing the real-time flight video data to generate preprocessed real-time flight video data; Step 8.2: converting the preprocessed real-time flight video data into real-time flight picture sequence frames; Step 8.3: Obtaining inter-frame information according to the real-time flight picture sequence frames; Step 8.4: Using a spatial convolution layer to extract the real-time flight picture sequence frames to generate spatial feature information; Step 8.5: extracting the inter-frame information using a temporal convolutional layer to generate temporal feature information; Step 8.6: concatenate the spatial feature information and the temporal feature information to obtain a joint spatiotemporal feature; Step 8.7: Input the joint spatiotemporal features into the first fully connected layer, the second fully connected layer and the Softmax layer in sequence to obtain the aircraft action; after obtaining the temporal feature information and the spatial feature information, if the temporal feature information and the spatial feature information are processed separately, firstly, the parameter calculation amount will increase, and secondly, the correlation information between the feature information is not extracted, which will reduce the calculation accuracy. To solve this problem, the temporal features and the spatial features are spliced ​​to obtain the joint spatiotemporal features. The probability of each action is calculated for the joint spatiotemporal features to increase the calculation accuracy; The step 8.3: obtaining inter-frame information according to the real-time flight picture sequence frames includes: Step 8.3.1: converting the preprocessed real-time flight video data into real-time flight picture sequence frames; Step 8.3.2: Use a differential formula to calculate the real-time flight picture sequence frame to obtain a differential picture sequence frame; wherein the differential formula is: Where D(x,y) represents the difference image between two consecutive frames, I(t) represents the image in the real-time flight image sequence frame at time t, I(t-1) represents the image in the real-time flight image sequence frame at time t-1, and T is the grayscale threshold value; Step 8.3.3: Arrange the difference picture sequence frames in ascending order of time to obtain the inter-frame information; The step 8.4: using a spatial convolution layer to extract the real-time flight picture sequence frames to generate spatial feature information, includes: Inputting the real-time flight picture sequence frames into the first spatial convolution layer, the second spatial convolution layer, the pooling layer, the third spatial convolution layer, the fourth spatial convolution layer, the first spatial pooling layer and the spatial Flatten layer in sequence to generate spatial feature information; wherein the convolution kernels in the first spatial convolution layer, the second spatial convolution layer, the third spatial convolution layer and the fourth spatial convolution layer are all 1×3×3 spatial convolution kernels; The step 8.5: extracting the inter-frame information by using a temporal convolution layer to generate temporal feature information includes: The inter-frame information is sequentially input into the first temporal convolution layer, the second temporal convolution layer, the first temporal pooling layer, the third temporal convolution layer, the fourth temporal convolution layer, the second temporal pooling layer and the temporal Flatten layer to generate temporal feature information; wherein the convolution kernels in the first temporal convolution layer, the second temporal convolution layer, the third temporal convolution layer and the fourth temporal convolution layer are all 3×1×1 temporal convolution kernels.