Unmanned aerial vehicle state representation and determination method based on CNN-RF classification model

Abstract

This invention discloses a method for representing and determining the state of a drone based on a CNN-RF classification model, comprising the following steps: S1, constructing a BP neural network to complete the data; for test signals with missing data during drone flight, the BP neural network uses the test signal data without missing data in the test signal set to predict the missing data, obtaining a completed test signal set; S2, designing a feature extractor based on a CNN algorithm involving multiple convolution calculations to extract features from the completed test signal set, obtaining a feature vector that can represent the drone state; S3, designing an RF as a classifier, using the drone state feature vector extracted by the CNN to complete the determination of the drone state. This invention, based on the CNN-RF classification model, improves the accuracy of drone state determination in the case of missing data.

Description

Technical Field

[0001] This invention relates to the field of data-driven fault diagnosis, and more particularly to a method for characterizing and determining the state of a UAV based on a CNN-RF classification model. Background Technology

[0002] Autonomous detection is widely used in drones, and it is generally recognized as the process of detecting whether variables or parameters are within acceptable ranges during a dynamic process to determine the drone's state. With the continuous development of technology, the requirements for the speed and reliability of equipment performance status analysis and confirmation before switching of multimodal systems are gradually increasing. At the same time, the equipment operating environment is becoming more complex and changeable. Self-testing algorithms need to complete the detection of system status in multi-task and complex environments in the shortest possible time, so as to enable timely detection and handling when anomalies occur in the system status.

[0003] The self-testing algorithm uses signals from various sensors as its information source. However, in practical engineering, the self-testing algorithm typically does not use all information source data, but rather performs system self-testing with limited information sources. When all sensor data from each module in the device can be acquired normally, each module has corresponding sensor data that can characterize its state. Anomaly detection of the data stream from each sensor node can determine whether the corresponding module is abnormal. However, test signals may be affected during transmission due to electromagnetic interference, causing the control end to be unable to acquire individual test signals or lose signals for a certain period. Incomplete test signal nodes reduce the information sources available to the self-testing algorithm, making it difficult to accurately locate the abnormal module when measurement points are incomplete. When some measurement node signals are missing, there is no signal directly characterizing the state of the corresponding module. The state characterization algorithm under full information source conditions has an incomplete input data structure and cannot determine the state of the module with missing test signals. Since there are coupling relationships between modules in the UAV, when the working state of one module changes, the test signal data of related modules will be affected and change accordingly. Therefore, when the test signal of a certain module cannot be acquired, it is necessary to consider using sensor data from other related modules to determine the working state of that module. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a method for representing and determining the state of unmanned aerial vehicles (UAVs) based on a CNN-RF classification model, which solves the problem of UAV state representation in the case of missing data. It uses a BP neural network to automatically identify missing data and fill it in accordingly, uses a CNN to extract features from the filled data to represent the system state, and combines it with an RF classifier to complete the system state judgment, thereby improving the accuracy of UAV state determination in the case of missing data.

[0005] Technical solution: The method for characterizing and determining the state of an unmanned aerial vehicle (UAV) of the present invention includes the following steps:

[0006] S1. Construct a BP neural network to complete the data; For test signals with missing data during UAV flight, use the BP neural network to predict the missing data using the test signal data in the test signal set that is not missing, and obtain the test signal set after completion.

[0007] S2, a feature extractor is designed based on the CNN algorithm that includes multiple convolution calculations to extract features from the padded test signal set and obtain feature vectors that can characterize the state of the UAV.

[0008] S3. RF is designed as a classifier. Based on the drone state feature vector extracted by CNN, RF is used to complete the judgment of the drone state.

[0009] Furthermore, in step S1, the specific steps for constructing the BP neural network to complete the data are as follows:

[0010] S11, Establish M three-layer BP neural networks to fill in the missing data when there is data missing in the corresponding test signal;

[0011] S12, train each BP neural network. The i-th BP neural network is used to complete the i-th test signal data. The training set is the offline dataset {X1,…,X} of each test signal without missing data. i-1 ,X i+1 ,…,X M}, where M is the number of test signals;

[0012] S13, save the trained BP neural network model;

[0013] S14, determine if there is missing data in the online test signal data. Let the test signal data at the current moment during the UAV's flight be {x1, x2, ..., x...}. M If a test signal has missing data, the corresponding BP neural network model is called to complete the data, resulting in the completed test signal {x1, x2, ..., x}. M}'.

[0014] Furthermore, in step S1, the specific steps for establishing the BP neural network include the following:

[0015] S111, the three-layer network structure including the input layer, hidden layer, and output layer is determined as follows:

[0016]

[0017] Where M-1 represents the number of input layer nodes, the number of output layer nodes is 1, and the number of hidden layer nodes is I;

[0018] S112, Input signal After passing through the input layer output, the input to the i-th neuron in the hidden layer... The output of the i-th neuron is obtained after processing. ω mi Input signal The weights in the i-th neuron of the hidden layer are expressed as follows:

[0019]

[0020]

[0021] Where f(·) is the activation function;

[0022] S113, the input of the j-th neuron in the output layer. ω ij for The weights in the j-th neuron of the output layer are processed to obtain the output. The output of the network is expressed as follows:

[0023]

[0024]

[0025] Where g(·) is the activation function.

[0026] Furthermore, in step S2, the CNN-based feature extraction specifically includes the following steps:

[0027] S21, set the window size k, and use a sliding window to display the test signal data X. i The data is converted into several k×1 time series, and then the time series of each test signal are combined into a matrix as the input of the CNN.

[0028] S22, Design the CNN network structure and set the hyperparameters according to the input and output sizes;

[0029] S23, train the CNN network structure. The CNN connects fully connected layers with softmax as the activation function to produce a classification result on the extracted features. The network performance is evaluated by the classification accuracy.

[0030] S24, After training, remove the softmax layer and save the CNN as a feature extractor for the offline dataset {X1,X2,…,X...}. M Feature extraction is performed to obtain a set {a1, a2, ..., a} of the UAV's state feature vectors at each time step. t};

[0031] S25, Input the completed test signal data {x1,x2,…,x} for the current time d. MThe test signal data from the previous k-1 time steps are used as inputs to the CNN to obtain the UAV state feature vector a at the current time step d. d .

[0032] Furthermore, in step S3, designing the classifier RF to determine the UAV state includes the following steps:

[0033] S31, Build an RF model and use CNN to extract the UAV state feature vector {a1,a2,…,a3} from the offline dataset. t Train RF and save the model after training.

[0034] S32, the current time-d UAV state feature vector a extracted by CNN. d Input the RF model to obtain the UAV state c at the current moment. d .

[0035] Compared with the prior art, the significant advantages of this invention are as follows:

[0036] 1. By using a BP neural network to automatically identify missing signals and fill in the missing data when test signal data is missing, the problem of uncertain input data topology structure caused by missing test signal data is effectively solved.

[0037] 2. Use a sliding window to preprocess the test signal data and convert the sensor data into a spatiotemporal matrix to facilitate feature extraction;

[0038] 3. A CNN-based feature extractor was designed to achieve dimensionality reduction representation of high-dimensional data. By comprehensively extracting temporal and spatial correlations from test signal data, high-quality state features can be obtained, thereby improving the classification performance of subsequent classifiers.

[0039] 4. This invention is completed through software algorithms, which can be directly used to characterize and determine the state of UAVs in the case of missing test signal data, thus realizing the flexibility and low cost of the algorithm. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating the structure of the present invention;

[0041] Figure 2 This is a schematic diagram of the CNN-RF model in this invention;

[0042] Figure 3(a) shows the data prediction effect of the BP neural network data completion algorithm of sensor 1 in the embodiment.

[0043] Figure 3(b) shows the data prediction effect of the BP neural network data completion algorithm of sensor 2 in the embodiment.

[0044] Figure 3(c) shows the data prediction effect of the BP neural network data completion algorithm of sensor 3 in the embodiment.

[0045] Figure 3(d) shows the data prediction effect of the BP neural network data completion algorithm of sensor 4 in the embodiment;

[0046] Figure 4(a) shows the test results of the CNN-RF classification model for determining the state of the UAV during the first flight in the embodiment.

[0047] Figure 4(b) shows the test results of the CNN-RF classification model for determining the state of the UAV during the second flight in the embodiment.

[0048] Figure 4(c) shows the test results of the CNN-RF classification model for determining the state of the UAV during the third flight in the embodiment.

[0049] Figure 4(d) shows the test results of the CNN-RF classification model for determining the state of the UAV during the fourth flight in the embodiment. Detailed Implementation

[0050] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0051] To ensure the normal operation of the drone, it is necessary to determine the system state based on the data measured by various sensors. Determining the system state can be divided into two steps: extracting features from sensor data to characterize the system state and judging the system state based on these features. When designing the state characterization and determination algorithms, it is necessary to design a reasonable feature extraction method that accurately represents the system state and establish a mapping relationship between the state features and the system state to accurately determine the system state.

[0052] like Figure 1 As shown, a method for UAV state representation and determination based on a CNN-RF (CNN, Convolutional Neural Network; RF, Random Forest) classification model includes the following steps:

[0053] Step 1: Design a BP neural network to complete the data; For test signals with missing data during UAV flight, the BP neural network uses the test signal set with no missing test signal data to predict the missing data and obtain the test signal set after completion.

[0054] The specific steps for designing a BP neural network to complete the data are as follows:

[0055] Offline training:

[0056] Input: An offline dataset {X1,X2,…,X} containing all test signals without missing data during the drone's flight. M}; Number of test signals M;

[0057] Step 11: Establish M three-layer BP neural networks to fill in missing data when there is data in the corresponding test signal; the specific steps for establishing the BP neural networks include the following:

[0058] Step 111: Determine the three-layer network structure as an input layer, a hidden layer, and an output layer. The number of nodes in the input layer is M-1, the number of nodes in the output layer is 1, and the number of nodes in the hidden layer I is determined according to the following empirical formula:

[0059]

[0060] Step 112, Network input signal After passing through the input layer output, the input to the i-th neuron in the hidden layer... for The weighted sum, ω mi Input signal The weights in the i-th neuron of the hidden layer are processed by the activation function f(·) to obtain the output of the i-th neuron. As shown in equations (2) and (3):

[0061]

[0062]

[0063] Step 113, the input of the j-th neuron in the output layer of the network. for The weighted sum, ω ij for The weights in the j-th neuron of the output layer are processed by the activation function g(·) to obtain the output. This is the output of the network, as shown in equations (4) and (5):

[0064]

[0065]

[0066] Step 12: Train each BP neural network. The i-th BP neural network is used to complete the i-th test signal data. The training set is {X1,…,X...} i-1 ,X i+1 ,…,X M};

[0067] Step 13: Save the trained BP neural network model;

[0068] Complete online:

[0069] Input: Online test signal data {x1, x2, ..., x} during the current flight of the UAV. M}

[0070] Step 14: Determine if there is missing data in the online test signal data. If a test signal has missing data, call the corresponding BP neural network model to complete the data, obtaining the completed test signal {x1, x2, ..., x...}. M}'.

[0071] Step 2: Design a feature extractor based on the CNN algorithm which includes multiple convolution calculations, and extract features from the padded test signal set to obtain feature vectors that can characterize the state of the UAV.

[0072] CNN-based feature extraction specifically includes the following steps:

[0073] Offline training:

[0074] Input: An offline dataset {X1,X2,...,X} containing all test signals from the drone flight without any missing data. M}; Number of test signals M; Sliding window size k;

[0075] Step 21, data preprocessing: The test signal data X is processed using a sliding window. i The data is converted into several k×1 time series, and then the time series of each test signal are combined into a matrix as the input of the CNN.

[0076] Step 22: Design the CNN network structure, set the hyperparameters according to the input and output sizes, and further adjust the hyperparameters during subsequent training.

[0077] Step 23: Train the CNN. During training, the CNN is connected to fully connected layers with softmax as the activation function to generate a classification result for the extracted features. The network performance is evaluated by the classification accuracy.

[0078] Step 24: After training, remove the softmax layer and save the CNN as a feature extractor for the offline dataset {X1,X2,...,X...}. M Feature extraction is performed to obtain a set {a1, a2, ..., a} of the UAV's state feature vectors at each time step. t}

[0079] Online state representation:

[0080] Input: For a given time d, the input consists of the padded test signal data {x1, x2, ..., x...} at the current time. M}' and the test signal data for the first k-1 time steps;

[0081] Step 25: Input the time series matrix of each test signal data at the current time d into the CNN to obtain the UAV state feature vector a at the current time d. d .

[0082] Step 3: Design an RF classifier. Based on the state feature vector extracted by the CNN, the RF classifier is used to determine the state of the drone.

[0083] Designing a classifier (RF) to determine the state of a drone involves the following steps:

[0084] Offline training:

[0085] Input: Offline dataset of test signals {X1,X2,…,X} M The corresponding state feature vectors at each time step {a1, a2, ..., a} t};

[0086] Step 31: Build the RF model and use CNN to extract the state feature vector {a1,a2,…,a...} from the offline dataset. t Train RF and save the model after training.

[0087] Online status confirmed:

[0088] Input: The current state feature vector of the UAV, a d

[0089] Step 32, extract the current time d UAV state feature vector a from the CNN. d Input RF to obtain the current drone state c. d .

[0090] Example:

[0091] The present invention was simulated and verified on a dataset from a real flight experiment of a UAV. The dataset contains 26 sensor data points from 47 flight processes, totaling 141,000 samples, including data from five operating states: normal operation of the UAV, engine failure, elevator failure, aileron failure, and rudder failure.

[0092] The computer system used for testing was configured with Windows 10, an Intel i5 quad-core CPU, 8GB of RAM, and 1TB of hard disk space. The BP neural network was developed using MATLAB. The model's input consisted of 25 test signals (M=25) occurring simultaneously, and the output was the predicted test signal data for that specific moment. The input dimension was 25×1, and the output dimension was 1×1. The BP neural network was constructed using the `newff` function and trained using the `train` function. Training was set to run 100 times or until the mean squared error decreased to 1×10⁻⁶. -4 Training should be stopped at the following times; such as Figure 2 As shown, the CNN-RF model is developed using Python, and the convolutional neural network model is built using the deep learning framework TensorFlow. The network is trained using the Keras package in TensorFlow, and the RandomForestClassifier function in sklearn is used to build the RF model.

[0093] In the test, one flight data point was extracted, and the trained network model was used to predict the data from sensors 1 to 4 in this data to verify the performance of the BP neural network in data completion. The results are shown in Figure 3, based on the accuracy coefficient R. 2 The accuracy of the network-completed data was evaluated. As shown in Figure 3, using a BP neural network model to complete the sensor data yielded good results. The predicted data accurately reflected the changing trends of the sensor data, thus reflecting the working status of the corresponding module.

[0094] To validate the CNN-RF model, four flight data points containing five system states were extracted from 47 flights in the dataset. Missing data in these data points were filled in using the BP neural network, and the CNN-RF model was then used for state representation and determination. The results are shown in Figure 4. As can be seen from Figure 4, the CNN-RF model's determination of the UAV state based on the complete data is very close to the actual situation, and it can accurately determine the five types of UAV states.

[0095] As can be seen from the above steps and figures, the present invention can effectively realize a method for UAV state representation and determination based on CNN-RF, accurately determine the system state in the case of missing data, and is of great significance for system state self-checking under limited information sources.

[0096] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the inventive principle, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for representing and determining the state of a UAV based on a CNN-RF classification model, characterized in that, Includes the following steps: S1. Construct a BP neural network to complete the data; For test signals with missing data during UAV flight, use the BP neural network to predict the missing data using the test signal data in the test signal set that is not missing, and obtain the test signal set after completion. S2, a feature extractor is designed based on the CNN algorithm that includes multiple convolution calculations to extract features from the padded test signal set and obtain feature vectors that can characterize the state of the UAV. S3, Design RF as a classifier, and use RF to judge the state of the drone based on the drone state feature vector extracted by CNN. In step S1, the specific steps for constructing the BP neural network to complete the data are as follows: S11, Establish M three-layer BP neural networks to fill in the missing data when there is data missing in the corresponding test signal; S12, train each BP neural network. The i-th BP neural network is used to complete the i-th test signal data. The training set is the offline dataset of each test signal without missing data. M is the number of test signals; S13, save the trained BP neural network model; S14, determine if there is any missing data in the online test signal data, assuming that the test signal data during the current time of the UAV's flight are as follows: When a test signal has missing data, the corresponding BP neural network model is called to complete the data, resulting in a completed test signal. ; In step S2, the CNN-based feature extraction specifically includes the following steps: S21, Let the sliding window size be k, and use the sliding window to display the test signal data. Convert to several The time series of the test signals are then combined into a matrix and used as the input to the CNN. S22, Design the CNN network structure and set the hyperparameters according to the input and output sizes; S23, train the CNN network structure. The CNN connects fully connected layers with softmax as the activation function to produce a classification result on the extracted features. The network performance is evaluated by the classification accuracy. S24, After training, remove the softmax layer and save the CNN as a feature extractor for the offline dataset. Feature extraction is performed to obtain a set of UAV state feature vectors at each time step. ; S25, Input the completed test signal data for the current time d. and the former The test signal data at each time point d are used; the time series matrix of each test signal at the current time d is input into the CNN to obtain the UAV state feature vector at the current time d. .

2. The method for UAV state representation and determination based on the CNN-RF classification model according to claim 1, characterized in that, In step S1, the specific steps for establishing the BP neural network include the following: S111, the three-layer network structure including the input layer, hidden layer, and output layer is determined as follows: ， in, The number of input layer nodes is 1, the number of output layer nodes is 0, and the number of hidden layer nodes is 1. ; S112, Input signal After the input layer output, the hidden layer's... Input of each neuron After processing, the first The output of each neuron , Input signal In the hidden layer The weights in each neuron are expressed as follows: ， ， in, For activation functions; S113, Output Layer Input of each neuron , for In the output layer The weights of each neuron are processed to obtain the output. The output of the network is expressed as follows: ， ， in, This is the activation function.

3. The method for UAV state representation and determination based on the CNN-RF classification model according to claim 1, characterized in that, In step S3, designing the classifier RF to determine the UAV state includes the following steps: S31. Build an RF model and use CNN to extract the UAV state feature vector from the offline dataset. Train RF and save the model after training is complete; S32, extract the UAV state feature vector at the current time d from the CNN. Input the RF model to obtain the current state of the UAV. .

Images