Unmanned aerial vehicle fault diagnosis method and device based on multi-dimensional characteristic spectrum and Ghost network

Abstract

The invention provides an unmanned aerial vehicle fault diagnosis method and device based on a multi-dimensional characteristic spectrum and a Ghost network, and relates to the technical field of unmanned aerial vehicles, and the method comprises the steps: obtaining the time sequence flight data of a target unmanned aerial vehicle; carrying out Gramer angle field transformation on the time sequence flight data to obtain a first atlas, and carrying out continuous wavelet transformation on the time sequence flight data to obtain a second atlas; performing feature extraction on the first map to obtain a first initial feature, performing feature extraction on the second map to obtain a second initial feature, and performing feature fusion on the first initial feature and the second initial feature to obtain a fused map feature; inputting the fused atlas features into a fault diagnosis model to obtain a fault diagnosis result of the target unmanned aerial vehicle output by the fault diagnosis model; the fault diagnosis model is built based on a Ghost network. According to the method and the device provided by the invention, the time sequence characteristics of each dimension in the flight data are fully mined, and the accuracy of unmanned aerial vehicle fault diagnosis is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and particularly to a method and device for unmanned aerial vehicle fault diagnosis based on a multi-dimensional feature map and a Ghost network. Background Art

[0002] The technology of unmanned aerial vehicles has developed rapidly and has been widely used in both military and civilian fields. During the flight of an unmanned aerial vehicle, a variety of precise components need to cooperate with each other, such as gyroscopes, accelerometers, magnetometers, navigation modules, motors, and propellers. During the use of unmanned aerial vehicles, faults will inevitably occur due to factors such as collisions, electromagnetic interference, equipment aging, and component jams. Depending on the location and degree of the faults, various impacts will be imposed on the unmanned aerial vehicles. At the lightest level, it will affect the control accuracy and flight stability of the unmanned aerial vehicle, and at the severest level, it will lead to irreparable serious consequences such as crashing and crashing.

[0003] Related technologies use methods based on expert systems, methods based on physical models, and methods based on data-driven approaches, etc. to diagnose faults of unmanned aerial vehicles. With the increasing complexity of unmanned aerial vehicles, data from expert methods and physical model methods based on experience cannot meet the requirements of fault diagnosis, and data-driven methods have received increasing attention from researchers.

[0004] Therefore, how to make full use of unmanned aerial vehicle flight data and improve the accuracy of unmanned aerial vehicle fault diagnosis has become a technical problem that needs to be solved urgently in the industry. Summary of the Invention

[0005] The present invention provides a method and device for unmanned aerial vehicle fault diagnosis based on a multi-dimensional feature map and a Ghost network, which are used to solve the technical problem of how to make full use of unmanned aerial vehicle flight data and improve the accuracy of unmanned aerial vehicle fault diagnosis.

[0006] The present invention provides a method for unmanned aerial vehicle fault diagnosis based on a multi-dimensional feature map and a Ghost network, including: Obtaining the time-series flight data of a target unmanned aerial vehicle; Performing Gramian angular field transformation on the time-series flight data to obtain a first map, and performing continuous wavelet transformation on the time-series flight data to obtain a second map; Performing feature extraction on the first map to obtain first initial features, performing feature extraction on the second map to obtain second initial features, and performing feature fusion on the first initial features and the second initial features to obtain fused map features; Inputting the fused map features into a fault diagnosis model to obtain a fault diagnosis result of the target unmanned aerial vehicle output by the fault diagnosis model; the fault diagnosis model is built based on a Ghost network.

[0007] In some embodiments, before performing the Gramian Angular Field (GAF) transform on the time-series flight data to obtain a first atlas and performing the continuous wavelet transform on the time-series flight data to obtain a second atlas, the method further includes: Performing a normalization operation on the data of each dimension in the time-series flight data.

[0008] In some embodiments, performing the Gramian Angular Field (GAF) transform on the time-series flight data to obtain a first atlas and performing the continuous wavelet transform on the time-series flight data to obtain a second atlas includes: Performing the Gramian Angular Field (GAF) transform on the data of each dimension in the time-series flight data to obtain Gramian Angular Field images of each dimension; Performing a feature stacking operation on the Gramian Angular Field images of each dimension to obtain the first atlas; Performing the continuous wavelet transform on the data of each dimension in the time-series flight data to obtain continuous wavelet images of each dimension; Performing a feature stacking operation on the continuous wavelet images of each dimension to obtain the second atlas.

[0009] In some embodiments, performing feature extraction on the first atlas to obtain first initial features, performing feature extraction on the second atlas to obtain second initial features, and performing feature fusion on the first initial features and the second initial features to obtain fused atlas features includes: Performing feature dimensionality reduction operations on the first atlas and the second atlas respectively to obtain a first reduced-dimensional atlas and a second reduced-dimensional atlas; Performing feature extraction on the first reduced-dimensional atlas based on a first convolutional layer to obtain first initial features, and performing feature extraction on the second reduced-dimensional atlas based on a second convolutional layer to obtain second initial features; Performing a feature fusion operation on the first initial features and the second initial features based on a fully connected layer to obtain the fused atlas features.

[0010] In some embodiments, inputting the fused atlas features into a fault diagnosis model to obtain the fault diagnosis result of the target unmanned aerial vehicle output by the fault diagnosis model includes: Inputting the fused atlas features into a feature extraction network in the fault diagnosis model to obtain fault diagnosis features output by the feature extraction network; Inputting the fault diagnosis features into a feature classification layer in the fault diagnosis model to obtain the fault diagnosis result output by the feature classification layer; Among them, the feature extraction network includes a feature extraction module, a convolutional layer, and a fully connected layer connected in sequence; the feature extraction module includes a plurality of Ghost Bottleneck modules connected in sequence; the Ghost Bottleneck module includes an expansion layer and a compression layer connected in sequence; the output of the expansion layer is added to the input of the compression layer and used as the output of the Ghost Bottleneck module.

[0011] In some embodiments, the fault diagnosis model is trained based on the following steps: Obtain sample time-series flight data; the sample time-series flight data is determined based on the real flight data and / or simulation flight data of the target unmanned aerial vehicle. Crop the sample time-series flight data and determine the fault diagnosis result label of the sample time-series flight data. Based on the cross-entropy loss between the predicted value of the fault diagnosis result of the sample time-series flight data output by the initial model and the fault diagnosis result label, update the parameters of the initial model to obtain the fault diagnosis model; the initial model is built based on the Ghost network.

[0012] The present invention provides a drone fault diagnosis device based on a multi-dimensional feature map and a Ghost network, including: An acquisition unit, configured to acquire the time-series flight data of the target drone. A transformation unit, configured to perform Gram angular field transformation on the time-series flight data to obtain a first map, and perform continuous wavelet transformation on the time-series flight data to obtain a second map. An extraction unit, configured to extract first initial features from the first map, extract second initial features from the second map, and perform feature fusion on the first initial features and the second initial features to obtain fused map features. A diagnosis unit, configured to input the fused map features into a fault diagnosis model to obtain the fault diagnosis result of the target drone output by the fault diagnosis model; the fault diagnosis model is built based on the Ghost network.

[0013] The present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor implements the drone fault diagnosis method based on a multi-dimensional feature map and a Ghost network when executing the program.

[0014] The present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the drone fault diagnosis method based on a multi-dimensional feature map and a Ghost network is implemented.

[0015] The present invention provides a computer program product, including a computer program. When the computer program is executed by a processor, the drone fault diagnosis method based on a multi-dimensional feature map and a Ghost network is implemented.

[0016] The drone fault diagnosis method and device based on a multi-dimensional feature map and a Ghost network provided by the present invention obtain the sequential flight data of a target drone; perform Gram angular field transformation on the sequential flight data to obtain a first map, and perform continuous wavelet transformation on the sequential flight data to obtain a second map; extract features from the first map to obtain first initial features, extract features from the second map to obtain second initial features, and perform feature fusion on the first initial features and the second initial features to obtain fused map features; input the fused map features into a fault diagnosis model to obtain the fault diagnosis result of the target drone output by the fault diagnosis model; because Gram angular field transformation and continuous wavelet transformation are performed on the sequential flight data, and feature extraction and fusion are performed according to the transformed maps, the sequential features of each dimension of data in the sequential flight data are fully mined; because the fault diagnosis model is built based on a Ghost network, the generation process of redundant features is optimized, thereby improving the calculation efficiency of the model and the accuracy of drone fault diagnosis. Description of the Drawings

[0017] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present invention and used together with the specification to explain the principles of the present invention.

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a flowchart showing the drone fault diagnosis method provided by the present invention.

[0020] Figure 2 It is a structural diagram showing the Ghost Bottleneck module provided by the present invention.

[0021] Figure 3 It is a structural diagram showing the fault diagnosis model provided by the present invention.

[0022] Figure 4 It is a schematic flow chart of the UAV fault diagnosis method based on multi-dimensional feature maps and Ghost network provided by the present invention.

[0023] Figure 5 It is a schematic structural diagram of the UAV fault diagnosis device provided by the present invention.

[0024] Figure 6 It is a schematic structural diagram of the electronic device provided by the present invention. Detailed implementation manners

[0025] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the present invention are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units or modules does not have to be limited to those steps or units or modules clearly listed, but may include other steps or units or modules not clearly listed or inherent to these processes, methods, products or devices.

[0027] Although there are data-driven methods for UAV fault diagnosis in the related art, these methods cannot fully exploit the features of the data, resulting in a low accuracy of the results.

[0028] To solve the deficiencies of the related art, Figure 1 It is a schematic flow chart of the UAV fault diagnosis method provided by the present invention. As Figure 1 shown, the method includes step 110, step 120, step 130, and step 140.

[0029] Step 110: Obtain the time-series flight data of the target UAV.

[0030] Specifically, the execution subject of the UAV fault diagnosis method based on the multi-dimensional feature map and the Ghost network provided by the embodiments of the present invention is a UAV fault diagnosis device. This device can be implemented by software, such as a UAV fault diagnosis program; or by hardware, such as a flight controller that executes the UAV fault diagnosis method.

[0031] The target UAV refers to the UAV that needs to be fault diagnosed. UAVs are usually equipped with a variety of sensors (such as navigation devices, inertial measurement units, barometers, cameras, etc.), and these sensors can record the sequential flight data of the UAV in real time. The sequential flight data refers to various parameter data of the UAV during flight recorded in chronological order. These data usually include position information (such as longitude, latitude, altitude), speed, acceleration, attitude (pitch angle, roll angle, yaw angle), engine parameters (such as rotational speed), environmental data (temperature, pressure, wind speed), and other sensor data.

[0032] The sequential flight data is multi-dimensional data. The data of each dimension can represent a data type.

[0033] Step 120: Perform Gramian Angular Field transformation on the sequential flight data to obtain a first map, and perform continuous wavelet transform on the sequential flight data to obtain a second map.

[0034] Specifically, the methods for processing the sequential flight data include Gramian Angular Field (GAF) transformation and continuous wavelet transform (CWT). These two methods can respectively convert the sequential data into maps for further analysis or model training.

[0035] The Gramian Angular Field transformation is a method for converting time series data into images. Through polar coordinate transformation and the construction of the Gramian matrix, the time dependence and numerical information of the sequential flight data are encoded into the image. Performing the Gramian Angular Field transformation on the sequential flight data can obtain a first map.

[0036] The continuous wavelet transform is a time-frequency analysis method that can capture the local features of signals and is particularly suitable for processing non-stationary signals. Performing the continuous wavelet transform on the sequential flight data can obtain a second map.

[0037] The first map and the second map represent the sequential flight data in an image form, which is convenient for further analysis or for machine learning models.

[0038] Step 130: Extract features from the first spectrogram to obtain the first initial features, extract features from the second spectrogram to obtain the second initial features, and perform feature fusion on the first initial features and the second initial features to obtain the fused spectrogram features.

[0039] Specifically, shallow feature extraction can be performed on the first spectrogram and the second spectrogram respectively to obtain the first initial features and the second initial features. For example, a convolutional layer can be used for feature extraction.

[0040] After extracting the shallow features, feature fusion can be performed on the first initial features and the second initial features to obtain the fused spectrogram features.

[0041] The first initial features are extracted from the first spectrogram and can reflect the global structure and time dependence in the time-of-flight data, which is suitable for analyzing the overall trend and periodicity of the data. The second initial features are extracted from the second spectrogram and can reflect the time-frequency characteristics and local features in the time-of-flight data, which is suitable for analyzing the transient changes and frequency distribution of the signal.

[0042] Step 140: Input the fused spectrogram features into the fault diagnosis model to obtain the fault diagnosis result of the target UAV output by the fault diagnosis model; the fault diagnosis model is built based on the Ghost network.

[0043] Specifically, the Ghost network (also known as the Phantom network) can generate more feature maps through simple linear transformations. While reducing the computational complexity, it can generate sufficiently flexible feature representations to meet the requirements of different tasks. The fused spectrogram features can combine the advantages of the time domain and the frequency domain, improving the robustness, generalization ability, and classification performance of the model.

[0044] The fault diagnosis model can be built through the Ghost network. Input the fused spectrogram features into the fault diagnosis model, and the fault diagnosis result of the target UAV can be inferred by the fault diagnosis model.

[0045] The UAV fault diagnosis method based on the multi-dimensional feature map and the Ghost network provided by the embodiment of the present invention obtains the time-series flight data of the target UAV; performs Gram angle field transformation on the time-series flight data to obtain the first map, and performs continuous wavelet transformation on the time-series flight data to obtain the second map; extracts features from the first map to obtain the first initial features, extracts features from the second map to obtain the second initial features, and performs feature fusion on the first initial features and the second initial features to obtain the fused map features; inputs the fused map features into the fault diagnosis model to obtain the fault diagnosis result of the target UAV output by the fault diagnosis model; because Gram angle field transformation and continuous wavelet transformation are performed on the time-series flight data, and feature extraction and fusion are performed according to the transformed maps, the time-series features of each dimension of data in the time-series flight data are fully mined; because the fault diagnosis model is built based on the Ghost network, the generation process of redundant features is optimized, thereby improving the calculation efficiency of the model and the accuracy of UAV fault diagnosis.

[0046] It should be noted that each embodiment of the present invention can be freely combined, the order can be swapped, or each can be executed independently, and does not need to rely on or depend on a fixed execution order.

[0047] In some embodiments, before performing Gram angle field transformation on the time-series flight data to obtain the first map, and performing continuous wavelet transformation on the time-series flight data to obtain the second map, the method further includes: Performing a normalization operation on the data of each dimension in the time-series flight data.

[0048] Specifically, data normalization is an important step in data preprocessing, which scales the data to a specific range.

[0049] The time-series flight data may include data of multiple dimensions, and the data of each dimension can be represented by a separate column. Performing a normalization operation on the data of each dimension in the time-series flight data can be expressed by the formula:

[0050] where represents the value after normalization of the -th column in the time-series flight data, , represents the total dimension of the time-series flight data, represents the value of the -th column in the time-series flight data, represents the maximum value of the values of the -th column in the time-series flight data, represents the minimum value of the values of the -th column in the time-series flight data.

[0051] According to the above method, all dimensions in the time-series flight data can be normalized to obtain the normalized time-series flight data.

[0052] For the UAV fault diagnosis method based on the multi-dimensional feature map and the Ghost network provided by the embodiments of the present invention, normalizing the data before performing the Gram angular field transform and the continuous wavelet transform can significantly improve the stability, calculation efficiency and model performance of feature extraction, and at the same time facilitate the visualization and interpretation of the data.

[0053] In some embodiments, performing a Gram angular field transform on the time-series flight data to obtain a first map, and performing a continuous wavelet transform on the time-series flight data to obtain a second map, includes: Performing a Gram angular field transform on the data of each dimension in the time-series flight data to obtain Gram angular field images of each dimension; Performing a feature stacking operation on the Gram angular field images of each dimension to obtain a first map; Performing a continuous wavelet transform on the data of each dimension in the time-series flight data to obtain continuous wavelet images of each dimension; Performing a feature stacking operation on the continuous wavelet images of each dimension to obtain a second map.

[0054] Specifically, the time-series flight data can be divided into two parts. One part is used for the Gram angular field transform, and the other part is used for the continuous wavelet transform.

[0055] The Gram angular field is a way of image encoding using the Gram matrix. By calculating the linear correlation relationship of one-dimensional vectors through the Gram matrix, the time series is encoded into a two-dimensional image and the time dependence in the time series is retained. The data contained in the column of the normalized time-series flight data is ={ }, , represents the data in the th row in the th column. The th column has a total of rows, . The timestamp is encoded as the radius, the sequence value is encoded as the cosine angle, and the one-dimensional data is converted to the polar coordinate system through the following formula:

[0056] where, is the polar coordinate angle, is the radius, is the timestamp, is the time span, represents the The data in the row of the column. The encoded polar coordinate points and the one-dimensional data correspond to each other. Then, the Gram inner product is defined by the following formula:

[0057] where and are the data in the column and the and rows of the time-of-flight data sequence respectively. Then, the Gram matrix is obtained according to the definition formula, and the calculation formula is as follows:

[0058]

[0059] where is the Gram angular field image of the column (the th dimension). Finally, the Gram angular field images of all dimensions are stacked into the first atlas (Gram angular field atlas) according to the following formula, which is expressed as:

[0060] where is the first atlas, is the feature stacking operation.

[0061] Continuous wavelet transform is a time-frequency analysis method that can convert one-dimensional signal data that is time-varying and non-stationary into a two-dimensional time-frequency diagram containing time-frequency domain information. The specific operation is as follows: First, a column of time-of-flight data is converted into a wavelet image through the following calculation formula:

[0062] where is the one-dimensional time-frequency signal (the time-of-flight data of the column), is the translation factor, is the scaling factor, is the wavelet basis function. In the embodiments of the present invention, the Morlet wavelet is selected as the basis function, is the transformed two-dimensional wavelet image (continuous wavelet image). Then, the following formula is used to obtain the stacked second atlas (wavelet atlas):

[0063] where is the second atlas.

[0064] The UAV fault diagnosis method based on multi-dimensional feature maps and Ghost network provided by the embodiments of the present invention performs Gramian angular field transformation on time-series flight data to obtain a first map, and performs continuous wavelet transformation on time-series flight data to obtain a second map, fully mining the time-series features of each dimension data in the time-series flight data.

[0065] In some embodiments, feature extraction is performed on the first map to obtain first initial features, feature extraction is performed on the second map to obtain second initial features, and feature fusion is performed on the first initial features and the second initial features to obtain fused map features, including: Feature dimensionality reduction operations are respectively performed on the first map and the second map to obtain a first reduced-dimensional map and a second reduced-dimensional map; Based on a first convolutional layer, feature extraction is performed on the first reduced-dimensional map to obtain first initial features, and based on a second convolutional layer, feature extraction is performed on the second reduced-dimensional map to obtain second initial features; Based on a fully connected layer, feature fusion operation is performed on the first initial features and the second initial features to obtain fused map features.

[0066] Specifically, the principal component analysis method can be used to perform dimensionality reduction operations on the two maps respectively. First, according to the following formula, the centralized vector groups of the two feature maps are calculated:

[0067] Among them, represents the centralized vector group of the feature map, represents the feature map, including and two types, represents taking the mean value of each vector in the feature map , and the mean vector composed of taking the mean value of all column vectors. After that, the centralized vector group is transposed and multiplied by the centralized vector group to obtain the covariance matrix , and according to the following formula, the eigenvalues of the covariance matrix are calculated:

[0068] Among them, represents the determinant operation, represents the eigenvalue of the covariance matrix, represents the identity matrix, represents the covariance matrix. According to the following formula, the eigenvectors of the covariance matrix are calculated, and the first 3 eigenvectors are combined to obtain the transformation matrix:

[0069] Among them, Represents the eigenvector of the covariance matrix. Each vector in the vector group is successively multiplied by the transformation matrix, and the resulting 3D matrix is used as the three principal component images of the feature map, respectively obtaining the dimensionality-reduced feature map for neural network model training, including the first dimensionality-reduced map and the second dimensionality-reduced map .

[0070] The first convolutional layer is used to extract features from the first dimensionality-reduced map to obtain the first initial feature, and the second convolutional layer is used to extract features from the second dimensionality-reduced map to obtain the second initial feature. Both the first initial feature and the second initial feature are shallow features. The first convolutional layer and the second convolutional layer can use the same-sized convolutional kernels, for example, it can be (input channels, output channels, convolutional window length, convolutional window width). The fully connected layer is used to perform feature fusion operations on the first initial feature and the second initial feature to obtain the fused map feature.

[0071] The UAV fault diagnosis method based on multi-dimensional feature maps and Ghost network provided by the embodiments of the present invention extracts and fuses features of the first map and the second map, and can combine the advantages of the time domain and the frequency domain to improve the robustness, generalization ability and classification performance of the model.

[0072] In some embodiments, the fused map feature is input into the fault diagnosis model to obtain the fault diagnosis result of the target UAV output by the fault diagnosis model, including: The fused map feature is input into the feature extraction network in the fault diagnosis model to obtain the fault diagnosis feature output by the feature extraction network; The fault diagnosis feature is input into the feature classification layer in the fault diagnosis model to obtain the fault diagnosis result output by the feature classification layer; Among them, the feature extraction network includes a feature extraction module, a convolutional layer and a fully connected layer connected in sequence; the feature extraction module includes a plurality of Ghost Bottleneck modules connected in sequence; the Ghost Bottleneck module includes an expansion layer and a compression layer connected in sequence; the input of the expansion layer and the output of the compression layer are added together as the output of the Ghost Bottleneck module.

[0073] Specifically, Figure 2 is the structural schematic diagram of the Ghost Bottleneck module provided by the present invention. As Figure 2 shown, the Ghost Bottleneck module includes an expansion layer and a compression layer connected in sequence.

[0074] Both the expansion layer and the compression layer can be built through the Ghost module. The Ghost module can generate phantom features and improve the network calculation efficiency. For the input data where \(c\), \(k\), and \(q\) are the dimension, length, and width of the input data respectively. The Ghost module first uses a convolution kernel to perform a normal convolution operation on the input data . The output feature map is , and are the length and width of the output features, is the dimension of the output features, and the width and height of the convolution kernel are both . Then, a linear transformation is performed on the output feature map according to the following formula to generate a series of ghost features as follows:

[0075]

[0076] where is the data in the output ghost feature , is the -th dimension in the output feature map , represents the linear transformation function, indicates the type of linear transformation, that is, for the existing feature maps, each will be sequentially combined with \(s\) linear transformation operations to obtain \(s\) types of ghost features, and the final feature dimension is . Finally, and are concatenated to obtain the output of the Ghost module.

[0077] In the Ghost Bottleneck module, the first Ghost module that the input data passes through is the expansion layer, which is used to increase the number of channels of the features. The ratio of the number of input channels to the number of output channels is the expansion ratio. The second Ghost module is used to reduce the number of channels of the features so that the number of output features is the same as the input, which is called the compression layer. Finally, the input and output are added in the way of a shortcut connection. Batch Norm and ReLU are batch normalization processing and activation function respectively. That is, the output of the expansion layer is added to the input of the compression layer as the output of the Ghost Bottleneck module.

[0078] Figure 3 is a schematic structural diagram of the fault diagnosis model provided by the present invention, as shown in Figure 3As shown in the figure, the fault diagnosis model includes a feature extraction network and a feature classification layer. The feature extraction network includes a feature extraction module, a convolutional layer, and a fully connected layer connected in sequence. The feature extraction module includes a plurality of Ghost Bottleneck modules connected in sequence. For example, the feature extraction model may include 6 Ghost Bottleneck modules. The size of the convolutional layer can be (input channels, output channels, convolutional window length, convolutional window width). The feature classification layer can be a softmax classifier.

[0079] Input the fused spectral features into the feature extraction network in the fault diagnosis model to obtain the fault diagnosis features output by the feature extraction network; the fault diagnosis features are deep features. Then input the fault diagnosis features into the feature classification layer in the fault diagnosis model to obtain the fault diagnosis result output by the feature classification layer.

[0080] The UAV fault diagnosis method based on multi-dimensional feature spectra and Ghost network provided by the embodiments of the present invention uses the Ghost module to improve the convolutional neural network, optimizes the generation process of redundant features, thereby improving the computational efficiency of the model, and uses the Ghost Bottleneck module to extract the deep features of the fused spectra, improving the computational accuracy of the model.

[0081] In some embodiments, the fault diagnosis model is trained based on the following steps: Obtain sample time-series flight data; the sample time-series flight data is determined based on the real flight data and / or simulation flight data of the target UAV; Crop the sample time-series flight data and determine the fault diagnosis result label of the sample time-series flight data; Based on the cross-entropy loss between the predicted value of the fault diagnosis result of the sample time-series flight data output by the initial model and the fault diagnosis result label, update the parameters of the initial model to obtain the fault diagnosis model; the initial model is built based on the Ghost network.

[0082] Specifically, after building the initial model according to the Ghost network, the real flight data and / or simulation flight data of the target UAV can be collected as the sample time-series flight data. Then crop the sample time-series flight data and determine the fault diagnosis result label of the sample time-series flight data.

[0083] For example, the original sample time-series flight data is time-series flight data with 48 features. After cropping the data that is meaningless for analysis, such as communication flag bits and UAV connection flag bits, 35-dimensional features remain. Finally, the data is labeled with various fault diagnosis result labels, including propeller faults, gyroscope faults, accelerometer faults, etc. If there is no fault, it is determined as a no-fault label.

[0084] Input the sample time-series flight data into the initial model for training. The predicted value of the fault diagnosis result of the sample time-series flight data output by the initial model is used to update the parameters of the initial model according to the cross-entropy loss between the predicted value of the fault diagnosis result and the fault diagnosis result label, and a fault diagnosis model is obtained.

[0085] For example, use the cross-entropy loss function to train the built model, and set the input Gram angular field spectrogram and continuous wavelet spectrogram size to (spectrum width, spectrum height, number of channels), the number of model iterations (epoch) is 150, the learning rate is 0.001, the batch size is 100, and the Adam optimizer is selected for the model.

[0086] After the training is completed, the trained model (fault diagnosis model) can be used to test and classify the UAV fault data.

[0087] The UAV fault diagnosis method based on multi-dimensional feature spectrograms and Ghost network provided by the embodiments of the present invention builds an initial model according to the Ghost network, and obtains a fault diagnosis model through training, realizing the simultaneous diagnosis of multiple types of faults for the complex system of the UAV, and improving the accuracy and efficiency of fault diagnosis.

[0088] Based on the above various embodiments, Figure 4 is a schematic flowchart of the UAV fault diagnosis method based on multi-dimensional feature spectrograms and Ghost network provided by the present invention. As Figure 4 shown, the method includes step 410, step 420, step 430, step 440, step 450 and step 460.

[0089] Step 410: Obtain UAV data and perform cropping and sorting.

[0090] Obtain real or simulated UAV data containing multiple feature dimensions, crop the invalid features of the data and unify the number of feature dimensions of each data, and label various types of faults or no faults.

[0091] Step 420: Data normalization.

[0092] Normalize the sorted UAV data according to each feature dimension to obtain the normalized UAV fault data.

[0093] Step 430: Obtain the Gram angular field spectrogram and the wavelet spectrogram.

[0094] Divide the normalized drone data into two parts. For one part, convert each dimension of the data into a two-dimensional Gram angular field image through Gram angular field transformation, and then stack the Gram angular field images of all dimensions into a Gram angular field atlas.

[0095] For the other part, convert each dimension of the data into a two-dimensional wavelet image through wavelet transformation, and then stack the wavelet images of all dimensions into a wavelet atlas.

[0096] Step 440: Dimension reduction of atlas data.

[0097] Use principal component analysis to perform dimension reduction operations on the two atlases respectively to obtain the dimension-reduced atlas data.

[0098] Step 450: Construct and train a Ghost neural network model.

[0099] Use the Ghost module to construct Ghost Bottleneck to simplify the generation of redundant features, and then use GhostBottleneck to build a Ghost network.

[0100] For the dimension-reduced Gram angular field atlas and wavelet atlas data, extract their respective shallow features using a convolutional layer, then fuse the two shallow atlas features through a fully connected layer to obtain fused atlas features, and send the fused atlas features into the built Ghost network to extract deep features. Finally, use a softmax classifier to output the fault diagnosis result to complete the model construction. Use the cross-entropy loss function to train the built model.

[0101] Step 460: Use the trained model to test and classify the drone fault data.

[0102] The mixed dataset can be used for testing to implement a simulation experiment. The dataset can include software simulation data and real aircraft simulation data. The software simulation data is generated by a drone fault simulation platform for various flight postures of a quadrotor drone. The real aircraft simulation data is obtained after the drone simulates various fault flights in an experimental environment.

[0103] The overall mixed dataset includes five types of data: gyroscope fault, accelerometer fault, propeller fault, motor fault, and no fault under three states of the aircraft: uniform motion, hovering, and circling.

[0104] The present invention is compared with a fault diagnosis scheme based on a convolutional neural network (CNN) and a fault diagnosis method based on a long short-term memory network (LSTM). The results of the simulation experiment of the present invention can be objectively evaluated by three metrics. The first evaluation metric is the overall accuracy (OA), which represents the proportion of the number of samples correctly classified by the classifier used in each method to all samples. The larger this value, the better the diagnostic effect. The second evaluation metric is the average accuracy (AA), which represents the average of the classification accuracies of each class. The larger this value, the better the diagnostic effect. The third evaluation metric is the Kappa coefficient, which represents different weights in the confusion matrix. The larger this value, the better the diagnostic effect. The numerical results of the method of the present invention and the two comparison methods under the three metrics, as well as the classification diagnosis results of various faults or no faults, are shown in Table 1.

[0105] Table 1 Quantitative analysis list of classification results of each method

[0106] It can be seen from the comparison that: the embodiments of the present invention achieve the best results in terms of OA, AA, and Kappa coefficient. The methods based on CNN and LSTM decrease by 8.96% and 8.01% respectively compared with the present invention in terms of OA. At the same time, the method of the present invention also achieves the best results in the classification diagnosis results of various faults or no faults. It shows that rich features of high-dimensional UAV time-series data can be extracted through the feature map, and the constructed Ghost network can improve the diagnostic accuracy.

[0107] The device provided by the embodiments of the present invention will be described below. The device described below can be correspondingly referred to the method described above.

[0108] Figure 5 is a schematic structural diagram of a UAV fault diagnosis device provided by the present invention, as Figure 5 shown. The device includes: An acquisition unit 510, configured to acquire the time-series flight data of the target UAV; A transformation unit 520, configured to perform Gramian angular field transformation on the time-series flight data to obtain a first map, and perform continuous wavelet transformation on the time-series flight data to obtain a second map; An extraction unit 530, configured to extract first initial features from the first map, extract second initial features from the second map, and perform feature fusion on the first initial features and the second initial features to obtain fused map features; A diagnosis unit 540, configured to input the fused map features into a fault diagnosis model to obtain a fault diagnosis result of the target UAV output by the fault diagnosis model; the fault diagnosis model is constructed based on the Ghost network.

[0109] The UAV fault diagnosis device based on the multi-dimensional feature map and the Ghost network provided by the embodiment of the present invention acquires the time-series flight data of the target UAV; performs Gram angular field transformation on the time-series flight data to obtain a first map, and performs continuous wavelet transformation on the time-series flight data to obtain a second map; extracts features from the first map to obtain first initial features, extracts features from the second map to obtain second initial features, and performs feature fusion on the first initial features and the second initial features to obtain fused map features; inputs the fused map features into a fault diagnosis model to obtain the fault diagnosis result of the target UAV output by the fault diagnosis model; because Gram angular field transformation and continuous wavelet transformation are performed on the time-series flight data, and feature extraction and fusion are performed according to the transformed maps, the time-series features of each dimension of data in the time-series flight data are fully mined; because the fault diagnosis model is built based on the Ghost network, the generation process of redundant features is optimized, thereby improving the calculation efficiency of the model and the accuracy of UAV fault diagnosis.

[0110] Figure 6 is a schematic structural diagram of the electronic device provided by the present invention, as Figure 6 shown, the electronic device may include: a processor (Processor) 610, a communication interface (Communications Interface) 620, a memory (Memory) 630, and a communication bus (Communications Bus) 640. Among them, the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logical commands in the memory 630 to execute the methods described in the above embodiments, for example: acquire the time-series flight data of the target UAV; perform Gram angular field transformation on the time-series flight data to obtain a first map, and perform continuous wavelet transformation on the time-series flight data to obtain a second map; extract features from the first map to obtain first initial features, extract features from the second map to obtain second initial features, and perform feature fusion on the first initial features and the second initial features to obtain fused map features; input the fused map features into a fault diagnosis model to obtain the fault diagnosis result of the target UAV output by the fault diagnosis model; the fault diagnosis model is built based on the Ghost network.

[0111] In addition, when the logic commands in the above-mentioned memory can be implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0112] The processor in the electronic device provided by the embodiments of the present invention can call the logic instructions in the memory to implement the above method. The specific implementation manner is the same as that of the foregoing method embodiment, and the same beneficial effects can be achieved, which will not be elaborated here.

[0113] The embodiments of the present invention also provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the methods provided in the above various embodiments.

[0114] The specific implementation manner is the same as that of the foregoing method embodiment, and the same beneficial effects can be achieved, which will not be elaborated here.

[0115] The embodiments of the present invention provide a computer program product, including a computer program. When the computer program is executed by a processor, it implements the method as described above.

[0116] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative efforts.

[0117] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A UAV fault diagnosis method based on a multi-dimensional feature map and a Ghost network, characterized in that Including: Obtain the sequential flight data of the target UAV; Perform Gram Angular Field Transformation on the sequential flight data to obtain a first spectrogram, and perform continuous wavelet transformation on the sequential flight data to obtain a second spectrogram; Extract features from the first spectrogram to obtain first initial features, extract features from the second spectrogram to obtain second initial features, and perform feature fusion on the first initial features and the second initial features to obtain fused spectrogram features; Input the fused spectrogram features into a fault diagnosis model to obtain a fault diagnosis result of the target UAV output by the fault diagnosis model; the fault diagnosis model is built based on the Ghost network.

2. The drone fault diagnosis method based on the multi-dimensional feature map and the Ghost network according to claim 1, wherein, Before performing Gram Angular Field Transformation on the sequential flight data to obtain a first spectrogram and performing continuous wavelet transformation on the sequential flight data to obtain a second spectrogram, the method further includes: Perform a normalization operation on the data of each dimension in the sequential flight data.

3. The drone fault diagnosis method based on the multi-dimensional feature map and the Ghost network according to claim 1, wherein Performing Gram Angular Field Transformation on the sequential flight data to obtain a first spectrogram and performing continuous wavelet transformation on the sequential flight data to obtain a second spectrogram includes: Perform Gram Angular Field Transformation on the data of each dimension in the sequential flight data to obtain Gram Angular Field images of each dimension; Perform a feature stacking operation on the Gram Angular Field images of each dimension to obtain the first spectrogram; Perform continuous wavelet transformation on the data of each dimension in the sequential flight data to obtain continuous wavelet images of each dimension; Perform a feature stacking operation on the continuous wavelet images of each dimension to obtain the second spectrogram.

4. The method for diagnosing UAV faults based on a multi-dimensional feature map and a Ghost network according to claim 1, wherein Extracting features from the first spectrogram to obtain first initial features, extracting features from the second spectrogram to obtain second initial features, and performing feature fusion on the first initial features and the second initial features to obtain fused spectrogram features includes: Perform feature dimensionality reduction operations on the first spectrogram and the second spectrogram respectively to obtain a first dimensionality-reduced spectrogram and a second dimensionality-reduced spectrogram; Extract first initial features from the first dimensionality-reduced spectrogram based on a first convolutional layer, and extract second initial features from the second dimensionality-reduced spectrogram based on a second convolutional layer; Perform a feature fusion operation on the first initial features and the second initial features based on a fully connected layer to obtain the fused spectrogram features.

5. The method for diagnosing UAV faults based on a multi-dimensional feature map and a Ghost network according to claim 1, wherein Inputting the fused spectrogram features into a fault diagnosis model to obtain a fault diagnosis result of the target UAV output by the fault diagnosis model includes: Input the fused spectrogram features into the feature extraction network in the fault diagnosis model to obtain fault diagnosis features output by the feature extraction network; Input the fault diagnosis features into the feature classification layer in the fault diagnosis model to obtain the fault diagnosis result output by the feature classification layer; Among them, the feature extraction network includes a feature extraction module, a convolutional layer, and a fully connected layer connected in sequence; the feature extraction module includes a plurality of Ghost Bottleneck modules connected in sequence; the Ghost Bottleneck module includes an expansion layer and a compression layer connected in sequence; the output of the compression layer is added to the input of the expansion layer and used as the output of the Ghost Bottleneck module.

6. The drone fault diagnosis method based on the multi-dimensional feature map and the Ghost network according to claim 1, characterized in that The fault diagnosis model is obtained by training based on the following steps: Obtain sample time-series flight data; the sample time-series flight data is determined based on the real flight data and / or simulation flight data of the target unmanned aerial vehicle; Crop the sample time-series flight data and determine the fault diagnosis result label of the sample time-series flight data; Based on the cross-entropy loss between the predicted value of the fault diagnosis result of the sample time-series flight data output by the initial model and the fault diagnosis result label, update the parameters of the initial model to obtain the fault diagnosis model; the initial model is built based on the Ghost network.

7. An unmanned aerial vehicle fault diagnosis device based on a multi-dimensional feature map and a Ghost network, characterized in that, It includes: An acquisition unit for acquiring the time-series flight data of the target unmanned aerial vehicle; A transformation unit for performing Gram angular field transformation on the time-series flight data to obtain a first spectrogram, and performing continuous wavelet transformation on the time-series flight data to obtain a second spectrogram; An extraction unit for extracting first initial features from the first spectrogram, extracting second initial features from the second spectrogram, and performing feature fusion on the first initial features and the second initial features to obtain fused spectrogram features; A diagnosis unit for inputting the fused spectrogram features into a fault diagnosis model to obtain the fault diagnosis result of the target unmanned aerial vehicle output by the fault diagnosis model; the fault diagnosis model is built based on the Ghost network.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the unmanned aerial vehicle fault diagnosis method based on multi-dimensional feature spectrograms and the Ghost network according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the unmanned aerial vehicle fault diagnosis method based on multi-dimensional feature spectrograms and the Ghost network according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the unmanned aerial vehicle fault diagnosis method based on multi-dimensional feature spectrograms and the Ghost network according to any one of claims 1 to 6.

Images