UAV Health State Evaluation Method Based on Enhanced Spatiotemporal Graph Neural Network

Abstract

The present invention discloses a method for evaluating the health state of an unmanned aerial vehicle (UAV) based on an enhanced spatio-temporal graph neural network, including: collecting the flight parameters of the UAV to be evaluated, and inputting the flight parameters and the adjacency matrix at each moment into the trained spatio-temporal graph neural network, and outputting the prediction label corresponding to each moment through the spatio-temporal graph neural network to obtain the category of the health state of the UAV; wherein, when the spatio-temporal graph neural network is trained, firstly, a flight parameter data set of the UAV in different health states is obtained and a new data set is generated by using a denoising autoencoder generative adversarial network; kinematic and dynamic modeling is performed on the flight parameters of the UAV to obtain a correlation relationship model among the flight parameters; the flight parameters are used as nodes, and the connection relationship between the nodes is determined based on the correlation relationship model, and an adjacency matrix is constructed as a spatio-temporal graph; the samples in the new data set and the adjacency matrix are used as spatio-temporal data to train the spatio-temporal graph neural network.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle (UAV) health status assessment, and particularly to a UAV health status assessment method based on an enhanced spatio-temporal graph neural network. Background Art

[0002] The rapid development of UAV technology has led to its increasingly widespread application in fields such as logistics, agriculture, and surveying and mapping. However, due to the complex and variable working environment of UAVs and their high-load operation, the failure rate of UAVs has gradually increased, affecting their stability and reliability. Therefore, how to monitor the health status of UAVs in real time and ensure their safety and reliability during flight has become an important research direction in the field of UAV health management.

[0003] Traditional UAV health status monitoring methods mainly rely on shallow machine learning methods and statistical modeling methods. These methods usually perform time series analysis on sensor data or use statistical models for evaluation, and it is difficult to effectively capture the spatio-temporal correlations and complex dynamic changes among various flight parameters of UAVs. Summary of the Invention

[0004] The purpose of the present invention is to provide a UAV health status assessment method based on an enhanced spatio-temporal graph neural network, which can better capture the spatio-temporal correlations of various flight parameters of UAVs, thereby improving the accuracy and reliability of health status assessment.

[0005] To achieve the above task, the present invention adopts the following technical solutions:

[0006] A UAV health status assessment method based on an enhanced spatio-temporal graph neural network, comprising:

[0007] Collect the flight parameters of the UAV to be evaluated, and input the flight parameters and the adjacency matrix at each moment into the trained spatio-temporal graph neural network. Through the spatio-temporal graph neural network, output the predicted label corresponding to each moment to obtain the category of the health status of the UAV.

[0008] Wherein, when the spatio-temporal graph neural network is being trained, first obtain the flight parameter dataset of the UAV in different health states, preprocess the flight parameter dataset to obtain the input dataset, and then input the input dataset and random noise into the denoising autoencoder generative adversarial network for data cleaning and data augmentation to obtain a new dataset.

[0009] Model the flight parameters of the UAV based on kinematics and dynamics to obtain the correlation relationship model among the flight parameters; use the flight parameters as nodes, and determine the connection relationship between the nodes based on the correlation relationship model, thereby constructing an adjacency matrix as a spatio-temporal graph; use the samples and the adjacency matrix in the new dataset as spatio-temporal data to train the spatio-temporal graph neural network.

[0010] Further, the flight parameter dataset contains multiple flight parameters of the UAV; multiple flight parameters collected at the same moment are used as a sample, and the category of the health state of the UAV when the flight parameters are collected is used as the label of the sample.

[0011] Further, standard deviation normalization method is used for preprocessing the flight parameter dataset.

[0012] Further, the denoising autoencoder generative adversarial network includes a first encoder, a decoder, a second encoder, and a discriminator, where:

[0013] The first encoder includes a convolutional layer, a batch normalization layer, an activation function layer, and a fully connected layer connected in sequence; samples in the input dataset are input into the first encoder after adding random noise through a noise reduction mechanism, and a compact feature representation matrix is output;

[0014] The decoder includes a connection layer, a transposed convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; the compact feature representation matrix is processed by the decoder to obtain a generated dataset, which contains new samples generated by the decoder; the first encoder and the decoder form a denoising autoencoder;

[0015] The structure of the second encoder is the same as that of the first encoder, and is used to learn and generate a latent representation matrix from the generated dataset; the first encoder, the decoder, and the second encoder form a generative adversarial network;

[0016] The structure of the discriminator is basically the same as that of the second encoder, except that a binary classifier is added after the fully connected layer of the discriminator to determine the probability that the sample comes from the input dataset or the generated dataset;

[0017] A total loss function is set, and the total loss function includes the reconstruction loss of the denoising autoencoder, the reconstruction losses of the first encoder and the second encoder, and the adversarial loss of the generative adversarial network.

[0018] Further, the reconstruction loss of the denoising autoencoder measures the reconstruction effect between the nth sample in the input dataset and the nth sample generated in the generated dataset by mean square error, which is expressed as where

[0019] The reconstruction losses of the first encoder and the second encoder are expressed as ; where represents the latent representation matrix the number of latent representations in is the compact feature representation matrix the th compact feature representation in is the latent representation matrix the th latent representation in

[0020] The adversarial loss of the generative adversarial network is expressed as , where represents the processing process expression of the discriminator without the binary classifier for the generated dataset and the input dataset , represents the first encoder and the decoder for the input dataset , is the L2 norm.

[0021] Furthermore, taking flight parameters as nodes, the connection relationships between nodes are determined based on the association relationship model, and thus an adjacency matrix is constructed as a spatio-temporal graph, including:

[0022] Based on the association relationship model, it is determined whether there is a connection relationship between nodes. There is an edge between nodes with a connection relationship, and the value of the corresponding element in the adjacency matrix is 1; there is no edge between nodes without a connection relationship, and the value of the corresponding element in the adjacency matrix is 0, thereby obtaining the constructed adjacency matrix as a spatio-temporal graph; the rows and columns of the adjacency matrix correspond one-to-one with the flight parameters; it is considered that there is an association relationship between each node and itself.

[0023] Furthermore, the spatio-temporal graph neural network includes two layers of spatio-temporal graph convolution modules; each layer of spatio-temporal graph convolution module includes a gated convolutional layer, a graph convolutional layer, a gated convolutional layer, a batch normalization layer, and an activation function layer connected in sequence;

[0024] For each sample in the new dataset, the sample and the adjacency matrix are input into the spatio-temporal graph neural network as a set of spatio-temporal data; the input spatio-temporal data first passes through the first layer of spatio-temporal graph convolution module to obtain the first aggregated feature; the first aggregated feature is input into the second layer of spatio-temporal graph convolution module to obtain the second aggregated feature; the second aggregated feature is then processed by a gated convolutional layer, and the output feature enters the global average pooling layer; the output of the global average pooling layer is used as the input of the fully connected layer, and finally the health state is evaluated through the fully connected layer to obtain the prediction label, that is, the category of the health state of the UAV.

[0025] Furthermore, the total loss function of the spatio-temporal graph neural network is the weighted sum of the interactive circle loss and the cross-entropy loss, which is expressed as , represents the cross-entropy loss, represents the interactive circle loss, is the weight coefficient of the interactive circle loss.

[0026] Furthermore, the expression of the interactive circle loss is as follows:

[0027] ;

[0028] wherein, respectively represent the th positive sample, the total number of positive samples, the th negative sample and the total number of negative samples; and are the decision boundaries of positive samples and negative samples respectively; and represent the weights of positive samples and negative samples respectively, represents the margin parameter, represents the scale parameter, represents the similarity of positive sample pairs, represents the similarity of negative sample pairs; wherein, for the current sample in the new dataset, if other samples in the new dataset have the same class label as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current sample is paired with positive samples pairwise to form positive sample pairs; the current sample is paired with negative samples pairwise to form negative sample pairs.

[0029] A drone health status evaluation device includes a processor, a memory, and a computer program stored in the memory; when the processor executes the computer program, the drone health status evaluation method based on the enhanced spatio-temporal graph neural network is implemented.

[0030] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the drone health status evaluation method based on the enhanced spatio-temporal graph neural network is implemented.

[0031] Compared with the prior art, the present invention has the following technical features:

[0032] 1. The method of the present invention effectively captures the spatial correlation information and temporal dynamic characteristics among various flight parameters during the complex flight of the drone, thereby accurately evaluating the health status of the drone, timely discovering potential fault risks, and being of great significance to the healthy operation of the drone.

[0033] 2. The present invention adopts data cleaning and data augmentation techniques based on a denoising autoencoder generative adversarial network. For complex multi-modal flight parameters, it can effectively process the noise and abnormal data generated during the flight of the unmanned aerial vehicle (UAV), improving the data quality. By generating new and diverse flight data samples, the input data set is expanded to solve the problem of UAV health state assessment under limited data.

[0034] 3. The present invention adopts an interactive circle loss function, which solves the problem that the traditional loss function is easily affected by class imbalance or data scarcity in the case of fewer samples, resulting in the model being unable to accurately identify the subtle differences in different health states. Training a spatio-temporal graph neural network using the interactive circle loss function achieves high accuracy and stability, with strong practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 is a schematic flowchart of the method of the present invention;

[0036] Figure 2 is a schematic diagram of the denoising autoencoder generative adversarial network in the present invention;

[0037] Figure 3 is the structure diagram of the spatio-temporal graph neural network constructed by the present invention;

[0038] Figure 4 is a schematic diagram of the UAV structure in an embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of the adjacency matrix constructed using flight parameters in an embodiment of the present invention;

[0040] Figure 6 is a schematic diagram of the health state assessment result in an embodiment of the present invention;

[0041] Figure 7 is a schematic diagram of the health state assessment confusion matrix in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] The flight parameters of the UAV, as important indicators for monitoring the flight state of the UAV, cover the speed, acceleration, attitude angles (such as pitch angle, roll angle, yaw angle), flight altitude, direction, thrust, flight time, GPS positioning data, etc. during the flight of the UAV; these data are of great significance in real-time monitoring of the UAV state, optimizing flight control, assessing health state, and predicting potential faults.

[0043] Accurately and fully utilizing the obtained flight parameters of the unmanned aerial vehicle (UAV) can achieve real-time monitoring and precise evaluation of the UAV's flight state, which helps to identify potential faults in advance and perform predictive maintenance. Through in-depth analysis of the flight parameters, real-time monitoring and health assessment of the UAV's performance can be realized, potential faults and anomalies can be identified in advance, ensuring the safety, stability, and reliability of the flight mission. At the same time, flight control strategies can be optimized, flight efficiency can be improved, flight paths can be adjusted, task execution can be optimized, energy consumption can be reduced, and it can promote the development of UAV health monitoring technology towards a higher level of intelligence and automation.

[0044] Since flight parameters usually have characteristics such as high-dimensionality, non-linearity, and temporality, and are affected by external environmental factors, simple machine learning models and statistical models are difficult to capture the complex correlation relationships between parameters. In addition, problems such as noise, outliers, and insufficient samples in flight data further pose challenges to the health state assessment of UAVs. Therefore, adopting more advanced deep learning models, especially technologies based on spatio-temporal graph neural networks and generative adversarial networks, can better mine the potential patterns and dynamic changes in the acquired data, effectively improving the accuracy and robustness of health state assessment. The model provided by the present invention can accurately predict the health state of the UAV under complex flight tasks and limited sample conditions, providing more reliable fault warnings and maintenance support.

[0045] The present invention provides a method for evaluating the health state of a UAV based on an enhanced spatio-temporal graph neural network, as Figure 1 shown, including:

[0046] Collect the flight parameters of the UAV to be evaluated, and input the flight parameters and the adjacency matrix at each moment into the trained spatio-temporal graph neural network. Through the spatio-temporal graph neural network, output the prediction label corresponding to each moment to obtain the category of the health state of the UAV;

[0047] Among them, when the spatio-temporal graph neural network is trained, first obtain the flight parameter data set of the UAV in different health states, preprocess the flight parameter data set to obtain the input data set, and then input the input data set and random noise into the denoising autoencoder generative adversarial network for data cleaning and data augmentation to obtain a new data set;

[0048] Model the flight parameters of the UAV based on kinematics and dynamics to obtain the correlation relationship model between each flight parameter. Take the flight parameters as nodes, and determine the connection relationship between nodes based on the correlation relationship model, so as to construct an adjacency matrix as a spatio-temporal graph. Use the samples and adjacency matrix in the new data set as spatio-temporal data to train the spatio-temporal graph neural network.

[0049] The following will explain the specific implementation process of the present invention in detail with reference to the accompanying drawings.

[0050] Step 1, construct a flight parameter dataset of the drone under different health states ; where n represents the number of flight parameters; represents the th flight parameter collected, .

[0051] In one embodiment, the flight parameters include the motor speeds of six rotors during the flight of the drone , roll angle , pitch angle , yaw angle , roll angular velocity , pitch angular velocity , yaw angular velocity , the acceleration of the drone on the axis , roll angular acceleration , pitch angular acceleration , yaw angular acceleration , that is, n = 18; Take the n flight parameters collected at the same moment as a sample, and use the category of the health state of the drone when collecting the flight parameters as the label of the sample; where the category of the health state is given by expert judgment.

[0052] Step 2, process the flight parameters in the flight parameter dataset using the standard deviation normalization method to obtain the input dataset as , where represents the result of standard deviation normalization of the flight parameter .

[0053] Among them, the standard deviation normalization method refers to, for each flight parameter in the flight parameter dataset, calculating the maximum value mean and standard deviation at different moments, and then subtracting the maximum value mean from each moment's flight parameter and multiplying by the standard deviation to obtain the standard deviation normalization result of the flight parameter at the corresponding moment; Complete the standard deviation normalization processing of each flight parameter in turn to obtain the input dataset , where each row of data corresponds to the flight parameters at each moment, which is a sample after standard deviation normalization; This standard deviation normalization method can keep the distribution shape of the data unchanged.

[0054] Step 3, input the dataset Together with the random noise N as the input, the denoising autoencoder generative adversarial network is used to perform data cleaning and data augmentation on the samples in the dataset. At the same time, by combining the random noise with the features of the samples, new and realistic samples are generated. After the denoising autoencoder generative adversarial network processes all the samples, a new dataset is obtained. ; where represents the flight parameters The result after being processed by the denoising autoencoder generative adversarial network.

[0055] Denote the number of samples before processing as m. The denoising autoencoder generative adversarial network will generate a new sample for each sample. Therefore, the number of samples after processing is 2m.

[0056] Such as Figure 2 shown, the specific design of the denoising autoencoder generative adversarial network is as follows:

[0057] Step 31, build the first encoder , including a convolutional layer, a batch normalization (BatchNormalization, BN) layer, a rectified linear unit (ReLU) activation function layer, and a fully connected layer connected in sequence; among them, a noise reduction mechanism (proposed by Vincent et al. in 2008) is set in the convolutional layer to learn robust features from the samples with added random noise N.

[0058] The samples in the input dataset are added with random noise N through the noise reduction mechanism, so that the first encoder has the ability to extract effective features from the interference information. Then the samples pass through the convolutional layer to extract key information features in the local space or time. The extracted key information features are processed by the batch normalization layer to alleviate the internal covariate shift, accelerate the convergence speed, and enhance the model stability. Then, through the activation function layer, the output result of the batch normalization layer is enhanced in nonlinear modeling ability, improving the model's ability to express complex relationships. Finally, through the fully connected layer, the deep features are compressed and mapped to output a potential compact feature representation matrix , thus providing a high-quality feature basis for subsequent modeling and discrimination tasks; the whole process is expressed as: .

[0059] Step 32, build the decoder , consisting of a connection layer, a transposed convolutional layer, a batch normalization layer, and a rectified linear unit (ReLU) activation function layer connected in sequence, used to reconstruct the compact feature representation matrix into a generated dataset with the same dimension as the input dataset , and the whole process is expressed as: ; among them, generate a dataset which is the newly generated sample.

[0060] In the decoder the connection layer adjusts the compact feature representation matrix to a shape suitable for deconvolution operations; then, the original spatial or temporal structure information is gradually restored through the deconvolution layer to achieve the reduction from low-dimensional features to high-dimensional representations; next, the batch normalization layer normalizes the output of the deconvolution layer to eliminate the shift of the activation value distribution and improve the training stability; finally, the activation function layer introduces non-linearity to enhance the model's expressive ability and outputs the final generated dataset ; the goal of the whole process is to restore the input dataset as precisely as possible so as to achieve effective feature decoding and information retention.

[0061] The first encoder mentioned above and the decoder constitute a denoising autoencoder.

[0062] Step 33, build a second encoder , whose structure is the same as that of the first encoder and is used to learn and generate a latent representation matrix from the generated dataset output by the decoder ; the dimension of the latent representation matrix is the same as ; the whole process is expressed as: .

[0063] The first encoder , the decoder and the second encoder constitute a generative adversarial network.

[0064] Step 34, build a discriminator , the input of the discriminator is the generated dataset output by the decoder and the input dataset ; the discriminator has the same structure as the second encoder , the difference is that a binary classifier is added after the fully connected layer, and the sigmoid function is used to obtain the probability that the sample comes from the input dataset or the generated dataset , thus helping to train the discriminator to identify which samples are real and which are generated; through continuous optimization, new samples with high quality and low noise can be generated, thus effectively removing noise and restoring clear flight parameters; the whole process is expressed as: .

[0065] Step 35, set a total loss function, which includes the reconstruction loss of the denoising autoencoder , the first encoder and the second encoder 's reconstruction loss and the adversarial loss of the generative adversarial network :

[0066] Reconstruction loss measures the reconstruction effect between the ith sample in the input data set and the th sample generated in the generated data set , where represents the number of samples in the input data set .

[0067] Reconstruction loss where represents the number of latent representations in the latent representation matrix ; is the ith compact feature representation in the compact feature representation matrix , and is the th

[0068] Adversarial loss makes the generator and the discriminator finally reach the Nash equilibrium through the game process of generative adversarial, and its calculation method is , where represents the processing process expression of the discriminator without the binary classifier for the generated data set and the input data set , represents the processing process expression of the first encoder and the decoder for the input data set , is the L2 norm.

[0069] Therefore, the total loss function of the denoising autoencoder generative adversarial network is 。

[0070] Step 36, train the denoising autoencoder generative adversarial network until the total loss function converges or reaches the set number of training times. At this time, the reconstruction ability of the denoising autoencoder reaches a relatively optimal level, and a game balance state is achieved between the denoising autoencoder and the generative adversarial network, that is, the discriminator cannot effectively distinguish whether the sample comes from the input data set or the generated data set , and its output is close to random guessing.

[0071] After the training is completed, input the input data set to be cleaned and enhanced into the trained denoising autoencoder generative adversarial network to obtain the generated data set ; finally, the new data set output by the denoising autoencoder generative adversarial network , that is, it contains the original input data set , and also contains the generated data set obtained by sample cleaning and the generative adversarial process 。

[0072] Step 4, construct a spatio-temporal graph for the new data set according to prior knowledge; among them, the number of samples of the input data set changes from to after being processed by the denoising autoencoder generative adversarial network; for the n flight parameters of the unmanned aerial vehicle, construct an association relationship model between each flight parameter based on kinematics and dynamics, and determine the adjacency matrix as the spatio-temporal graph based on the association relationship model; input each sample in the new data set and the adjacency matrix A as a set of spatio-temporal data into the spatio-temporal graph neural network for training; save the trained spatio-temporal graph neural network for the health state assessment of the unmanned aerial vehicle; the specific implementation process of this step is as follows:

[0073] Step 41, model the n flight parameters of the unmanned aerial vehicle based on kinematics and dynamics to obtain an association relationship model between each flight parameter; then take each flight parameter as a node, and determine whether there is a connection relationship between the nodes based on the association relationship model; there is an edge between the nodes with a connection relationship, and the value of the corresponding element in the adjacency matrix is 1; there is no edge between the nodes without a connection relationship, and the value of the corresponding element in the adjacency matrix is 0, so as to obtain the constructed adjacency matrix A as the spatio-temporal graph; among them, the rows and columns of the adjacency matrix A correspond one-to-one to the n flight parameters; it is considered that there is an association relationship between each node and itself.

[0074] The six-rotor unmanned aerial vehicle used in an embodiment of the present invention is as Figure 4As shown, a total of 18 flight parameters are used, namely the motor speeds of six rotors , roll angle , pitch angle , yaw angle , the acceleration of the UAV in the axis ; the dot in the superscript of the parameter represents its derivative, and the number of dots represents the order.

[0075] Based on kinematics and dynamics, the correlation relationship model between each flight parameter is as follows:

[0076] ;

[0077] Among them, represents the motor speed parameter, are respectively the moments of inertia when the airframe rotates around the axis, is the moment of inertia of the propeller rotor around the motor axis, is the distance from the center of the propeller to the center of the UAV airframe, is the gravitational acceleration, is the airframe mass, are respectively the vertical, roll, pitch and yaw control quantities.

[0078] , , , and The calculation process is as follows:

[0079] ;

[0080] Among them, are respectively the lift coefficient and drag coefficient of the propeller rotor.

[0081] From the expression of the correlation relationship model of the UAV, it can be seen that the acceleration of the UAV in the axis is affected by the vertical control quantity , and the value of depends on the motor speeds of the six rotors. Therefore, it can be known that and are all related; so in the constructed space-time diagram, the nodes are all nodes For the neighbor nodes, the elements at the corresponding positions in the adjacency matrix for the nodes with an association relationship are 1, that is, there is an edge between the nodes with an association relationship; otherwise, there is no edge, and the elements at the corresponding positions in the adjacency matrix are 0. It should be noted that an association relationship is considered to exist between each node and itself, that is, the element values on the diagonal of the adjacency matrix are all 1; thus, the constructed adjacency matrix A is obtained as the spatio-temporal graph, as Figure 5 shown.

[0082] Step 42, establish a spatio-temporal graph neural network; as Figure 3 shown, the spatio-temporal graph neural network includes two stacked spatio-temporal graph convolution modules to sequentially transmit and fuse temporal and spatial features, improving the accuracy and richness of feature representation.

[0083] For each sample in the new data set , the sample and the adjacency matrix A are input into the spatio-temporal graph neural network as a set of spatio-temporal data; where the spatio-temporal graph represented by the adjacency matrix A contains the association relationship between flight parameters, and the sample contains the specific values of flight parameters, that is, specific data attributes are given to the spatio-temporal graph.

[0084] The input spatio-temporal data first passes through the first spatio-temporal graph convolution module. In this module, the spatio-temporal features of the flight parameters in the sample are extracted through a gated convolution layer, a graph convolution layer, and a gated convolution layer in sequence, and combined with the structural information of the spatio-temporal graph to capture the dependence relationship between time series and space; then, after being processed by a batch normalization layer, the distribution of the features becomes more stable, accelerating the training process of the network; then, a ReLU activation function layer is used to introduce a non-linear transformation to further enhance the expression ability of the model, and the first aggregated feature is obtained; the second spatio-temporal graph convolution module has the same structure as the first spatio-temporal graph convolution module, the difference being that the number of convolution kernels is more than that of the first spatio-temporal convolution module; it is used to perform deeper spatio-temporal feature fusion on the first aggregated feature again to obtain the second aggregated feature; after the second aggregated feature is processed by a gated convolution layer, the output feature enters the global average pooling layer to reduce the feature dimension, while reducing the computational complexity, and retaining the global feature information of the nodes; the output of the global average pooling layer is used as the input of the fully connected layer, and finally the health state is evaluated through the fully connected layer to obtain the predicted label, that is, the category of the health state of the UAV; this process not only improves the accuracy of the features, but also can effectively process large-scale graph data, improving the performance and generalization ability of the model.

[0085] Step 43, train the spatio-temporal graph neural network, specifically as follows:

[0086] The total loss function of the spatio-temporal graph neural network is the weighted sum of the interactive circle loss and the cross-entropy loss, which is expressed as , represents the cross-entropy loss, Represents the interactive circle loss proposed by the present invention, is the weight coefficient of the interactive circle loss, which is set to 1 in the embodiments of the present invention.

[0087] (1) Interactive circle loss has two parameters: the margin parameter and the scale parameter , and their values will be dynamically adjusted during the training process of the spatio-temporal graph neural network; the margin parameter is used to control the constraint strength of the model on the decision boundary or the tolerance of classification errors; the expression of the interactive circle loss is:

[0088] ;

[0089] wherein, respectively represent the th positive sample, the total number of positive samples, the th negative sample and the total number of negative samples; and are the decision boundaries of positive samples and negative samples respectively, and initial assumed values are given and then updated during the training process; and represent the weights of positive samples and negative samples respectively, represents the scale parameter, the lower the similarity of the positive sample pair, the more difficult it is to separate the positive sample, and the higher its weight; the higher the similarity of the negative sample pair, the more difficult it is to separate the negative sample, and the higher its weight; the similarity is measured by the Euclidean distance; wherein, for the current sample in the new dataset, if other samples in the new dataset have the same class label as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current sample is paired with positive samples pairwise to form positive sample pairs; the current sample is paired with negative samples pairwise to form negative sample pairs.

[0090] The update process of the margin parameter is , and are the th and the th times of the spatio-temporal graph neural network training respectively, represents the margin parameter of the spatio-temporal graph neural network training at the th time, the scale parameter controls the scale or influence of the interactive circle loss, and adjusts the response mode of the spatio-temporal graph neural network to errors; the update process of the scale parameter is , Denote the scale parameter at the th training of the spatio-temporal graph neural network; and are the partial derivatives of the total loss function with respect to the edge parameter and the scale parameter , denotes the learning rate of the spatio-temporal graph neural network.

[0091] (2) Cross-entropy loss Measures the logarithmic difference between the probability predicted by the spatio-temporal graph neural network and the true label, prompting the spatio-temporal graph neural network to gradually adjust its network parameters during training, maximizing the prediction probability of the correct class, minimizing the difference between the predicted class distribution of the model and the actual label distribution, thereby improving the accuracy of the model; it is expressed as:

[0092] ;

[0093] where and represent the total number of samples and the total number of label classes in the new dataset respectively, and represent the th sample and the th class label respectively; is the one-hot encoding of the predicted label output by the spatio-temporal graph neural network. When the sample truly belongs to the th class, , otherwise, .

[0094] The first spatio-temporal graph convolution module set contains 32 convolutional kernels, and the second spatio-temporal graph convolution module contains 64 convolutional kernels. A larger batch_size will accelerate the training speed but may cause the model to be unstable. A smaller batch_size can help the model converge more stably, but the training speed will decrease. Through multiple experiments, the batch_size is set to 32; the learning rate of the network parameters is set to 0.0001, the number of iterations epoch is set to 50, and the adam optimizer is used to update and optimize the network parameters by the batch gradient descent method to reduce the value of the total loss function.

[0095] Step 5, when the present invention is actually applied, first collect the flight parameters of the drone to be evaluated, and input the flight parameters at each moment and the adjacency matrix into the trained spatio-temporal graph neural network. The spatio-temporal graph neural network outputs the predicted label corresponding to each moment, that is, the category of the health state of the drone, so as to realize the health state evaluation of the drone.

[0096] The prediction result of an embodiment of the present invention is asFigure 6 As shown; it can be seen that the method of the present invention can accurately evaluate the health state recognition of the UAV, and its accuracy rate is at least 94.5%; the confusion matrix of the health state evaluation is as Figure 7 shown; from the confusion matrix, for the three categories of the health state of the UAV: namely normal H, fault F5 and fault F10, the present invention has a high recognition accuracy for the two fault states F5 and F10, but a low recognition accuracy for the normal state H; in the embodiment, 8 health states are recognized as the F5 fault state, and 1 F10 fault state is recognized as the F5 fault state, which may be due to the existence of a certain similarity between the normal state H and the fault state F5 in the feature space. Generally speaking, the present invention can accurately identify the health state of the UAV and can provide stable and accurate health state monitoring in actual industrial applications.

[0097] In summary, the present invention can combine the time dynamic characteristics and spatial dependence relationship in the UAV flight process to improve the accuracy of health state evaluation and assessment, which not only helps to identify potential faults in advance and avoid accidents, but also can extend the service life of the UAV, enable maintenance personnel to perform repairs and replace parts more targeted, reduce unnecessary downtime and maintenance time, reduce maintenance costs, improve the reliability of the UAV flight process, and has important theoretical value and application prospects in promoting the safety, reliability and intelligent management of the UAV health management technology.

[0098] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An unmanned aerial vehicle health state evaluation method based on an enhanced spatio-temporal graph neural network, characterized in that Including: Collect the flight parameters of the drone to be evaluated, and input the flight parameters and the adjacency matrix at each moment into the trained spatio-temporal graph neural network. Output the predicted label corresponding to each moment through the spatio-temporal graph neural network to obtain the category of the health state of the drone. The spatio-temporal graph neural network includes two layers of spatio-temporal graph convolution modules. Each layer of spatio-temporal graph convolution module includes a gated convolutional layer, a graph convolutional layer, a gated convolutional layer, a batch normalization layer, and an activation function layer connected in sequence. The total loss function of the spatio-temporal graph neural network is the weighted sum of the interactive circle loss and the cross-entropy loss, which is expressed as , represents the cross-entropy loss, represents the interactive circle loss, is the weight coefficient of the interactive circle loss; The interactive circle loss has the following expression: ； Among them, respectively represent the th positive sample, the total number of positive samples, the th negative sample, and the total number of negative samples; and are respectively the decision boundaries of positive samples and negative samples; and respectively represent the weights of positive samples and negative samples, represents the margin parameter, represents the scale parameter, represents the similarity of positive sample pairs, represents the similarity of negative sample pairs; Among them, when the spatio-temporal graph neural network is trained, first, a flight parameter dataset of the unmanned aerial vehicle (UAV) in different health states is obtained. After preprocessing the flight parameter dataset, an input dataset is obtained. Then, the input dataset and random noise are input into a denoising autoencoder generative adversarial network for data cleaning and data augmentation to obtain a new dataset; The denoising autoencoder generative adversarial network includes a first encoder, a decoder, a second encoder, and a discriminator, where: the first encoder includes a convolutional layer, a batch normalization layer, an activation function layer, and a fully connected layer connected in sequence; the decoder includes a connection layer, a transposed convolutional layer, a batch normalization layer, and an activation function layer connected in sequence; the structure of the second encoder is the same as that of the first encoder and is used to learn and generate a latent representation matrix from the generated dataset; the first encoder, the decoder, and the second encoder constitute a generative adversarial network; the structure of the discriminator is basically the same as that of the second encoder, except that a binary classifier is added after the fully connected layer of the discriminator to determine the probability that a sample comes from the input dataset or the generated dataset; Based on kinematics and dynamics, a correlation relationship model between flight parameters of the UAV is established; taking the flight parameters as nodes, the connection relationship between nodes is determined based on the correlation relationship model, and thus an adjacency matrix is constructed as a spatio-temporal graph; taking the samples in the new dataset and the adjacency matrix as spatio-temporal data, the spatio-temporal graph neural network is trained.

2. The method for evaluating the health status of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 1, wherein, The flight parameter dataset contains multiple flight parameters of the UAV; multiple flight parameters collected at the same moment are taken as a sample, and the category of the health state of the UAV when the flight parameters are collected is taken as the label of the sample.

3. The method for evaluating the health status of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 1, wherein Samples in the input dataset are added with random noise through a noise reduction mechanism and then input into the first encoder to output a compact feature representation matrix; The compact feature representation matrix is processed by the decoder to obtain a generated dataset, which contains new samples generated by the decoder; the first encoder and the decoder constitute a denoising autoencoder; a total loss function is set, and the total loss function includes the reconstruction loss of the denoising autoencoder, the reconstruction losses of the first encoder and the second encoder, and the adversarial loss of the generative adversarial network.

4. The method for evaluating the health state of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 3, wherein, The reconstruction loss of the denoising autoencoder measures the reconstruction effect between the ith sample in the input dataset and the ith sample generated in the generated dataset , which is expressed as , where represents the number of samples in the input dataset ; The reconstruction losses of the first encoder and the second encoder are expressed as ; where represents the number of latent representations in the latent representation matrix ; is the compact feature representation matrix in the th compact feature representation, is the th latent representation in the latent representation matrix ; The adversarial loss of the generative adversarial network is expressed as , where represents the processing process expression of the discriminator without the binary classifier for the generated dataset and the input dataset , represents the first encoder and the decoder for the input dataset processing process expression, is the L2 norm.

5. The method for evaluating the health state of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 1, wherein Taking the flight parameters as nodes, determining the connection relationship between nodes based on the correlation relationship model, and thus constructing an adjacency matrix as a spatio-temporal graph, including: Based on the correlation relationship model, it is determined whether there is a connection relationship between nodes. Nodes with a connection relationship have an edge between them, and the value of the corresponding element in the adjacency matrix is 1; nodes without a connection relationship have no edge between them, and the value of the corresponding element in the adjacency matrix is 0, thus obtaining the constructed adjacency matrix as a spatio-temporal graph; the rows and columns of the adjacency matrix correspond one-to-one to the flight parameters; it is considered that there is a correlation relationship between each node and itself.

6. The method for evaluating the health status of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 1, wherein, For each sample in the new dataset, the sample and the adjacency matrix are input into the spatio-temporal graph neural network as a set of spatio-temporal data; the input spatio-temporal data first passes through the first spatio-temporal graph convolution module to obtain the first aggregated feature; the first aggregated feature is input into the second spatio-temporal graph convolution module to obtain the second aggregated feature; after the second aggregated feature is processed by a gated convolution layer, the output feature enters the global average pooling layer; the output of the global average pooling layer is used as the input of the fully connected layer, and finally, the health state is evaluated through the fully connected layer to obtain the prediction label, that is, the category of the health state of the drone.

7. The method for evaluating the health status of an unmanned aerial vehicle based on an enhanced spatio-temporal graph neural network according to claim 1, wherein For the current sample in the new dataset, if other samples in the new dataset have the same category label as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current sample is paired with the positive samples pairwise to form positive sample pairs; the current sample is paired with the negative samples pairwise to form negative sample pairs.

8. An unmanned aerial vehicle health status evaluation device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that, When the processor executes the computer program, it implements the method for evaluating the health state of a drone based on an enhanced spatio-temporal graph neural network according to any one of claims 1-7.

9. A computer-readable storage medium storing a computer program therein; characterized in that, When the computer program is executed by the processor, it implements the method for evaluating the health state of a drone based on an enhanced spatio-temporal graph neural network according to any one of claims 1-7.

Images