Unmanned aerial vehicle health state assessment method based on enhanced space-time diagram neural network

Abstract

The invention discloses an unmanned aerial vehicle health state assessment method based on an enhanced space-time diagram neural network, and the method comprises the steps: collecting the flight parameters of a to-be-assessed unmanned aerial vehicle, and inputting the flight parameters at each moment and an adjacent matrix into a trained space-time diagram neural network, outputting the prediction label corresponding to each moment through a space-time diagram neural network, and obtaining the type of the health state of the unmanned aerial vehicle; when the time-space diagram neural network is trained, firstly, flight parameter data sets of the unmanned aerial vehicle in different health states are obtained, and a denoising auto-encoder is used for generating a confrontation network to generate a new data set; modeling the flight parameters of the unmanned aerial vehicle based on kinematics and dynamics to obtain an incidence relation model among the flight parameters; taking the flight parameters as nodes, determining a connection relation between the nodes based on an incidence relation model, and constructing an adjacent matrix as a space-time diagram; and taking the sample and the adjacent matrix in the new data set as spatio-temporal data, and training the spatio-temporal diagram neural network.

Description

Technical Field

[0001] The present invention relates to the field of drone health status assessment, and in particular to a drone health status assessment method based on an enhanced spatiotemporal graph neural network. Background Art

[0002] The rapid development of drone technology has led to its increasingly widespread application in logistics, agriculture, surveying and mapping, and other fields. However, due to the complex and changeable working environment of drones and their high-load operation, the failure rate of drones has gradually increased, affecting their stability and reliability. Therefore, how to monitor the health status of drones in real time and ensure their safety and reliability during flight has become an important research direction in the field of drone health management.

[0003] Traditional methods for monitoring the health status of drones mainly rely on shallow machine learning methods and statistical modeling methods. These methods usually perform time series analysis on sensor data or use statistical models for evaluation, which makes it difficult to effectively capture the spatiotemporal correlations and complex dynamic changes between various flight parameters of drones. Summary of the invention

[0004] The purpose of the present invention is to provide a UAV health status assessment method based on an enhanced spatiotemporal graph neural network, which can better capture the spatiotemporal correlation of various flight parameters of the UAV, thereby improving the accuracy and reliability of health status assessment.

[0005] In order to achieve the above tasks, the present invention adopts the following technical solutions:

[0006] The UAV health status assessment method based on enhanced spatiotemporal graph neural network includes:

[0007] The flight parameters of the drone to be evaluated are collected, and the flight parameters and adjacency matrix at each moment are input into the trained spatiotemporal graph neural network. The spatiotemporal graph neural network outputs the corresponding prediction label at each moment to obtain the category of the drone's health status.

[0008] When training the spatiotemporal graph neural network, the flight parameter data sets of the UAV in different health states are first obtained, the flight parameter data sets are preprocessed to obtain the input data sets, and then the input data sets and random noise are input into the denoising autoencoder generative adversarial network for data cleaning and data enhancement to obtain a new data set;

[0009] The flight parameters of the UAV are modeled based on kinematics and dynamics to obtain the correlation model between the flight parameters; the flight parameters are used as nodes, and the connection relationship between the nodes is determined based on the correlation model, so as to construct the adjacency matrix as the space-time graph; the samples and adjacency matrix in the new data set are used as space-time data to train the space-time graph neural network.

[0010] Furthermore, the flight parameter data set includes multiple flight parameters of the UAV; the multiple flight parameters collected at the same time are taken as a sample, and the category of the health status of the UAV when the flight parameters are collected is taken as a label of the sample.

[0011] Furthermore, the preprocessing of the flight parameter data set adopts a standard deviation normalization method.

[0012] Furthermore, the denoising autoencoder generative adversarial network includes a first encoder, a decoder, a second encoder and a discriminator, wherein:

[0013] The first encoder includes a convolutional layer, a batch normalization layer, an activation function layer, and a fully connected layer connected in sequence. The samples in the input data set are input into the first encoder after adding random noise through the denoising mechanism, and the compact feature representation matrix is ​​output.

[0014] The decoder includes a connection layer, a deconvolution layer, a batch normalization layer, and an activation function layer connected in sequence; the compact feature representation matrix is ​​processed by the decoder to obtain a generated data set, which contains new samples generated by the decoder; the first encoder and the decoder constitute a denoising autoencoder;

[0015] The structure of the second encoder is the same as that of the first encoder, and is used to learn to generate a potential representation matrix from the generated dataset; the first encoder, decoder, and second encoder form a generative adversarial network;

[0016] The structure of the discriminator is basically the same as that of the second encoder, except that a binary classifier is added after the fully connected layer of the discriminator to determine the probability of whether the sample comes from the input dataset or the generated dataset;

[0017] Set the total loss function, which includes the reconstruction loss of the denoising autoencoder, the reconstruction loss of the first encoder and the second encoder, and the adversarial loss of the generative adversarial network.

[0018] Furthermore, the reconstruction loss of the denoising autoencoder is measured using the mean square error of the input data set. The Samples and generate datasets The generated Samples The reconstruction effect between ,in, Represents the input dataset The number of samples in ;

[0019] The reconstruction loss of the first encoder and the second encoder is expressed as ;in Represents the latent representation matrix the number of potential representations in ; is the compact feature representation matrix The A compact feature representation, is the potential representation matrix The potential representation;

[0020] The adversarial loss of the generative adversarial network is expressed as ,in Represents the removal of the discriminator of the binary classifier to generate a data set and the input dataset The processing expression of Represents the first encoder and decoder For the input data set The processing expression of is the L2 norm.

[0021] Furthermore, the flight parameters are used as nodes, and the connection relationship between the nodes is determined based on the association relationship model, so as to construct an adjacency matrix as a spatiotemporal graph, including:

[0022] Based on the association relationship model, it is determined whether there is a connection relationship between nodes. There is an edge between nodes with a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​1; there is no edge between nodes without a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​0, thereby obtaining a constructed adjacency matrix as a space-time graph; the rows and columns of the adjacency matrix correspond one-to-one to the flight parameters; each node is considered to have an association relationship with itself.

[0023] Furthermore, the spatiotemporal graph neural network includes two layers of spatiotemporal graph convolution modules; each layer of the spatiotemporal graph convolution module includes a gated convolution layer, a graph convolution layer, a gated convolution layer, a batch normalization layer, and an activation function layer connected in sequence;

[0024] For each sample in the new data set, the sample and the adjacency matrix are input into the spatiotemporal graph neural network as a set of spatiotemporal data; the input spatiotemporal data first passes through the first layer of spatiotemporal graph convolution module to obtain the first aggregated feature; the first aggregated feature is input into the second layer of spatiotemporal graph convolution module to obtain the second aggregated feature; the second aggregated feature is processed by a gated convolution layer, and the output feature enters the global average pooling layer; the output of the global average pooling layer is used as the input of the fully connected layer, and finally the health status is evaluated through the fully connected layer to obtain the predicted label, that is, the category of the health status of the drone.

[0025] Furthermore, the total loss function of the spatiotemporal graph neural network is the weighted sum of the interactive circle loss and the cross entropy loss, which is expressed as , represents the cross entropy loss, represents the interactive circle loss, is the weight coefficient of the interactive circle loss.

[0026] Furthermore, the interactive circle loss The expression is:

[0027] ;

[0028] in, Respectively represent positive samples, the total number of positive samples, negative samples and the total number of negative samples; and They are the decision boundaries of positive samples and negative samples respectively; and Represent the weight of positive samples and the weight of negative samples respectively, represents the edge parameter, represents the scale parameter, Represents the similarity of the positive sample pair, Represents the similarity of negative sample pairs; wherein, for the current sample in the new data set, if other samples in the new data set have the same category label as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current sample is paired with the positive sample in pairs to form a positive sample pair; the current sample is paired with the negative sample in pairs to form a negative sample pair.

[0029] A drone health status assessment device comprises a processor, a memory and a computer program stored in the memory; when the processor executes the computer program, the drone health status assessment method based on an enhanced spatiotemporal graph neural network is implemented.

[0030] A computer-readable storage medium stores a computer program; when the computer program is executed by a processor, the method for evaluating the health status of a drone based on an enhanced spatiotemporal graph neural network is implemented.

[0031] Compared with the prior art, the present invention has the following technical features:

[0032] 1. The method of the present invention effectively captures the spatial correlation information and temporal dynamic characteristics between various flight parameters of the UAV during a complex flight process, and then accurately evaluates the health status of the UAV and promptly discovers potential failure risks, which is of great significance to the healthy operation of the UAV.

[0033] 2. This invention adopts data cleaning and data enhancement technology based on the denoising self-compiler generative adversarial network, facing complex multimodal flight parameters, and can effectively process the noise and abnormal data generated during the flight of the drone and improve the data quality. By generating new and diverse flight data samples and expanding the input data set, the problem of drone health status assessment under limited data is solved.

[0034] 3. The present invention adopts an interactive circle loss function, which solves the problem that the traditional loss function is easily affected by class imbalance or data scarcity when there are few samples, resulting in the model being unable to accurately identify the subtle differences in different health states. The use of the interactive circle loss function to train the spatiotemporal graph neural network achieves higher accuracy and stability, and has strong practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 It is a schematic flow diagram of the method of the present invention;

[0036] Figure 2 Schematic diagram of a denoising autoencoder generating adversarial network in the present invention;

[0037] Figure 3 It is a structure diagram of a spatiotemporal graph neural network constructed by the present invention;

[0038] Figure 4 is a schematic diagram of the structure of a drone in one embodiment of the present invention;

[0039] Figure 5 is a schematic diagram of an adjacency matrix constructed using flight parameters in one embodiment of the present invention;

[0040] Figure 6 is a schematic diagram of health status assessment results in one embodiment of the present invention;

[0041] Figure 7 Schematic diagram of a confusion matrix for health status assessment in one embodiment of the present invention. DETAILED DESCRIPTION

[0042] As an important indicator for monitoring the flight status of a drone, the flight parameters of a drone cover the speed, acceleration, attitude angle (such as pitch angle, roll angle, yaw angle), flight altitude, direction, thrust, flight time, GPS positioning data, etc. during the flight of the drone. These data are of great significance in real-time monitoring of the drone status, optimizing flight control, evaluating health status, and predicting potential failures.

[0043] Accurately and fully utilizing the acquired UAV flight parameters can achieve real-time monitoring and precise evaluation of the UAV flight status, which helps to identify potential faults in advance and perform predictive maintenance; through in-depth analysis of flight parameters, real-time monitoring and health evaluation of UAV performance can be achieved, potential faults and anomalies can be identified in advance, and the safety, stability and reliability of flight missions can be ensured. At the same time, flight control strategies can be optimized, flight efficiency can be improved, flight paths can be adjusted, mission execution can be optimized, energy consumption can be reduced, and the development of UAV health monitoring technology towards a higher level of intelligence and automation can be promoted.

[0044] Since flight parameters usually have the characteristics of high dimensionality, nonlinearity and time series, and are affected by external environmental factors, simple machine learning models and statistical models are difficult to capture the complex correlation between parameters. In addition, the noise, outliers and insufficient samples of flight data further pose challenges to the health status assessment of drones. To this end, the use of more advanced deep learning models, especially those based on spatiotemporal graph neural networks and generative adversarial networks, can better explore the potential patterns and dynamic changes in the acquired data, and effectively improve the accuracy and robustness of health status assessment; the model provided by the present invention can accurately predict the health status of drones under complex flight missions and limited sample conditions, and provide more reliable fault warnings and maintenance support.

[0045] The present invention provides a method for evaluating the health status of unmanned aerial vehicles based on an enhanced spatiotemporal graph neural network. Figure 1 As shown, including:

[0046] The flight parameters of the drone to be evaluated are collected, and the flight parameters and adjacency matrix at each moment are input into the trained spatiotemporal graph neural network. The spatiotemporal graph neural network outputs the corresponding prediction label at each moment to obtain the category of the drone's health status.

[0047] When training the spatiotemporal graph neural network, the flight parameter data sets of the UAV in different health states are first obtained, the flight parameter data sets are preprocessed to obtain the input data sets, and then the input data sets and random noise are input into the denoising autoencoder generative adversarial network for data cleaning and data enhancement to obtain a new data set;

[0048] The flight parameters of the UAV are modeled based on kinematics and dynamics to obtain the correlation model between the flight parameters. The flight parameters are used as nodes, and the connection relationship between the nodes is determined based on the correlation model, so as to construct an adjacency matrix as a space-time graph. The samples and adjacency matrix in the new data set are used as space-time data to train the space-time graph neural network.

[0049] The specific implementation process of the present invention is described in detail below with reference to the accompanying drawings.

[0050] Step 1: Build a data set of flight parameters of drones in different health states ; Where n represents the number of flight parameters; Indicates the collected flight parameters, .

[0051] In one embodiment, the flight parameters include the motor speeds of the six rotors of the drone during flight. , Roll Angle , Pitch angle , yaw angle , Roll angle angular velocity , pitch angle angular velocity , yaw angle, angular velocity , drones in Axis acceleration , roll angular acceleration , pitch angular acceleration , yaw angular acceleration , that is, n=18; the n flight parameters collected at the same time are taken as a sample, and the category of the health status of the drone when the flight parameters are collected is taken as the label of the sample; the category of the health status is given by the expert judgment.

[0052] Step 2: The flight parameters in the flight parameter data set are processed using the standard deviation normalization method to obtain the input data set as ,in Indicates flight parameters The results are normalized by standard deviation.

[0053] The standard deviation normalization method refers to calculating the maximum mean of the flight parameter at different times for each flight parameter in the flight parameter data set. and standard deviation , then subtract the maximum mean value from the flight parameter at each moment Then multiply by the standard deviation , thus obtaining the standard deviation normalization result of the flight parameters at the corresponding time; completing the standard deviation normalization processing of each flight parameter in turn, and obtaining the input data set , where each row of data corresponds to the flight parameters at each moment, that is, a sample normalized by standard deviation; this standard deviation normalization method can keep the distribution shape of the data unchanged.

[0054] Step 3: Input the dataset and random noise N as input, and use the denoising autoencoder to generate adversarial networks to clean and enhance the samples in the dataset. At the same time, by combining random noise with the characteristics of the samples, new and realistic samples are generated. After all samples are processed by the denoising autoencoder generative adversarial network, a new dataset is obtained. ;in Indicates flight parameters The result after being processed by a denoising autoencoder and a generative adversarial network.

[0055] Let the number of samples before processing be m. The denoising autoencoder generative adversarial network will generate a new sample for each sample, so the number of samples after processing is 2m.

[0056] like Figure 2 As shown in the figure, the specific design of the denoising autoencoder generation adversarial network is as follows:

[0057] Step 31: Build the first encoder , It includes sequentially connected convolutional layers, batch normalization (BatchNormalization, BN) layers, activation function (ReLU) layers and fully connected layers; among them, a denoising mechanism (proposed by Vincent et al. in 2008) is set in the convolutional layer to learn robust features from samples with random noise N added.

[0058] Input dataset The samples in are added with random noise N through the denoising mechanism, so that the first encoder It has the ability to extract effective features from interference information, and then the samples are passed through the convolution layer to extract key information features in local space or time; the extracted key information features are processed by the batch normalization layer to alleviate internal covariate shift, accelerate convergence and enhance model stability; then the output results of the batch normalization layer are activated through the activation function layer to enhance the nonlinear modeling ability and improve the model's ability to express complex relationships; finally, the deep features are compressed and mapped through the fully connected layer to output a potential compact feature representation matrix , thus providing a high-quality feature basis for subsequent modeling and discrimination tasks; the whole process is expressed as: .

[0059] Step 32: Build the decoder , It is composed of a connection layer, a deconvolution layer, a batch normalization layer and an activation function (ReLU) layer connected in sequence to transform the compact feature representation matrix Reconstructed to be the same as the input dataset Generated datasets of the same dimension , the whole process is expressed as: ; Among them, generate data set The new sample is generated.

[0060] In the decoder In the connection layer, the compact feature representation matrix Adjust to a shape suitable for deconvolution operation; then, the original spatial or temporal structure information is gradually restored through the deconvolution layer to achieve the restoration of low-dimensional features to high-dimensional representation; then, the batch normalization layer normalizes the output of the deconvolution layer, eliminates the offset of the activation value distribution, and improves the training stability; finally, the activation function layer introduces nonlinear characteristics, enhances the model's expressiveness and outputs the final generated data set ; The goal of the whole process is to restore the input data set as accurately as possible , thereby achieving effective feature decoding and information retention.

[0061] The first encoder mentioned above and decoder Construct a denoising autoencoder.

[0062] Step 33: Build the second encoder , The structure and first encoder The same structure is used to decode Output generated dataset Learning to Generate Latent Representation Matrix , the latent representation matrix The dimensions and The same; the whole process is expressed as: .

[0063] The first encoder , Decoder and the second encoder Construct a generative adversarial network.

[0064] Step 34: Build the discriminator , the input of the discriminator is the decoder Output generated dataset and the input dataset ; Discriminator With the second encoder The structure is the same as that of the network, except that a binary classifier is added after the fully connected layer, and the sigmoid function is used to get the samples from the input dataset. Or from generating a dataset Probability , which helps the training discriminator identify which samples are real and which are generated; through continuous optimization, new samples with high quality and low noise can be generated, thereby effectively removing noise and restoring clear flight parameters; the whole process is expressed as: .

[0065] Step 35, set a total loss function, which includes the reconstruction loss of the denoising autoencoder , the first encoder and the second encoder The reconstruction loss and the adversarial loss of the generative adversarial network :

[0066] Reconstruction loss Use mean square error to measure the input data set The Samples and generate datasets The generated Samples The reconstruction effect between ,in, Represents the input dataset The number of samples in .

[0067] Reconstruction loss in, Represents the latent representation matrix the number of potential representations in ; is the compact feature representation matrix The A compact feature representation, is the potential representation matrix The A potential representation.

[0068] Fighting Losses Through the generation of adversarial game process, the generator and the discriminator finally reach Nash equilibrium, which is calculated as follows: ,in Represents the removal of the discriminator of the binary classifier to generate a data set and the input dataset The processing expression of Represents the first encoder and decoder For the input data set The processing expression of is the L2 norm.

[0069] Therefore, the total loss function of the denoising autoencoder generation adversarial network is .

[0070] Step 36, training the denoising autoencoder generative adversarial network until the total loss function converges or reaches the set number of training times, at which point the reconstruction ability of the denoising autoencoder reaches a relatively good level, and at the same time, the denoising autoencoder and the generative adversarial network reach a game equilibrium state, that is, the discriminator cannot effectively distinguish whether the sample comes from the input data set Or from generating a dataset , whose output is close to random guessing.

[0071] After training, the input data set to be cleaned and enhanced Input into the trained denoising autoencoder generative adversarial network to obtain the generated data set ; Finally, the denoising autoencoder generates a new dataset of adversarial network output , which contains the original input data set , which also includes Generate a dataset by performing sample cleaning and generating adversarial processes .

[0072] Step 4: For the new dataset Construct a spatiotemporal graph based on prior knowledge; the input dataset After the denoising autoencoder generates an adversarial network, the number of samples is reduced by becomes ; For the n flight parameters of the UAV, a correlation model between the flight parameters is constructed based on kinematics and dynamics, and the adjacency matrix is ​​determined as a space-time graph based on the correlation model; the new data set Each sample and the adjacency matrix A in the image are input as a set of spatiotemporal data into the spatiotemporal graph neural network for training; the trained spatiotemporal graph neural network is saved for drone health status assessment; the specific implementation process of this step is as follows:

[0073] Step 41, modeling the n flight parameters of the UAV based on kinematics and dynamics to obtain a correlation model between the flight parameters; then taking each flight parameter as a node, determining whether there is a connection relationship between the nodes based on the correlation model; there is an edge between the nodes with a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​1; there is no edge between the nodes without a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​0, thereby obtaining a constructed adjacency matrix A as a space-time graph; wherein the rows and columns of the adjacency matrix A correspond one-to-one to the n flight parameters; and each node is considered to have a correlation relationship with itself.

[0074] The six-rotor drone used in one embodiment of the present invention is as follows: Figure 4As shown, a total of 18 flight parameters are used, which are the motor speeds of the six rotors. , Roll Angle , Pitch angle , yaw angle , drones in Axis acceleration ; The dot above the parameter indicates its derivative, and the number of dots indicates the order.

[0075] Based on kinematics and dynamics, the correlation model between various flight parameters is as follows:

[0076] ;

[0077] in, Indicates the motor speed parameter, The body is The moment of inertia of the shaft when it rotates, is the moment of inertia of the propeller rotor about the motor shaft, is the distance from the center of the propeller to the center of the drone body, is the acceleration due to gravity, is the body mass, They are vertical, roll, pitch and yaw control quantities respectively.

[0078] , , , and The calculation process is:

[0079] ;

[0080] in, are the lift coefficient and drag coefficient of the propeller rotor respectively.

[0081] From the expression of the drone association model, we can see that drones are Axis acceleration Vertical control amount The impact of The value of depends on the motor speed of the six rotors , so we can know and There is an association relationship; so in the constructed space-time graph, the nodes All nodes The neighbor nodes of the nodes with associated relationships have the corresponding elements in the adjacency matrix set to 1, that is, there is an edge between the nodes with associated relationships; otherwise, there is no edge, and the corresponding elements in the adjacency matrix are set to 0. It should be noted that each node is considered to have an associated relationship with itself, that is, the diagonal element values ​​of the adjacency matrix are all 1; thus, the constructed adjacency matrix A is obtained as a space-time graph, as shown in Figure 5 shown.

[0082] Step 42, establish a spatiotemporal graph neural network; Figure 3 As shown in the figure, the spatiotemporal graph neural network includes two layers of superimposed spatiotemporal graph convolution modules to transfer and fuse temporal and spatial features layer by layer to improve the accuracy and richness of feature representation.

[0083] For new datasets For each sample in , the sample and the adjacency matrix A are input into the spatiotemporal graph neural network as a set of spatiotemporal data; the spatiotemporal graph represented by the adjacency matrix A contains the correlation between the flight parameters, and the sample contains the specific values ​​of the flight parameters, which gives the spatiotemporal graph specific data attributes.

[0084] The input spatiotemporal data first passes through the first layer of spatiotemporal graph convolution module, in which the spatiotemporal features of the flight parameters in the sample are extracted through the gated convolution layer, graph convolution layer, and gated convolution layer in turn, and the dependency between time and space is captured by combining the structural information of the spatiotemporal graph; then, it is processed by the batch normalization layer to make the distribution of features more stable and accelerate the training process of the network; then, the nonlinear transformation is introduced through the ReLU activation function layer to further enhance the expression ability of the model and obtain the first aggregated feature; the second layer of spatiotemporal graph convolution module has the same structure as the first layer of spatiotemporal graph convolution module, the difference is that the number of convolution kernels is more than that of the first layer. A layer of spatiotemporal convolution module; used to perform a deeper spatiotemporal feature fusion on the first aggregated feature again to obtain the second aggregated feature; after the second aggregated feature is processed by a gated convolution layer, the output feature enters the global average pooling layer to reduce the feature dimension, reduce the computational complexity, and retain the global feature information of the node; the output of the global average pooling layer is used as the input of the fully connected layer, and finally the health status is evaluated through the fully connected layer to obtain the predicted label, that is, the category of the health status of the drone; this process not only improves the accuracy of the feature, but also can effectively process large-scale graph data, improving the performance and generalization ability of the model.

[0085] Step 43, training the spatiotemporal graph neural network, as follows:

[0086] The total loss function of the spatiotemporal graph neural network is the weighted sum of the interactive circle loss and the cross entropy loss, which is expressed as , represents the cross entropy loss, represents the interactive circle loss proposed by the present invention, is the weight coefficient of the interactive circle loss, which is set to 1 in the embodiment of the present invention.

[0087] (1) Interactive Circle Loss There are two parameters: edge parameters and scale parameter , whose values ​​will be dynamically adjusted during the training process of the spatiotemporal graph neural network; the edge parameters Used to control the model's constraint on the decision boundary or tolerance for classification errors; interactive circle loss The expression is:

[0088] ;

[0089] in, Respectively represent positive samples, the total number of positive samples, negative samples and the total number of negative samples; and They are the decision boundaries of positive samples and negative samples, respectively. The hypothesized values ​​are given first and then updated during the training process. and Represent the weight of positive samples and the weight of negative samples respectively, Represents the scale parameter, the similarity of the positive sample pair The lower it is, the more difficult it is to distinguish the positive sample, and the higher its weight; the similarity of the negative sample pair The higher the value, the harder it is to distinguish negative samples, and the higher the weight. Similarity is measured by Euclidean distance. For the current sample in the new dataset, such as the new dataset If the other samples in the dataset have the same category labels as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current samples are paired with positive samples in pairs to form positive sample pairs; the current samples are paired with negative samples in pairs to form negative sample pairs.

[0090] Edge parameters The update process is , and They are the first Second and Second-rate, Represents the training of spatiotemporal graph neural network The edge parameter, scale parameter Controls the scale or influence of the interactive circle loss, adjusting how the spatiotemporal graph neural network responds to errors; scale parameter The update process is , Represents the training of spatiotemporal graph neural network The scale parameter of the second; and is the total loss function over the marginal parameters and scale parameter The partial guide, Represents the learning rate of the spatiotemporal graph neural network.

[0091] (2) Cross Entropy Loss The logarithmic difference between the probability predicted by the spatiotemporal graph neural network and the true label is measured, which prompts the spatiotemporal graph neural network to gradually adjust its network parameters during the training process to maximize the prediction probability of the correct category and minimize the difference between the category distribution predicted by the model and the actual label distribution, thereby improving the accuracy of the model; it is expressed as:

[0092] ;

[0093] in, and Represents the new data sets The total number of samples and the total number of label categories, and Respectively represent Samples and Class labels; is the one-hot encoding of the predicted label output by the spatiotemporal graph neural network. Class time, ,otherwise, .

[0094] The first layer of the spatiotemporal graph convolution module is set to contain 32 convolution kernels, and the second spatiotemporal graph convolution module contains 64 convolution kernels. A larger batch_size will speed up the training, but may cause the model to be unstable. A smaller batch_size can help the model converge more stably, but the training speed will decrease. After multiple experiments, the batch_size is set to 32; the network parameter learning rate is set to 0.0001, the number of iterations epoch is set to 50, and the adam optimizer is used to update and optimize the network parameters through the batch gradient descent method to reduce the value of the total loss function.

[0095] Step 5, when the present invention is actually applied, the flight parameters of the UAV to be evaluated are first collected, and the flight parameters at each moment and the adjacency matrix are input into the trained spatiotemporal graph neural network. The spatiotemporal graph neural network outputs the prediction label corresponding to each moment, that is, the category of the health status of the UAV, thereby realizing the health status evaluation of the UAV.

[0096] The prediction result of one embodiment of the present invention is as follows Figure 6 As shown; it can be seen that the method of the present invention can accurately evaluate the health status of the drone, and its accuracy is as low as 94.5%; the confusion matrix of the health status evaluation is as follows Figure 7 As shown; through the confusion matrix, it can be seen that for the categories of the health status of three drones: normal H, fault F5 and fault F10, the present invention has a high recognition accuracy for the two fault states F5 and F10, but a low recognition accuracy for the normal state H; in the embodiment, 8 health states are identified as F5 fault states, and 1 F10 fault state is identified as F5 fault state, which may be due to the certain similarity between the normal state H and the fault state F5 in the feature space. In general, the present invention can accurately identify the health status of drones and can provide stable and accurate health status monitoring in actual industrial applications.

[0097] To sum up, the present invention can combine the temporal dynamic characteristics and spatial dependencies of the UAV flight process to improve the accuracy of health status assessment and evaluation, which not only helps to identify potential faults in advance and avoid accidents, but also extends the service life of the UAV, allowing maintenance personnel to carry out maintenance and replacement of parts more targeted, reducing unnecessary downtime and maintenance time, reducing maintenance costs, and improving the reliability of the UAV flight process. It has important theoretical value and application prospects in promoting the safety, reliability and intelligent management of UAV health management technology.

[0098] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A UAV health status assessment method based on enhanced spatiotemporal graph neural network, characterized in that: include: The flight parameters of the drone to be evaluated are collected, and the flight parameters and adjacency matrix at each moment are input into the trained spatiotemporal graph neural network. The spatiotemporal graph neural network outputs the corresponding prediction label at each moment to obtain the category of the drone's health status. When training the spatiotemporal graph neural network, the flight parameter data sets of the UAV in different health states are first obtained, the flight parameter data sets are preprocessed to obtain the input data sets, and then the input data sets and random noise are input into the denoising autoencoder generative adversarial network for data cleaning and data enhancement to obtain a new data set; The flight parameters of the UAV are modeled based on kinematics and dynamics to obtain the correlation model between the flight parameters; the flight parameters are used as nodes, and the connection relationship between the nodes is determined based on the correlation model, so as to construct the adjacency matrix as the space-time graph; the samples and adjacency matrix in the new data set are used as space-time data to train the space-time graph neural network.

2. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 1 is characterized in that: The flight parameter data set includes multiple flight parameters of the UAV; the multiple flight parameters collected at the same time are taken as a sample, and the category of the health status of the UAV when the flight parameters are collected is taken as a label of the sample.

3. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 1 is characterized in that: The denoising autoencoder generative adversarial network comprises a first encoder, a decoder, a second encoder and a discriminator, wherein: The first encoder includes a convolutional layer, a batch normalization layer, an activation function layer, and a fully connected layer connected in sequence. The samples in the input data set are input into the first encoder after adding random noise through the denoising mechanism, and the compact feature representation matrix is ​​output. The decoder includes a connection layer, a deconvolution layer, a batch normalization layer, and an activation function layer connected in sequence; the compact feature representation matrix is ​​processed by the decoder to obtain a generated data set, which contains new samples generated by the decoder; the first encoder and the decoder constitute a denoising autoencoder; The structure of the second encoder is the same as that of the first encoder, and is used to learn to generate a potential representation matrix from the generated dataset; the first encoder, decoder, and second encoder form a generative adversarial network; The structure of the discriminator is basically the same as that of the second encoder, except that a binary classifier is added after the fully connected layer of the discriminator to determine the probability of whether the sample comes from the input dataset or the generated dataset; Set the total loss function, which includes the reconstruction loss of the denoising autoencoder, the reconstruction loss of the first encoder and the second encoder, and the adversarial loss of the generative adversarial network.

4. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 3 is characterized in that: The reconstruction loss of the denoising autoencoder uses the mean square error to measure the input data set The Samples and generate datasets The generated Samples The reconstruction effect between ,in, Represents the input dataset The number of samples in ; The reconstruction loss of the first encoder and the second encoder is expressed as ;in Represents the latent representation matrix the number of potential representations in ; is the compact feature representation matrix The A compact feature representation, is the potential representation matrix The potential representation; The adversarial loss of the generative adversarial network is expressed as ,in Represents the removal of the discriminator of the binary classifier to generate a data set and the input dataset The processing expression of Represents the first encoder and decoder For the input data set The processing expression of is the L2 norm.

5. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 1 is characterized in that: Taking the flight parameters as nodes, the connection relationship between the nodes is determined based on the association relationship model, so as to construct the adjacency matrix as a spatiotemporal graph, including: Based on the association relationship model, it is determined whether there is a connection relationship between nodes. There is an edge between nodes with a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​1; there is no edge between nodes without a connection relationship, and the value of the corresponding element in the adjacency matrix is ​​0, thereby obtaining a constructed adjacency matrix as a space-time graph; the rows and columns of the adjacency matrix correspond one-to-one to the flight parameters; each node is considered to have an association relationship with itself.

6. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 1 is characterized in that: The spatiotemporal graph neural network includes two layers of spatiotemporal graph convolution modules; each layer of the spatiotemporal graph convolution module includes a gated convolution layer, a graph convolution layer, a gated convolution layer, a batch normalization layer, and an activation function layer connected in sequence; For each sample in the new data set, the sample and the adjacency matrix are input into the spatiotemporal graph neural network as a set of spatiotemporal data; the input spatiotemporal data first passes through the first layer of spatiotemporal graph convolution module to obtain the first aggregated feature; the first aggregated feature is input into the second layer of spatiotemporal graph convolution module to obtain the second aggregated feature; the second aggregated feature is processed by a gated convolution layer, and the output feature enters the global average pooling layer; the output of the global average pooling layer is used as the input of the fully connected layer, and finally the health status is evaluated through the fully connected layer to obtain the predicted label, that is, the category of the health status of the drone.

7. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 1 is characterized in that: The total loss function of the spatiotemporal graph neural network is the weighted sum of the interactive circle loss and the cross entropy loss, which is expressed as , represents the cross entropy loss, represents the interactive circle loss, is the weight coefficient of the interactive circle loss.

8. The method for evaluating the health status of unmanned aerial vehicles based on enhanced spatiotemporal graph neural network according to claim 7 is characterized in that: The interactive circle loss The expression is: ； in, Respectively represent positive samples, the total number of positive samples, negative samples and the total number of negative samples; and They are the decision boundaries of positive samples and negative samples respectively; and Represent the weight of positive samples and the weight of negative samples respectively, represents the edge parameter, represents the scale parameter, Represents the similarity of the positive sample pair, Represents the similarity of negative sample pairs; wherein, for the current sample in the new data set, if other samples in the new data set have the same category label as the current sample, then the other samples are positive samples, otherwise they are negative samples; the current sample is paired with the positive sample in pairs to form a positive sample pair; the current sample is paired with the negative sample in pairs to form a negative sample pair.

9. A drone health status assessment device, comprising a processor, a memory, and a computer program stored in the memory; characterized in that: When the processor executes the computer program, it implements the method for assessing the health status of a drone based on an enhanced spatiotemporal graph neural network according to any one of claims 1 to 8.

10. A computer-readable storage medium, wherein a computer program is stored in the medium; characterized in that: When the computer program is executed by a processor, the method for assessing the health status of a drone based on an enhanced spatiotemporal graph neural network according to any one of claims 1 to 8 is implemented.

Images