Real-time remaining service life prediction method and system for multi-rotor unmanned aerial vehicles

Abstract

The present invention provides a method and system for predicting the remaining service life of a multi-rotor UAV in real time, including: constructing a multi-level fusion Transformer model based on multi-sensor data and target external factors of the multi-rotor UAV; the multi-level fusion Transformer model includes a multi-scale feature extraction layer and a decoder attention layer of the input tensor embedded with the target external factors; training the multi-level fusion Transformer model based on a preset training data set to obtain a trained multi-level fusion Transformer model; and predicting the remaining service life of the multi-rotor UAV using the trained multi-level fusion Transformer model. The present invention alleviates the technical problems of large remaining service life prediction deviation, poor real-time performance and inability to perform sequence prediction in the current data-driven UAV remaining service life prediction method.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning, and in particular to a method and system for predicting the real-time remaining service life of a multi-rotor unmanned aerial vehicle. Background Art

[0002] Multi-rotor UAVs have the ability to take off and land vertically, hover in the air, carry payloads, fly autonomously or remotely, and have flexible flight modes. The above unique advantages make them widely used in civil fields such as agriculture and transportation. At the same time, they can switch between multiple flight modes more arbitrarily in low-altitude, narrow and harsh space environments, and have great potential for military applications. Faced with the ever-changing battlefield situation, high-maneuverability multi-rotor UAVs need to overcome difficulties such as complex disturbances, physical limitations and flight constraints to ensure rapid response capabilities in various working conditions. In order to maximize the mission execution capabilities of multi-rotor UAVs, the remaining service life of the flight is a necessary criterion for flight mission planning and remaining flight capability assessment. Flight durability is directly related to the total mass of the UAV, so high-energy-density lithium polymer (Li-Po) batteries are usually used as power sources to drive high-maneuverability multi-rotors. The remaining useful life (RUL) prediction of Li-Po batteries has attracted the attention of experts from all walks of life and has become a research hotspot in the field of UAV fault prediction and health management (PHM).

[0003] The RUL of Li-Po batteries is a variable that cannot be directly observed and measured. It must be obtained through indirect partial correlation measurements, which makes its estimation and prediction have high uncertainty, resulting in highly conservative flight plans, which is not conducive to the use of the strike capability of high-mobility UAVs. The driving ability of the battery is temperature-dependent and power load-dependent. The RUL is related to the battery voltage, discharge current, load, temperature and flight mode. The existing RUL prediction technology is mainly divided into two types, based on statistical modeling and based on machine learning. The former, such as the Bayesian estimation method, can make full use of prior information, but it is highly dependent on the estimation model based on the system degradation mechanism. The battery discharge process itself is difficult to accurately model its probability distribution through a solvable mathematical model. The strong coupling and strong nonlinearity of high-mobility UAVs further increase the inaccuracy of the model. The latter includes classical machine learning methods and deep learning methods.

[0004] Classic machine learning methods such as support vector machines and random forests use advanced data processing technology and powerful algorithm technology to train data to produce the best performing classifiers. However, the data feature extraction process is often very cumbersome and time-consuming due to high-precision requirements, which is not conducive to real-time tasks such as online planning. In the field of deep learning methods, scholars have used recurrent neural networks (RNNs) as the basic framework for sequence data processing and developed cutting-edge baseline methods such as gated recurrent units (GRUs) and long short-term memory networks (LSTMs). However, these methods are based on sequential processing of sequences and cannot perform sequence-to-sequence predictions. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a real-time remaining useful life prediction method and system for a multi-rotor UAV, so as to alleviate the technical problems of large remaining useful life prediction deviation, poor real-time performance and inability to perform sequence prediction in the current data-driven UAV remaining useful life prediction method.

[0006] In the first aspect, an embodiment of the present invention provides a method for predicting the real-time remaining service life of a multi-rotor UAV, comprising: constructing a multi-level fusion Transformer model based on multi-sensor data and target external factors of the multi-rotor UAV; the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on the multi-sensor data based on a preset mining operator to obtain a time feature vector; the prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer that embeds the input tensor of the target external factors; the target external factors include the load weight, flight speed and ambient temperature of the multi-rotor UAV; training the multi-level fusion Transformer model based on a preset training data set to obtain a trained multi-level fusion Transformer model; and using the trained multi-level fusion Transformer model to predict the remaining service life of the multi-rotor UAV.

[0007] Furthermore, based on the multi-sensor data and target external factors of the multi-rotor drone, a multi-level fusion Transformer model is constructed, including: by adding the multi-scale feature extraction layer in the observation encoder of the preset Transformer model, the observation encoder of the multi-level fusion Transformer model is constructed; the multi-scale feature extraction layer is also used to perform one-dimensional convolution calculation on the multi-sensor data along the time dimension through the preset mining operator to obtain a time feature vector; by adding a decoder attention layer of the input tensor embedded with the target external factors in the prediction decoder of the preset Transformer model, the prediction decoder of the multi-level fusion Transformer model is constructed.

[0008] Furthermore, the preset mining operator is M(·; Θ); Θ:=(k, p) determines the mining scale, k represents the kernel size, and p represents the filling size; D preset mining operators are set simultaneously in each scale; D is the dimension of the multi-sensor data.

[0009] Furthermore, the preset training data set includes: a prediction data set and an observation data set; training the multi-level fusion Transformer model based on the preset training data set includes: based on the preset training data set, using a target loss function to train the multi-level fusion Transformer model; the target loss function includes: is the network prediction output, is the label value of the remaining service life, and PairwiseDistance() is the pixel-level ohmic distance.

[0010] In the second aspect, an embodiment of the present invention also provides a real-time remaining service life prediction system for a multi-rotor UAV, comprising: a building module, a training module and a prediction module; wherein the building module is used to construct a multi-level fusion Transformer model based on multi-sensor data and target external factors of the multi-rotor UAV; the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on the multi-sensor data based on a preset mining operator to obtain a time feature vector; the prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer of the input tensor embedded with the target external factors; the target external factors include the load weight, flight speed and ambient temperature of the multi-rotor UAV; the training module is used to train the multi-level fusion Transformer model based on a preset training data set to obtain the trained multi-level fusion Transformer model; the prediction module is used to predict the remaining service life of the multi-rotor UAV using the trained multi-level fusion Transformer model.

[0011] Furthermore, the multi-level fusion Transformer model includes an observation encoder; the observation encoder includes: a multi-head self-attention layer, a residual connection layer, a feedforward fully connected layer, a residual connection layer and the multi-scale feature extraction layer; the multi-scale feature extraction layer is also used to perform one-dimensional convolution calculation on the multi-sensor data along the time dimension through the preset mining operator to obtain a time feature vector.

[0012] Furthermore, the multi-level fusion Transformer model also includes a prediction decoder; the prediction decoder includes: a multi-head self-attention layer, a residual connection layer, a feedforward fully connected layer, a residual connection layer, an encoder-decoder attention layer and a decoder attention layer of the input tensor embedded with the target external factor.

[0013] Furthermore, the preset training data set includes: a prediction data set and an observation data set; the training module is further used to: train the multi-level fusion Transformer model based on the preset training data set using a target loss function; the target loss function includes: is the network prediction output, is the label value of the remaining service life, and PairwiseDistance() is the pixel-level ohmic distance.

[0014] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the first aspect when executing the computer program.

[0015] In a fourth aspect, an embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method described in the first aspect.

[0016] The embodiment of the present invention provides a real-time remaining service life prediction method and system for a multi-rotor unmanned aerial vehicle, which takes into account the influence of input saturation of the multi-rotor unmanned aerial vehicle in an edge state on the remaining flight time, uses a Transformer model network to perform multi-sensor data feature layer fusion to reduce the dependence on a single discharge voltage signal, and greatly increases the prediction accuracy and speed in various flight maneuvering situations; and the Transformer model processes input data in parallel through an attention mechanism, which speeds up the processing speed, enhances the temporal feature representation and can process partially missing observation data; at the same time, on the basis of the Transformer model, a multi-scale processing mechanism is added to process the representation of fine time features, and a multi-level embedding layer is further added to add external influencing factors to the Transformer model network to solve the feasibility evaluation of the scheduled flight plan, thereby alleviating the technical problems of large remaining service life prediction deviation, poor real-time performance and inability to perform sequence prediction in the current data-driven unmanned aerial vehicle remaining service life prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A flowchart of a method for predicting the remaining service life of a multi-rotor drone in real time provided by an embodiment of the present invention;

[0019] Figure 2 A comparison chart of a single remaining useful life prediction result provided by an embodiment of the present invention;

[0020] Figure 3 A schematic diagram of the remaining useful life prediction results of a sequence of a multi-level fusion Transformer model provided in an embodiment of the present invention;

[0021] Figure 4A schematic diagram of a real-time remaining service life prediction system for a multi-rotor UAV provided by an embodiment of the present invention;

[0022] Figure 5 A schematic diagram of the structure of a multi-level fusion Transformer model provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Embodiment 1:

[0025] Figure 1 1 is a flow chart of a method for predicting the remaining service life of a multi-rotor drone in real time according to an embodiment of the present invention. Figure 1 As shown, the method specifically comprises the following steps:

[0026] Step S102, constructing a multi-level fusion Transformer model based on the multi-sensor data of the multi-rotor UAV and target external factors.

[0027] Among them, the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on multi-sensor data based on a preset mining operator to obtain a temporal feature vector.

[0028] The prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer that embeds the input tensor of the target external factors; the target external factors include the payload weight, flight speed and ambient temperature of the multi-rotor drone.

[0029] Step S104, training the multi-level fusion Transformer model based on a preset training data set to obtain a trained multi-level fusion Transformer model.

[0030] Step S106, using the trained multi-level fusion Transformer model to predict the remaining service life of the multi-rotor UAV.

[0031] The embodiment of the present invention provides a real-time remaining service life prediction method for a multi-rotor unmanned aerial vehicle, which takes into account the influence of input saturation of the multi-rotor unmanned aerial vehicle in an edge state on the remaining flight time, and uses a Transformer model network to perform multi-sensor data feature layer fusion to reduce the dependence on a single discharge voltage signal, thereby greatly increasing the prediction accuracy and speed in various flight maneuvering situations; and the Transformer model processes input data in parallel through an attention mechanism, thereby accelerating the processing speed, enhancing the temporal feature representation and being able to process partially missing observation data; at the same time, on the basis of the Transformer model, a multi-scale processing mechanism is added to process the representation of fine time features, and a multi-level embedding layer is further added to add external influencing factors to the Transformer model network to solve the feasibility evaluation of a predetermined flight plan, thereby alleviating the technical problems of large remaining service life prediction deviation, poor real-time performance and inability to perform sequence prediction in the current data-driven unmanned aerial vehicle remaining service life prediction method.

[0032] In the embodiment of the present invention, the multi-sensor data of the multi-rotor UAV can be obtained by obtaining experimental data, or by simulation.

[0033] Preferably, the embodiment of the present invention obtains multi-sensor data of a multi-rotor drone by means of simulation. Specifically, a simulation model is constructed based on a high-precision mathematical model of a quad-rotor power mechanism taking into account disturbances and a lithium-ion battery discharge state equation taking into account the influence of temperature, and the simulation model parameters are obtained by fitting the real flight data of the quad-rotor drone at a specific flight speed, flight mode, and flight environment temperature.

[0034] Establish propeller aerodynamic model (including propeller thrust T and propeller torque M):

[0035]

[0036] Where N is the propeller speed, is the air density, h is the flight altitude, T t is the ambient temperature, C T is the propeller thrust coefficient, C M is the propeller torque coefficient, D p is the blade diameter.

[0037] Build the motor model:

[0038]

[0039] Where U m is the motor equivalent voltage, I mis the motor equivalent current, M is the motor load torque, N is the motor speed (i.e. propeller speed), Θ m These are the basic parameters of the motor.

[0040] Build an electronic speed regulator model:

[0041]

[0042] In the formula, Θ e , Θ b are the basic parameters of the ESC and the battery respectively, σ is the throttle input of the ESC, U e is the ESC input voltage, I e Input current for the ESC.

[0043] The state transition equation for establishing the battery model is as follows:

[0044]

[0045] In the formula, R int is the internal resistance of the battery, SOC is the discharge state, E crit is the initial total battery energy, P is the power consumption, k is the sampling time, and ω is the process noise. The measurement equation is established as follows:

[0046] V(k)=v oc (k)-i(k)·R(k)(k) int

[0047] In the formula, v oc is the open circuit voltage, i is the discharge current, and η is the measurement noise.

[0048] The power consumption is mainly determined by the flight mode and speed of the drone. The energy consumption model of the quadcopter is established in the hovering state:

[0049]

[0050] In climbing state:

[0051]

[0052] In descending state:

[0053]

[0054] In level flight:

[0055]

[0056] Where W is the total weight of the drone, A t is the total rotor area, V c 、V d 、Vhor is the speed under various flight modes. c , η d , η hor is the efficiency factor under various flight modes, v hor is the induced speed in level flight, a v is the angle of attack, and their changing trends with flight speed can be obtained through linear fitting of actual flight.

[0057] Based on the above state space equation, the particle filter algorithm is applied to obtain a high-precision battery discharge failure model. Based on the battery voltage estimated by filtering, the effect of temperature on battery discharge is further considered:

[0058] V=(v oc -θ1(ii std ))·exp(θ2·sign(T t -T std )·||T t -T std || 2 )

[0059] Where θ1, θ2 are temperature model parameters, i std , T std are the discharge current and temperature under baseline conditions.

[0060] The constructed high-precision simulation model is run to obtain multi-sensor data, including: battery voltage, discharge current, motor throttle and discharge state (SOC), among which SOC is indirectly obtained by integrating the discharge current. Each simulation has a small degree of random decrease in the initial SOC to simulate the initial power loss. The simulation stops when one of the following two conditions is met (i.e., the remaining service life of the drone is 0): (1) the battery voltage drops to the cut-off voltage; (2) the throttle exceeds the maximum throttle limited by the minimum attitude control capability when the drone is hovering. The total weight (empty weight + load weight), ambient temperature and flight speed of the quadcopter drone are set to the set values ​​of the automatic control program to simulate the preset remote control values ​​and weather forecast data in the actual flight, which are used as external factor signals of future mutations to input the prediction network.

[0061] After obtaining the simulation data, it is necessary to construct the prediction target label. The RUL curve of the quadcopter changes with different simulation conditions. Since the impact of system degradation in a single flight is very slight and negligible, the linear descent assignment method is used to adaptively generate RUL data as the prediction label: RUL(t) = min({t|{σ hover (t)≥σ max}∪{t|U(t)≤U thr}}). where σ hover is the hover throttle, σmax is the maximum throttle upper limit (input saturation value), U thr is the cut-off voltage.

[0062] Formally speaking, for the ith flight, T sld The data obtained by intercepting the intervals are used to construct observation data sets composed of multi-sensor signals. The prediction data set consists of the incremental RUL label value and the initial value at the start of RUL prediction External input dataset consisting of external factor signals The size of each dataset is After z-score standardization, the original data is divided into training set, validation set and test set according to a certain ratio.

[0063] Optionally, step S102 further includes:

[0064] By adding a multi-scale feature extraction layer to the observation encoder of the preset Transformer model, an observation encoder of a multi-level fusion Transformer model is constructed; the multi-scale feature extraction layer is also used to perform one-dimensional convolution calculation on multi-sensor data along the time dimension through a preset mining operator to obtain a time feature vector.

[0065] Optionally, the preset mining operator is M(·; Θ); Θ:=(k, p) determines the mining scale, k represents the kernel size, and p represents the filling size; D preset mining operators are set simultaneously in each scale; D is the dimension of the multi-sensor data.

[0066] By adding a decoder attention layer that embeds the input tensor of the target external factors in the prediction decoder of the preset Transformer model, a prediction decoder of the multi-level fusion Transformer model is constructed.

[0067] In the embodiment of the present invention, after the input data is obtained, a linear projection is performed. Mapping the original input data to a high-dimensional feature space to achieve distributed expression makes it easier for the multi-level fusion TF (i.e., multi-level fusion Transformer model) to process input features:

[0068]

[0069] where f linear is the linear projection layer, W x is the coefficient matrix. Correspondingly, the output of the i-th flight at time t is a D-dimensional vector The above inverse transformation will be used to reproject it back to the prediction space dimension, thereby realizing RUL prediction data embedding.

[0070] TF uses the attention mechanism to take into account the correlation between data at different times and between different data features, and realizes the feature layer fusion of multi-sensor data, but its parallel computing mechanism is not sensitive to time series features. In order to solve the above problems, the "position encoding" operation is applied to encode each historical and future moment, and stamp the corresponding timestamp for each data to be embedded.

[0071]

[0072] Among them, the timestamp is defined as:

[0073]

[0074] Ensure its uniqueness within 10000 time steps.

[0075] When a high-maneuverability quadrotor (i.e., a multi-rotor drone in an embodiment of the present invention) is carrying an extreme weight load or is in an extreme temperature environment, it is very likely to lose control during takeoff due to input saturation or the drive voltage dropping to the cut-off voltage level, resulting in a crash. In order to shorten the prediction blank period at the beginning of takeoff, the TF network is expected to achieve the expected prediction effect using as little observation data as possible. In this case, the time information of the observation data is limited. The embodiment of the present invention is inspired by the idea of ​​multi-granularity scanning in the deep forest, constructs a multi-scale feature extraction mechanism and combines it with the TF encoder to refine the time channel, and deeply mines the timing information embedded in the multi-sensor data.

[0076] Specifically, the multi-level fusion Transformer model obtains the above embedded multi-sensor input The mining operator M(·; Θ) performs 1d convolution on the input tensor along the time dimension. Where Θ:=(k, p) determines the mining scale, k represents the kernel size, and p represents the padding size. For a specific s scales, the mining operator processes the input tensor at each scale to obtain a tensor with a time dimension of Tensor, D mining operators are set at each scale to ensure the multi-sensor feature dimension. After the time features are refined by multiple scales, the elite features are selected as the final processing results:

[0077]

[0078] It should be noted that the above formula restricts the corresponding relationship between k and p:

[0079] In the multi-level fusion Transformer model constructed by the embodiment of the present invention, in addition to the embedded multi-scale feature extraction layer, the observation encoder and the prediction decoder are composed of multiple basic layers with attention mechanisms, each of which has three components: a multi-head self-attention layer, a feedforward fully connected layer, and two residual connection layers.

[0080] Specifically, the multi-head self-attention layer is implemented by parallel computing of h self-attention modules. For j self-attention modules, the trainable hyperparameter query vector Keypoint vector and value vector Respectively, the query matrix WQ and the key point matrix W K and the value matrix W V Determines and together constitutes the weight calculation mechanism of attention:

[0081]

[0082] Where, d k =D / h is the dimension of the hyperparameter matrix. After each self-attention module is calculated, parallel attention calculations are applied to achieve information integration from different representation subspaces:

[0083]

[0084] Where W A is the attention matrix, and Concat(·) is tensor concatenation. The feed-forward fully connected module consists of a linear transformation and a ReLU activation function, acting on each observation time step with the same weight:

[0085]

[0086] in, and is the coefficient matrix.

[0087] From the lithium-ion battery discharge failure state space equation and end-of-life conditions, we can see that RUL is also affected by many external factors, including load weight, flight speed and ambient temperature. These influencing factors are manifested as mutation signals, and the lack of signs makes their impact on the remaining service life unpredictable, and cannot match the demand for high-maneuverability flight capabilities in the future.

[0088] In order to enable the TF network to consider the strong correlation between the RUL time series and the mutation external factors, the embodiment of the present invention realizes the embedding fusion of external factors by adding an external-decoder attention layer in the decoder stage.

[0089] The purpose of the observation encoder is to create a time series representation for the multi-sensor embedded signal, thereby giving the TF network memory, and its trained key point vector K enc and the value vector V enc , will be shared to the decoder level. External factors can be regarded as features similar to multi-sensor observations, but in a different time space. The future knowledge makes it directly act on the prediction time period, so the processing of multi-sensor data is applied to the external factor input tensor Γ ext get In order to prevent the foreknowledge from changing the TF network's attention to historical observations, feature coupling updates are performed at the decoder embedding stage rather than the encoder-decoder attention stage:

[0090]

[0091] The learned potential representation is fed back to the TF network through the above multi-level encoder, which enhances the importance of external factors and the network's attention to them.

[0092] Optionally, in an embodiment of the present invention, the preset training data set includes: a prediction data set and an observation data set; training the multi-level fusion Transformer model based on the preset training data set includes:

[0093] Based on the preset training data set, the multi-level fusion Transformer model is trained using the target loss function;

[0094] The objective loss function includes:

[0095]

[0096] is the network prediction output, is the tag value of the remaining service life, and PairwiseDistance() is the pixel-level Ohm distance.

[0097] In the embodiment of the present invention, the remaining service life prediction algorithm of the high-maneuverability UAV learns the nonlinear change law of RUL with lithium battery discharge failure, external factors and multi-sensor data through offline training of the multi-level fusion Transformer model. The trained model will be directly applied to the online prediction stage to achieve accurate and real-time remaining service life prediction in various complex flight processes of the high-maneuverability UAV. During the offline training process, the back propagation algorithm uses the Adam optimizer to minimize the error to achieve nonlinear fitting.

[0098] In order to clearly describe the purpose and technical solution of the present invention, the present invention is further described in detail in conjunction with the following simulation examples.

[0099] Application Examples The simulation model established in the embodiment of the present invention is parameter fitted and set through normal flight experiments with multiple flight modes, variable flight speeds, and variable load weights of a quad-rotor drone and a battery temperature test box experiment. The quad-rotor drone has a frame wheelbase of 550mm and an empty weight of 1.357kg. The normal flight at 30°C and 120m is used as the baseline. The battery uses a 5100mAh 8C lithium polymer battery with an internal resistance of 27mΩ and a cut-off voltage of 10.3V. It uses T-MOTOR's Air Gear 350KV920 model motor and is equipped with 2 pairs of T-MOTOR T9545-B propellers and 4 T-MOTOR AIR20A electric regulators. The high-maneuverability quad-rotor simulation system uses PID to achieve automatic control, and takes into account the ground effect, gust disturbances, and sensor noise when it is close to the ground. The drone load weight range is within 0.5kg, the flight space is set to 1000m×1000m×200m, and the level flight speed is V hor，x ∈[-8, 8]m / s, V hor，y ∈[-8, 8]m / s, maximum descent speed V d，max =2m / s, maximum climbing speed V c，max =4.5m / s, the maximum throttle is 0.95. The temperature range is T∈[0,60]℃. A total of 850 flight data of four flight modes under different external influencing factors were obtained through simulation experiments with a sampling time of 1s. According to real data and actual needs, the prediction algorithm needs to predict the RUL of the next 48s through 32s of observed multi-sensor data, so 850 flight data are intercepted with a sliding time window of 80s and a sliding distance of 80s to form the original data set. The multi-sensor observation input dimension is 32×7, the external factor input dimension is 48×5, and the RUL prediction input and output dimensions are both 48×2.

[0100] The parameters of the multi-level fusion Transformer model that can achieve high-precision and real-time prediction requirements are set to D=512, the multi-scale mining factor selects three scales of k=1, 3, and 5, and the corresponding padding size p=0, 1, and 2. The encoder and decoder are composed of 6 basic layers, and each multi-head self-attention module has 8 attention heads. When training the model, the batch size is set to 80, the maximum number of iterations is 30, and the training set and test set size are set to 3:1:1. The Adam optimizer parameters are set to β1=0.9, β2=0.98, ε=10 -9The experimental results show that the proposed multi-level fusion Transformer model can quickly perform high-precision remaining useful life prediction, with an average prediction time of 0.017029s, the training set error converged to 0.1296, the validation set mean absolute deviation (MAD) converged to 24.20, and the test set MAD converged to 24.68. It has better results than LSTM and Transformer when predicting the remaining useful life at the next moment (single point prediction) and the remaining useful life for a period of time in the future (sequence prediction).

[0101] We further used the multi-level fusion Transformer to predict the remaining flight life of a quadrotor high-maneuverability flight. The prediction error was 58.4723, which was lower than the Transformer’s 61.3311. In addition, the remaining service life trend could be predicted more reasonably under various flight conditions. The specific prediction results are as follows: Figure 2 , Figure 3 As shown. Among them, Figure 2 A comparison diagram of a single remaining useful life prediction result provided according to an embodiment of the present invention, Figure 3 A schematic diagram of the remaining useful life prediction results of a sequence of a multi-level fusion Transformer model provided according to an embodiment of the present invention.

[0102] Embodiment 2:

[0103] Figure 4 Schematic diagram of a multi-rotor drone real-time remaining service life prediction system provided according to an embodiment of the present invention. Figure 4 As shown, the system includes: a construction module 10, a training module 20 and a prediction module 30.

[0104] Specifically, construction module 10 is used to construct a multi-level fusion Transformer model based on multi-sensor data of the multi-rotor UAV and target external factors.

[0105] Among them, the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on multi-sensor data based on a preset mining operator to obtain a temporal feature vector; the prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer that embeds the input tensor of the target external factors; the target external factors include the load weight, flight speed and ambient temperature of the multi-rotor drone.

[0106] The training module 20 is used to train the multi-level fusion Transformer model based on a preset training data set to obtain the trained multi-level fusion Transformer model.

[0107] The prediction module 30 is used to predict the remaining service life of the multi-rotor UAV by using the trained multi-level fusion Transformer model.

[0108] Optionally, the preset training data set includes: a prediction data set and an observation data set; the training module is further used to:

[0109] Based on the preset training data set, the multi-level fusion Transformer model is trained using the target loss function;

[0110] The objective loss function includes:

[0111]

[0112] is the network prediction output, is the tag value of the remaining service life, and PairwiseDistance() is the pixel-level Ohm distance.

[0113] The embodiment of the present invention provides a real-time remaining service life prediction system for a multi-rotor unmanned aerial vehicle, which takes into account the influence of input saturation of the multi-rotor unmanned aerial vehicle in an edge state on the remaining flight time, uses a Transformer model network to perform multi-sensor data feature layer fusion to reduce the dependence on a single discharge voltage signal, and greatly increases the prediction accuracy and speed in various flight maneuvering situations; and the Transformer model processes input data in parallel through an attention mechanism, which speeds up the processing speed, enhances the temporal feature representation and can process partially missing observation data; at the same time, on the basis of the Transformer model, a multi-scale processing mechanism is added to process the representation of fine time features, and a multi-level embedding layer is further added to add external influencing factors to the Transformer model network to solve the feasibility evaluation of a predetermined flight plan, thereby alleviating the technical problems of large remaining service life prediction deviation, poor real-time performance and inability to perform sequence prediction in the current data-driven unmanned aerial vehicle remaining service life prediction method.

[0114] Optionally, Figure 5 Schematic diagram of a multi-level fusion Transformer model according to an embodiment of the present invention. Figure 5 As shown, the multi-level fusion Transformer model includes an observation encoder; the observation encoder includes: a multi-head self-attention layer, a residual connection layer, a feed-forward fully connected layer, a residual connection layer and a multi-scale feature extraction layer;

[0115] The multi-scale feature extraction layer is also used to perform one-dimensional convolution calculation on multi-sensor data along the time dimension through a preset mining operator to obtain a time feature vector.

[0116] Specifically, Figure 5 As shown, the multi-level fusion Transformer model also includes a prediction decoder; the prediction decoder includes: a multi-head self-attention layer, a residual connection layer, a feedforward fully connected layer, a residual connection layer, an encoder-decoder attention layer, and a decoder attention layer of the input tensor embedding the target external factors.

[0117] An embodiment of the present invention further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method in the first embodiment when executing the computer program.

[0118] The embodiment of the present invention further provides a computer-readable medium having a non-volatile program code executable by a processor, wherein the program code enables the processor to execute the method in the above-mentioned embodiment 1.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the remaining service life of a multi-rotor drone in real time, characterized in that: include: Based on the multi-sensor data and target external factors of the multi-rotor UAV, a multi-level fusion Transformer model is constructed; the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on the multi-sensor data based on a preset mining operator to obtain a time feature vector; the prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer that embeds the input tensor of the target external factors; the target external factors include the load weight, flight speed and ambient temperature of the multi-rotor UAV; Training the multi-level fusion Transformer model based on a preset training data set to obtain a trained multi-level fusion Transformer model; Predicting the remaining service life of the multi-rotor UAV by using the trained multi-level fusion Transformer model; Based on the multi-sensor data of the multi-rotor UAV and the external factors of the target, a multi-level fusion Transformer model is constructed, including: By adding the multi-scale feature extraction layer to the observation encoder of the preset Transformer model, the observation encoder of the multi-level fusion Transformer model is constructed; The multi-scale feature extraction layer is further used to perform a one-dimensional convolution calculation on the multi-sensor data along the time dimension through the preset mining operator to obtain a time feature vector; The preset mining operator is M(·; Θ); Θ:=(k,p) determines the mining scale, k represents the kernel size, and p represents the filling size; D preset mining operators are set simultaneously in each scale; D is the dimension of the multi-sensor data.

2. The method according to claim 1, characterized in that: Based on the multi-sensor data of the multi-rotor drone and the external factors of the target, a multi-level fusion Transformer model is constructed, which also includes: By adding a decoder attention layer of the input tensor embedding the target external factor in the prediction decoder of the preset Transformer model, the prediction decoder of the multi-level fusion Transformer model is constructed.

3. The method according to claim 1, characterized in that The preset training data set includes: a prediction data set and an observation data set; training the multi-level fusion Transformer model based on the preset training data set includes: Based on the preset training data set, the multi-level fusion Transformer model is trained using a target loss function; The objective loss function includes: is the network prediction output, is the label value of the remaining service life, and PairwiseDistance() is the pixel-level ohmic distance.

4. A real-time remaining service life prediction system for multi-rotor drones, characterized in that: include: Building module, training module and prediction module; among them, The construction module is used to construct a multi-level fusion Transformer model based on multi-sensor data and target external factors of the multi-rotor UAV; the observation encoder of the multi-level fusion Transformer model includes a multi-scale feature extraction layer; the multi-scale feature extraction layer is used to perform convolution calculation on the multi-sensor data based on a preset mining operator to obtain a time feature vector; the prediction decoder of the multi-level fusion Transformer model includes a decoder attention layer that embeds the input tensor of the target external factors; the target external factors include the load weight, flight speed and ambient temperature of the multi-rotor UAV; The training module is used to train the multi-level fusion Transformer model based on a preset training data set to obtain a trained multi-level fusion Transformer model; The prediction module is used to predict the remaining service life of the multi-rotor UAV by using the trained multi-level fusion Transformer model; The multi-level fusion Transformer model includes an observation encoder; the observation encoder includes: a multi-head self-attention layer, a residual connection layer, a feedforward fully connected layer, a residual connection layer and the multi-scale feature extraction layer; The multi-scale feature extraction layer is further used to perform a one-dimensional convolution calculation on the multi-sensor data along the time dimension through the preset mining operator to obtain a time feature vector; The preset mining operator is M(·; Θ); Θ:=(k,p) determines the mining scale, k represents the kernel size, and p represents the filling size; D preset mining operators are set simultaneously in each scale; D is the dimension of the multi-sensor data.

5. The system according to claim 4, characterized in that The multi-level fusion Transformer model also includes a prediction decoder; the prediction decoder includes: a multi-head self-attention layer, a residual connection layer, a feedforward fully connected layer, a residual connection layer, an encoder-decoder attention layer, and a decoder attention layer of an input tensor embedded with the target external factor.

6. The system according to claim 4, characterized in that The preset training data set includes: a prediction data set and an observation data set; the training module is also used to: Based on the preset training data set, the multi-level fusion Transformer model is trained using a target loss function; The objective loss function includes: is the network prediction output, is the label value of the remaining service life, and PairwiseDistance() is the pixel-level ohmic distance.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 3 are implemented.

8. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that: The program code enables the processor to execute the method according to any one of claims 1 to 3.

Images