A method and system for modeling a multi-copter unmanned aerial vehicle based on machine learning

Abstract

The application provides a kind of multi-rotor unmanned aerial vehicle modeling method and system based on machine learning, wherein the method comprises: S1, collecting flight data of the rotor unmanned aerial vehicle, completing the cleaning of the preliminary data, and dividing the data set into training set and validation set;S2, according to the flight data, state quantity and action quantity are divided from the data;S3, according to the dimension of state quantity and action quantity, the corresponding deep neural network model is selected and the input and output are set;S4, using the adversarial imitation learning algorithm, defining the training termination condition, using the training data set to train the deep neural network model;S5, the trained neural network model is evaluated using the validation set, and the generalization accuracy of the model is evaluated.The application can reduce the requirement for data quantity and data coverage when modeling the rotor unmanned aerial vehicle data, and simplify the process of constructing the dynamics model and kinematics in advance by the personnel in the field with professional direction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-rotor unmanned aerial vehicle modeling, and particularly relates to a multi-rotor unmanned aerial vehicle modeling method and system based on machine learning. BACKGROUND

[0002] Multi-rotor unmanned aerial vehicles have been widely used in recent years due to their simple structure, low price, flexibility and other characteristics such as vertical take-off and landing in a small space. Building a digital model of a multi-rotor unmanned aerial vehicle plays a key role in its development. For example, more accurate modeling can achieve more accurate flight control. Generally, the modeling method of a multi-rotor unmanned aerial vehicle is based on Newton-Euler equation to build a dynamic and kinematic model. In the modeling process, it is usually assumed that the multi-rotor unmanned aerial vehicle is a uniform and symmetrical rigid body, the geometric center and the center of gravity are completely coincident, the mass and the moment of inertia are constant, and it is only subjected to gravity and propeller tension. These assumptions are often difficult to meet under the limitations of the actual conditions of the aircraft material, assembly accuracy, power stability (for example, low power will affect the control), etc. Therefore, during the actual flight of the multi-rotor unmanned aerial vehicle, the flight control result may deviate, and there is a safety hazard.

[0003] System identification method can combine actual operation data to calibrate and correct the kinematic mechanism model built under ideal conditions. This method can improve the consistency of the corrected kinematic model and the real model of the multi-rotor unmanned aerial vehicle. In order to complete the identification, this method needs to build a basic model structure (i.e. a dynamic model containing parameters) in advance, and needs to collect various data in various scenes, covering various flight attitudes and weather conditions. On the one hand, building a basic dynamic model still requires strong domain knowledge, and the dynamic model under ideal conditions which is too simplified may still have a large difference with the actual model; on the other hand, it is not easy to collect comprehensive data in the real scene.

[0004] In recent years, with the development of machine learning technology in the field of robotics, people in the field can use very limited data and deep neural networks to learn and imitate very efficient expert strategies for robots, mechanical dogs and unmanned aerial vehicles. Therefore, since the control strategy and the dynamic model have symmetry, the machine learning method can also be used to automatically learn the dynamic model from limited data without the need to build a basic model structure in advance. SUMMARY

[0005] To solve the above problems, the present application discloses a multi-rotor unmanned aerial vehicle modeling method and system based on machine learning.

[0006] The specific scheme is as follows:

[0007] A multi-rotor unmanned aerial vehicle modeling method based on machine learning comprises the following steps:

[0008] S1, collect flight data of the rotor unmanned aerial vehicle, split the trajectory segment according to the maximum length or task termination condition, complete the pre-data cleaning, and divide the data set into a training set and a validation set;

[0009] S2, according to the flight data of the rotor unmanned aerial vehicle and the basic field knowledge, the state quantity and the action quantity (usually the control lever quantity, the throttle quantity, etc.) are divided from the data;

[0010] S3, according to the dimension of the rotor unmanned aerial vehicle state quantity and action quantity, the corresponding deep neural network model is selected and the input and output are set;

[0011] S4, using the adversarial imitation learning algorithm, defining the training termination condition, using the training data set to train the deep neural network model defined in step S3, and using part of the subset of the training set to periodically evaluate the current trained model during the training process, thereby realizing the modeling of the rotor unmanned aerial vehicle;

[0012] S5, using the validation set to evaluate the trained neural network model, and evaluating the generalization accuracy of the model.

[0013] Further, in step S1, after collecting a certain amount of flight data (for example, 1 hour of flight data accumulated in wind and light wind environment), the flight data is divided into trajectories, for example, the maximum trajectory length is not more than 10 min as the division basis, the adjacent trajectories should maintain a certain amount of overlap, and the training set and the validation set are obtained by sampling according to the ratio of 7:3 or 8:2.

[0014] Further, in step S2, the pose, speed, and blade speed are set as the state quantity S, and the throttle and lever quantity in each direction are set as the action quantity a.

[0015] Further, in step S3, a multilayer perceptron model and a residual module are selected as the rotor unmanned aerial vehicle dynamics model, the input is the current state S and the executed action a, and the output is the next state S'. For example, a 5-layer multilayer perceptron is set, the middle layer unit number is set to [256, 256, 512, 512, 256], and a residual module is added between layers.

[0016] Further, step S4 is specifically:

[0017] S41, a multi-layer perceptron neural network MLP is established as a discriminator for discriminating the credibility of a generated (S, a), the final output of the discriminator is a real number between 0 and 1, the closer to 1, the more like real data, the closer to 0, the more like generated data, and a convergence condition (for example, the discriminator outputs close to 0.5 on both generated data and real data) and a maximum number of iterations (for example, 5000 times) are set;

[0018] S42, sample a batch of starting points of trajectories from real data as initial points, and interact with the defined neural network dynamics model, that is, input the current (S, a) to obtain S', wherein the action amount a is obtained from historical data or an additional neural network is established, until a specific trajectory length is reached or a termination condition is met, and a batch of trajectory sequences are generated;

[0019] S43, update the discriminator with (S, a) in the generated trajectory sequence and (S, a) in the trajectory sequence in the real data, record the real data set as D, and the generated data set in the training process as D', and the update target is as follows:

[0020]

[0021] Wherein f is the discriminator, f(S, a), f((S, a) ′ ) respectively represent the credibility of the discriminator output on a single real data and generated data;

[0022] S44, score each generated (S, a) with the updated discriminator as a single-step reward r, and update the neural network dynamics model using a reinforcement learning algorithm, wherein the reinforcement learning algorithm can be PPO or SAC when in continuous action space;

[0023] S45, repeat steps S42 to S44 until the maximum number of iterations or the convergence condition is reached.

[0024] The present application learns the corresponding aerodynamic model from a limited number of actual flight data of a rotor unmanned aerial vehicle, that is, a data modeling method in model construction. The previous data modeling method mainly uses the technical means of fitting part of the parameters by constructing the basic dynamics equation in advance, which greatly depends on the correctness of the constructed equation and whether the reserved parameters are reasonable. Another pure data modeling method uses deep learning, that is, a supervised learning method in machine learning to model. In order to obtain good results, this method usually needs to collect a large amount of data and cover various states and actions, which has high requirements for the data collection process. The machine learning method can avoid prior explicit modeling, especially in the generative adversarial imitation method, which can restore a model with high fidelity from relatively limited data.

[0025] The generalization accuracy of the model in the step S5 is evaluated, and specifically includes:

[0026] (1) the average absolute error of multiple steps on all trajectories in the verification set;

[0027] (2) the absolute error and relative proportion of the last step on all trajectories in the verification set;

[0028] (3) the visualization comparison of typical trajectories.

[0029] A multi-rotor unmanned aerial vehicle modeling system based on machine learning comprises:

[0030] A data integration and cleaning unit is configured to collect flight data of the multi-rotor unmanned aerial vehicle, split trajectory segments according to a maximum length or a task termination condition, complete preliminary data cleaning, and divide the data set into a training set and a verification set;

[0031] A state-action definition unit is configured to divide state quantities and action quantities from the data according to the flight data of the multi-rotor unmanned aerial vehicle and basic domain knowledge;

[0032] A neural network model selection unit is configured to select a corresponding deep neural network model according to the dimensions of the state quantities and the action quantities of the multi-rotor unmanned aerial vehicle and set inputs and outputs;

[0033] An adversarial imitation learning unit is configured to define a training termination condition by using an adversarial imitation learning algorithm, train the defined deep neural network model using the training data set, and periodically evaluate the current trained model using a part of the training set during the training process, so as to realize modeling of the multi-rotor unmanned aerial vehicle;

[0034] A model evaluation and verification unit is configured to evaluate the trained neural network model by using the verification set and evaluate the generalization accuracy of the model.

[0035] The present application has the following advantages:

[0036] 1. The present application proposes a general multi-rotor unmanned aerial vehicle modeling method based on machine learning, which can improve the modeling accuracy of the multi-rotor unmanned aerial vehicle while reducing the modeling difficulty, i.e., reducing the requirements for data quantity and data coverage during data modeling of the multi-rotor unmanned aerial vehicle, and simplifying the process of analyzing dynamic characteristics and constructing dynamic models and kinematics by personnel in the field with professional knowledge in advance.

[0037] 2、The application can directly use a deep neural network as a basic model structure without specifying a dynamic model in advance; equations can be used in parts where aerodynamics is clear, and a neural network model can be used in unclear parts, and learning is performed in combination with data and the learning method in the application. The model learned in the application is a digital model with high fidelity, which can be used to construct flight training simulation, train flight control strategies, or develop new types of rotor unmanned aerial vehicles based on the model. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 is a flowchart of the application.

[0039] Figure 2 is a detailed process diagram for adversarial imitation learning in the application. DETAILED DESCRIPTION

[0040] The application will be further illustrated in conjunction with the specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the application and not to limit the scope of the application.

[0041] The application proposes a multi-rotor unmanned aerial vehicle modeling method based on machine learning, which learns a rotor unmanned aerial vehicle model from pre-collected flight data of the rotor unmanned aerial vehicle based on adversarial imitation learning. The flight data can be collected in some basic flight tasks, and usually includes basic data of several processes such as take-off, landing, low-speed, medium-speed, and high-speed running. Figure 1 As shown in the specific operation steps of the application:

[0042] 110、Obtain flight data of the rotor unmanned aerial vehicle, divide the trajectory segments according to the maximum length or task termination condition, complete the cleaning of the preliminary data, and divide the data set into a training set and a validation set;

[0043] 210、According to the flight data of the rotor unmanned aerial vehicle and the basic field knowledge, divide the state quantity and the action quantity (usually the lever quantity, throttle quantity, etc.) from the data;

[0044] 220、According to the dimensions of the rotor unmanned aerial vehicle state quantity and action quantity, select the corresponding deep neural network model and set the input and output;

[0045] 230、Use the generative adversarial imitation learning algorithm, define the training termination condition, use the training data set to train the deep neural network model defined in step

[220] , and periodically evaluate the current trained model using part of the training set during the training process, thereby realizing the modeling of the rotor unmanned aerial vehicle;

[0046] 310、Use the validation set to evaluate the trained neural network model, and evaluate the generalization accuracy of the model.

[0047] wherein, after collecting a certain amount of flight data (for example, 1 hour of flight data accumulated in a wind and light wind environment), the flight data is trajectory segmented, for example, according to a maximum trajectory length of no more than 10 min as the division basis, a certain amount of overlapping part between adjacent trajectories should be maintained in the segmented trajectory, and a training set and a verification set are obtained according to a 7:3 or 8:2 ratio sampling, and the pose, speed, blade speed, etc. are set as the state quantity S, and the throttle, lever quantity in each direction, etc. are set as the action quantity a. This process corresponds to steps

[110] and

[210] in the present application. Then, a multi-layer perception model and a residual module are selected as the rotor unmanned aerial vehicle dynamics model, the input is the current state S and the executed action a, and the output is the next state S'. For example, a 5-layer multi-layer perception is set, the number of units in the middle layer is set to [256, 256, 512, 512, 256], and a residual module is added between layers. This process corresponds to step

[220] in the present application. After the data set is prepared and the neural network model is set, the step

[230] of performing adversarial imitation learning training is started. The detailed process of adversarial imitation learning is as shown in Figure 2

[0048]

A100

[0049]

A200

[0050]

A300

[0051]

[0052] wherein f is the discriminator, f(S, a), f((S, a) ′ ​respectively represent the confidence of the discriminator output on a single piece of real data and generated data.

[0053]

A400

[0054] Repeat

A200

A400

[0055] In step

[310] of the present application, after the model training is completed, the model can be evaluated on the validation set using a variety of evaluation indicators:

[0056] (1) The average absolute error of multiple steps on all trajectories in the validation set;

[0057] (2) The absolute error and relative proportion of the last step on all trajectories in the validation set;

[0058] (3) Visualization comparison of typical trajectories.

[0059] The optimal model after model evaluation can be used as the rotor unmanned aerial vehicle dynamics model constructed in the present application.

[0060] The technical means disclosed in the present application scheme is not limited to the technical means disclosed in the above embodiments, but also includes the technical solutions composed of any combination of the above technical features. It should be noted that for ordinary skilled persons in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered within the protection scope of the present application.

Claims

1. A method for modeling a multi-copter unmanned aerial vehicle based on machine learning, the method comprising: The method comprises the following steps: S1, collecting flight data of the rotor unmanned aerial vehicle, dividing the trajectory segments according to the maximum length or task termination condition, completing the preliminary data cleaning, and dividing the data set into a training set and a verification set; S2, dividing state quantities and action quantities from the flight data of the rotor unmanned aerial vehicle; S3, selecting a corresponding deep neural network model according to the dimensions of the state quantities and the action quantities of the rotor unmanned aerial vehicle and setting inputs and outputs; S4, using an adversarial imitation learning algorithm, defining a training termination condition, using the training data set to train the deep neural network model defined in step S3, periodically evaluating the currently trained model using a part of the training set during the training process, thereby realizing modeling of the rotor unmanned aerial vehicle; S5, evaluating the trained neural network model using the verification set and evaluating the generalization accuracy of the model; The adjacent trajectories in the trajectory divided in step S1 have overlapping parts; in step S2, the pose, velocity and blade speed are set as state quantities S, and the throttle and lever quantities in each direction are set as action quantities a; in step 3, a multilayer perceptron model and a residual module are selected as the rotor unmanned aerial vehicle dynamics model, the input is the current state quantity S and the executed action quantity a, and the output is the next state quantity S'; Step S4 specifically comprises: S41, establishing a multilayer perceptron neural network MLP as a discriminator for judging the credibility of a certain generation (S, a), the final output of the discriminator is a real number between 0 and 1, the closer to 1, the more like real data, the closer to 0, the more like generated data, and a convergence condition and a maximum number of iterations are set; S42, sampling a batch of starting points of trajectories from real data as initial points and interacting with the defined neural network dynamics model, i.e. inputting the current (S, a) to obtain S', wherein the action quantity a is obtained from historical data or an additional neural network is established, until a specific trajectory length is reached or a termination condition is met, and a batch of trajectory sequences are generated; S43, updating the discriminator with (S, a) in the generated trajectory sequence and (S, a) in the trajectory sequence in the real data, taking the real data set as D and the generated data set in the training process as D', and the update target is as follows: ； wherein is a discriminator, respectively represent the confidence of the discriminator output on a single real data and generated data; S44, scoring each pair of generated (S, a) using the updated discriminator as a single-step reward r, and updating the neural network dynamics model using a reinforcement learning algorithm, wherein the reinforcement learning algorithm can be PPO or SAC in a continuous action space; S45, repeating steps S42 to S44 until the maximum number of iterations or the convergence condition is reached.

2. The machine learning based multi-copter unmanned aerial vehicle modeling method of claim 1, wherein, The generalization accuracy of the model is evaluated in step S5, specifically including: (1) the average absolute error of multiple steps on all trajectories in the verification set; (2) the absolute error and relative proportion of the last step on all trajectories in the verification set; (3) visualization comparison of typical trajectories.

3. A multi-copter drone modeling system for implementing the method of any of claims 1 and 2, wherein, The method comprises: A data integration and cleaning unit is configured to collect flight data of the rotor unmanned aerial vehicle, split trajectory segments according to a maximum length or a task termination condition, complete preliminary data cleaning, and divide the data set into a training set and a verification set; A state-action definition unit is configured to divide state quantities and action quantities from the data according to the flight data of the rotor unmanned aerial vehicle and basic domain knowledge; A neural network model selection unit is configured to select a corresponding deep neural network model according to the dimensions of the state quantities and the action quantities of the rotor unmanned aerial vehicle and set inputs and outputs; An adversarial imitation learning unit is configured to define a training termination condition by using an adversarial imitation learning algorithm, train the defined deep neural network model using the training data set, and periodically evaluate the currently trained model using a part of the training set during the training process, thereby realizing modeling of the rotor unmanned aerial vehicle; A model evaluation and verification unit is configured to evaluate the trained neural network model using the verification set and evaluate the generalization accuracy of the model.

Images