Variable-pitch unmanned aerial vehicle simulation control method based on digital twinning and unmanned aerial vehicle

Abstract

The invention relates to the field of unmanned aerial vehicles, in particular to a variable-pitch unmanned aerial vehicle simulation control method based on digital twinning and an unmanned aerial vehicle. The method comprises the following steps: constructing a digital twinborn body of the variable-pitch unmanned aerial vehicle in a simulation environment; the digital twin body comprises a dynamics simulation model, a motor-paddle power consumption simulation model and a battery simulation model; training a control model of the variable-pitch unmanned aerial vehicle based on the simulation flight data of the digital twin as training data; and after the trained control model is deployed to the variable-pitch unmanned aerial vehicle, actual pose data of the variable-pitch unmanned aerial vehicle is used as input, and an accelerator instruction and an angular velocity instruction of the variable-pitch unmanned aerial vehicle are output. The adaptive capacity of the variable-pitch unmanned aerial vehicle under different tasks and environment conditions is enhanced, and the performance of the variable-pitch unmanned aerial vehicle under high maneuvering control is improved.

Description

Technical Field

[0001] The present application relates to the field of unmanned aerial vehicles (UAVs), and in particular to a variable pitch UAV simulation control method and a UAV based on digital twins. Background Art

[0002] As an emerging research field, drones have demonstrated strong mission execution capabilities and are applied in various fields. Common rotor drones need to adjust the control force by changing the rotor speed. Due to the low adjustment bandwidth of the speed, the control force of common rotor drones is slowly adjusted, which fundamentally limits their maneuverability and stability. In contrast, variable-pitch drones can directly adjust the angle of attack of the rotor through the servo, thereby greatly improving the response bandwidth of the control force and having the characteristics of high maneuverability. In addition, since its angle of attack can be negative, variable-pitch drones can easily achieve high-maneuverability flight.

[0003] There is little research and deployment application of high maneuverability control for variable pitch UAVs in the existing technology to solve the maneuverability control and deployment problems existing in the existing technology. Summary of the invention

[0004] Based on this, it is necessary to provide a variable-pitch UAV simulation control method and UAV based on digital twin to address the above technical problems.

[0005] In a first aspect, an embodiment of the present invention proposes a variable pitch UAV simulation control method based on digital twin, the method comprising: Constructing a digital twin of a variable pitch UAV in a simulation environment; the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model, and a battery simulation model; Training a control model of the variable pitch UAV based on the simulated flight data of the digital twin as training data; After the trained control model is deployed to the variable pitch UAV, the actual position and posture data of the variable pitch UAV is used as input to output the throttle command and angular velocity command of the variable pitch UAV.

[0006] In some embodiments, constructing the dynamics simulation model includes: Obtaining a force analysis formula for the variable pitch UAV; Based on the force analysis formula, the dynamic simulation model is constructed using blade element momentum theory.

[0007] In some embodiments, the dynamics simulation model is:

[0008] in, T is the lift of the blade, D is the resistance of the blade,ρ is the air density, V is the inflow velocity of the airflow coordinate relative to the blade, is the blade angle, and are the simplified thrust and torque coefficients respectively.

[0009] In some embodiments, constructing the power consumption simulation model of the motor-blade includes: Based on the dynamic simulation model, the instantaneous power consumption of the motor-blade combination is obtained; Based on the instantaneous power consumption, a power consumption simulation model of the motor-blade is constructed.

[0010] In some embodiments, the power consumption simulation model of the motor-blade is:

[0011] in, For the i The function of the angle of the blade and the instantaneous power consumption, P is the instantaneous power consumption of the motor-propeller, For the i The angle of the propeller blades.

[0012] In some embodiments, constructing the battery simulation model includes: Collect battery voltage, instantaneous power and energy consumption during the flight of UAVs with different variable pitches; Curve fitting is performed according to the relationship between the battery voltage, instantaneous power and energy consumption, and fitting parameters are extracted, and the fitting parameters are used to construct a battery simulation model.

[0013] In some embodiments, the battery simulation model is:

[0014] in, is the battery voltage, is the blade angle, For energy consumption.

[0015] In some embodiments, the simulated flight data includes the digital twin's own position, body speed, body angular velocity, and attitude; after deploying the trained control model to the variable pitch UAV, using the actual position and attitude data of the variable pitch UAV as input, outputting the throttle command and angular velocity command of the variable pitch UAV includes: The control model is constructed based on a neural network, and the control model comprises an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer is used to input the self-position, body speed, body angular velocity, and posture, and the output layer is used to output the throttle command and angular velocity command of the variable pitch UAV in different states; The control model is trained using the simulated flight data based on an improved reinforcement learning algorithm based on PPO.

[0016] In a second aspect, an embodiment of the present invention provides a variable pitch UAV, comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the steps described in the first aspect.

[0017] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and the processor implements the steps described in the first aspect when executing the computer program.

[0018] The above-mentioned method, variable-pitch UAV and storage medium construct a digital twin of the variable-pitch UAV in a simulation environment; the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model and a battery simulation model; based on the simulated flight data of the digital twin as training data, the control model of the variable-pitch UAV is trained; after the trained control model is deployed to the variable-pitch UAV, the actual position data of the variable-pitch UAV is used as input to output the throttle and angular velocity instructions of the variable-pitch UAV, thereby enhancing the adaptability of the variable-pitch UAV under different tasks and environmental conditions and improving the performance of the variable-pitch UAV under high-maneuverability control. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of a flow chart of a variable pitch UAV simulation control method based on digital twin in one embodiment; Figure 2 A battery voltage drop diagram under different power consumption of a battery simulation model in an embodiment; Figure 3 A comparison diagram of the simulated voltage of a battery simulation model and the sampled voltage of a real battery in one embodiment; Figure 4 Schematic diagram of the status result of a variable pitch drone completing a pot-brushing maneuver in one embodiment. DETAILED DESCRIPTION

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of the present invention. For ordinary technicians in this field, the present invention can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.

[0021] As shown in the present invention and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular, but also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list, and the method or device may also include other steps or elements.

[0022] Although the present invention makes various references to certain modules in the apparatus according to the embodiments of the present invention, any number of different modules can be used and run on a computing device and / or processor. The modules are only illustrative, and different aspects of the apparatus and method can use different modules.

[0023] It should be understood that when a unit or module is described as being "connected," "coupled" to other units, modules, or blocks, it may refer to being directly connected or coupled, or communicating with other units, modules, or blocks, or there may be intermediate units, modules, or blocks, unless the context clearly indicates otherwise. The term "and / or" as used herein may include any and all combinations of one or more of the associated listed items.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a variable pitch UAV simulation control method based on digital twin, comprising the following steps: S102, constructing a digital twin of the variable pitch UAV in a simulation environment; the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model, and a battery simulation model; S104, training a control model of the variable pitch UAV based on the simulated flight data of the digital twin as training data; S106, after deploying the trained control model to the variable pitch UAV, using the actual position and posture data of the variable pitch UAV as input, outputting throttle and angular velocity instructions of the variable pitch UAV.

[0025] In this embodiment, a digital twin of a variable-pitch UAV is constructed in a simulation environment; the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model, and a battery simulation model; the control model of the variable-pitch UAV is trained based on the simulated flight data of the digital twin as training data; after the trained control model is deployed to the variable-pitch UAV, the actual position data of the variable-pitch UAV is used as input to output throttle and angular velocity commands of the variable-pitch UAV, thereby enhancing the adaptability of the variable-pitch UAV under different tasks and environmental conditions, and improving the performance of the variable-pitch UAV under high-maneuverability control.

[0026] In step S102, in order for the control model based on reinforcement learning to obtain better control effects, it is necessary to establish highly similar modeling of the real model in the simulation environment. First, it is necessary to establish a relatively real simulation environment and build a digital twin of the variable-pitch UAV in the simulation environment, where the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model, and a battery simulation model.

[0027] Wherein, constructing the dynamics simulation model comprises: According to the force analysis of the variable pitch UAV, the following equation can be obtained:

[0028] in, m is the mass of the variable pitch UAV, R is the rotation matrix from the body to the world coordinate system, is the acceleration due to gravity, is the body angular velocity of the variable pitch UAV, is the derivative of the body’s angular velocity; is the inertia matrix of the variable pitch UAV, is the torque of the variable pitch UAV, is the power distribution matrix, air resistance The calculation formula is:

[0029] in, is the air resistance coefficient, ρ is the air density, S is the wind-exposed area of ​​the variable-pitch UAV, is the airspeed of the variable pitch UAV in the airflow coordinate. Among them, since the variable pitch UAV is based on a constant speed and the thrust change achieved by the variable pitch, the torque change caused by the change in rotor speed will be relatively small and is ignored in this formula.

[0030] Then the dynamic simulation of the variable pitch blade is carried out. The blade element theory simplifies the wind turbine blade into a finite number of blade elements, which are superimposed radially. Therefore, the three-dimensional aerodynamic characteristics of the wind wheel can be obtained by integrating the aerodynamic characteristics of the blade elements along the radial direction. According to the blade element momentum theory, we have:

[0031] In the formula, T is the lift of the blade, D is the resistance of the blade, is the air density, V is the inflow velocity of the airflow coordinate relative to the blade, is the blade length, is the axial induction factor, is the tangential induction factor, is the rotation speed. Since a variable pitch quadrotor is used, the rotation speed is a constant value, which can be simplified to:

[0032] in, and are the simplified thrust and torque coefficients respectively.

[0033] Constructing the power consumption simulation model of the motor-blade includes: Since the blades do not do work in the thrust direction, the power consumption of the variable pitch drone is approximately equal to the power consumption in the torque direction. The torque calculated above is the blade resistance. D , the instantaneous power consumption of the motor-propeller combination can be calculated P It can be expressed as:

[0034] in is the mechanical resistance of the motor, It is the efficiency ratio of mechanical work to electrical power, which is about 70%-85%. It has a certain relationship with the coordination of the motor blades and the rotation speed of the blades. The parameters need to be adjusted when used. In addition, since the speed of the variable pitch drone does not change significantly during operation, and the blade size is large, the power consumption of mechanical resistance is relatively small, so the product of efficiency and speed can be regarded as a constant, that is, instantaneous power consumption. P It can be simplified as:

[0035] Then, referring to the dynamic simulation of the propeller blade mentioned above, we can know the relationship between instantaneous power and propeller blade angle:

[0036] in, For the i The function of the angle of the blade and the instantaneous power consumption, P is the instantaneous power consumption of the motor-propeller, For the i The angle of the propeller blades.

[0037] Normally, the dynamics of the battery are not taken into account when simulating a drone, but for more detailed thrust simulation, the impact on the dynamic changes inside the battery must be considered.

[0038] Wherein, constructing the battery simulation model includes: The internal chemical reaction of the battery discharge process is relatively complex and is affected by many surrounding factors. It is time-varying and nonlinear. In order to establish a relatively accurate battery model, the Thevenin model is used, using an internal resistance and a parallel RC module to respectively reflect the internal resistance and polarization of the battery. U t is the load terminal voltage of the battery, U 1 is the polarization voltage, U 0 is the battery voltage, the actual total voltage of the battery, in V, I is the charge and discharge current of the battery, in A, R 0 is the ohmic internal resistance, R 1 is the polarization internal resistance of the battery, in ohms , time constant , t is the time in seconds, and the relationship between the load voltage and output voltage of the battery is constructed as follows:

[0039] Since current generally has a relatively linear relationship with power consumption, , substituting into the above formula we get:

[0040]

[0041] Indicates the electrical power output by the battery. represents the combined coefficient, represents the derivative of the polarization voltage.

[0042] In addition, we also know that the output voltage of the battery is related to the energy consumed, which is the integral of power over time. ,Finally, different battery-motor-propeller combinations were tested through real experiments, and the battery voltage, instantaneous power and energy consumption of different variable-pitch UAVs during flight were collected.

[0043] According to the relationship between the battery voltage, instantaneous power and energy consumption, curve fitting is performed, and fitting parameters are extracted to construct a battery simulation model:

[0044] in, is the battery voltage, is the blade angle, For energy consumption.

[0045] The above process simulates the voltage change response of the battery when facing a sudden change in the power of the drone, so as to better complete the simulation, and also simulate the flight time of the drone under fixed power consumption.

[0046] Figure 2 It is a battery voltage drop diagram under different power consumption of the battery simulation model used in the embodiment of the present invention. Lines of different colors and styles represent the battery voltage change and the continuous running time of a 6S 1500mAh battery under different power consumption. Figure 3 This is a comparison diagram of the simulated voltage of the battery simulation model used in the embodiment of the present invention and the sampled voltage of the actual battery.

[0047] In general, by combining the dynamic simulation model of the above-mentioned UAV, the power consumption simulation model of the motor-blade, and the battery simulation model and deploying them in a simulation environment, the difference between the data in the simulation environment and the real data is greatly reduced.

[0048] Traditional drone control systems usually rely on pre-programmed path planning or remote control. This method has problems such as poor adaptability to environmental changes and low optimization of flight performance. Reinforcement learning, as an autonomous learning method based on trial and error, can continuously optimize strategies through interaction with the environment. Reinforcement learning is essentially learning the mapping relationship of data, so it has high requirements for the accuracy of the model. At the same time, in order to reduce overfitting and enhance reinforcement learning's learning of rules, the reinforcement learning environment also needs to add a lot of noise to enhance its robustness. Based on the simulation environment established in the high-parallel environment mentioned above, a large amount of training data similar to real data can be quickly obtained.

[0049] In step S104, a modified reinforcement learning algorithm based on PPO is used to implement learning. The PPO (Proximal Policy Optimization) proximal policy optimization algorithm is an on-policy algorithm, but it has problems such as low sample utilization and slow training. Therefore, a variety of improvements are made on the basis of the original PPO algorithm, so that it can better meet the needs of high maneuverability control and the migration of real environments.

[0050] In terms of network design, in order to cope with complex simulation environments and high dynamic control requirements, a deep fully connected neural network architecture is adopted. The entire network consists of an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer receives the state representation of the variable pitch UAV, including its own position. p , Body speed v , body angular velocity Ω and attitude R The two hidden layers are responsible for feature extraction and nonlinear mapping. The output layer generates the probability distribution of each action and uses the tanh function to limit the action value to the range of -1~1, which is used to give the throttle command and angular velocity command of the variable pitch UAV in different states.

[0051] In addition, considering the control requirements for different tasks and states in the control algorithm, two target networks are designed: one target network is used to estimate the state value function and optimize the policy loss function; the other target network is used to introduce an auxiliary optimization objective, the entropy regularization of the policy, to encourage more extensive exploration. In each update, the two target networks are optimized alternately by gradient descent. This alternating optimization method enables the algorithm to balance exploration and development more during policy updates, reducing the risk of overfitting and policy collapse. In addition, the target network's parameter update frequency is low, and the training instability problem caused by frequent updates is avoided by delaying the policy update. Such a design not only improves the robustness of the model, but also effectively copes with complex dynamic environments.

[0052] Secondly, a fixed-size buffer is used to store experience tuples. Whenever the variable-pitch UAV interacts with the environment, the experience tuple (state, action, reward, next state) is stored in the buffer. During the training process, a batch of experience tuples are randomly sampled from the buffer for training. In addition, data from the normal operation process of the variable-pitch UAV on the real target platform is collected, and an independent additional buffer of fixed size is established to provide a part of offline data for mapping calibration, thereby further optimizing the mapping problem in the simulation environment and reducing the domain migration problem faced by the control network. Since the PPO algorithm can use KL divergence to achieve importance sampling, data learning under different strategies can be automatically completed. KL divergence, also called relative entropy, can be used to measure the difference between two distributions.

[0053] The goal of this application is to realize the automatic large-scale maneuver flight of a variable-pitch UAV. Therefore, the more complex maneuvering actions are selected to design the Reward algorithm, and the complex maneuvering actions are split into multiple simple indicators, including basic distance, speed, angular velocity, angle difference, etc.

[0054] Through the refined modeling mentioned above, the environment simulation is made more realistic, thereby improving the learning efficiency of the reinforcement learning algorithm. By training in an accurate simulation environment, the variable pitch drone can more quickly understand the feedback under different states and adjust the strategy accordingly, thereby completing the search for a better strategy more quickly.

[0055] In step S106, after completing the network training in the virtual environment, a CUDA-trained model is obtained, which needs to be migrated to an RKNN-based network to meet the requirements of NPU deployment and accelerate computing.

[0056] In actual deployment, the control system is also divided into two layers. The upper layer is used to realize external sensor data acquisition and NPU network-based onboard computer calculation. It sends the throttle command and angular velocity command of the variable pitch UAV to the lower MCU. The lower MCU is used to realize: 1) motor speed control based on the use of high-speed electric speed control to obtain motor speed and voltage information; 2) the inner loop controller of the UAV angular velocity following. It outputs the actual motor and the servo command of the target pitch controlled by the current servo.

[0057] Sensors are used to sense the state information of the surrounding environment and transmit it to the control model. The variable pitch UAV needs to obtain the measurement information of its own sensors, including the inertial guidance unit and the optical flow unit. The raw data is calculated based on the Kalman filter to obtain its own state information including position. p , Body speed v , body angular velocity Ω and attitude R ,The data sources for high-precision indoor motion also include high-precision external sensor devices such as Vicon, etc.,The algorithm of this module has strong fusion capability, which can fuse information from different sources, different delays, and different noises, thereby realizing the estimation of its current posture.

[0058] Figure 4 This is a schematic diagram of the state result of the variable pitch UAV completing the pot-brushing maneuver in an embodiment of the present invention, and it is expected that the nose of the aircraft rotates rapidly around the center point.

[0059] It should be understood that, although the various steps in the above flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the above flowchart may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0060] In one embodiment, an embodiment of the present invention provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in any one of the above-mentioned embodiments of the variable-pitch UAV simulation control method based on digital twins are implemented.

[0061] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0062] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A variable pitch UAV simulation control method based on digital twin, characterized in that: The method comprises: Constructing a digital twin of a variable pitch UAV in a simulation environment; the digital twin includes a dynamic simulation model, a motor-blade power consumption simulation model, and a battery simulation model; Training a control model of the variable pitch UAV based on the simulated flight data of the digital twin as training data; After the trained control model is deployed to the variable pitch UAV, the actual position and posture data of the variable pitch UAV is used as input to output the throttle command and angular velocity command of the variable pitch UAV.

2. The method according to claim 1, characterized in that Constructing the dynamic simulation model includes: Obtaining a force analysis formula for the variable pitch UAV; Based on the force analysis formula, the dynamic simulation model is constructed using blade element momentum theory.

3. The method according to claim 1, characterized in that The dynamic simulation model is: in, T is the lift of the blade, D is the resistance of the blade, ρ is the air density, V is the inflow velocity of the airflow coordinate relative to the blade, is the blade angle, and are the simplified thrust and torque coefficients respectively.

4. The method according to claim 3, characterized in that Constructing the power consumption simulation model of the motor-blade includes: Based on the dynamic simulation model, the instantaneous power consumption of the motor-blade combination is obtained; Based on the instantaneous power consumption, a power consumption simulation model of the motor-blade is constructed.

5. The method according to claim 4, characterized in that The power consumption simulation model of the motor-blade is: in, For the i The function of the angle of the blade and the instantaneous power consumption, P is the instantaneous power consumption of the motor-propeller, For the i The angle of the propeller blades.

6. The method according to claim 1, characterized in that Constructing the battery simulation model includes: Collect battery voltage, instantaneous power and energy consumption during the flight of UAVs with different variable pitches; Curve fitting is performed according to the relationship between the battery voltage, instantaneous power and energy consumption, and fitting parameters are extracted, and the fitting parameters are used to construct a battery simulation model.

7. The method according to claim 1, characterized in that The battery simulation model is: in, is the battery voltage, is the blade angle, For energy consumption.

8. The method according to claim 1, characterized in that The simulated flight data includes the digital twin's own position, body speed, body angular velocity, and posture; after deploying the trained control model to the variable pitch UAV, using the actual position and posture data of the variable pitch UAV as input, outputting the throttle command and angular velocity command of the variable pitch UAV includes: The control model is constructed based on a neural network, and the control model comprises an input layer, a first hidden layer, a second hidden layer, and an output layer. The input layer is used to input the self-position, body speed, body angular velocity, and posture, and the output layer is used to output the throttle command and angular velocity command of the variable pitch UAV in different states; The control model is trained using the simulated flight data based on an improved reinforcement learning algorithm based on PPO.

9. A variable pitch UAV, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images