A quadrotor unmanned aerial vehicle obstacle avoidance method based on a digital twin system

Abstract

A kind of quadrotor unmanned aerial vehicle obstacle avoidance method based on digital twin system, comprising the following steps: step 1: establishing unmanned aerial vehicle digital twin system;Step 2: design feedforward-feedback control strategy based on model predictive control in physical space;Step 3: design and deploy offline trajectory planning framework in virtual space;The present application improves the quality of unmanned aerial vehicle trajectory planning and motion control accuracy based on the establishment of unmanned aerial vehicle digital twin system through the design of trajectory planning framework and feedforward-feedback control strategy.

Description

A method for obstacle avoidance of quadrotor UAVs based on digital twin systems Technical Fields

[0001] This invention belongs to the field of obstacle avoidance technology for quadcopter drones, and specifically relates to an obstacle avoidance method for quadcopter drones based on a digital twin system. Background Technology:

[0002] With the development of the low-altitude economy, quadcopter drones are widely used in various scenarios such as agricultural plant protection, commercial delivery, military operations, and disaster relief. In actual flight missions, drones are required to have autonomous flight capabilities, that is, to simultaneously perform motion control, trajectory planning, and obstacle perception tasks. Achieving autonomous flight for drones in real-world scenarios faces challenges such as heavy computational burden, high hardware costs, and severe environmental interference.

[0003] Digital twin technology is commonly used in industrial production to create high-precision twins of physical processes in actual production, enabling remote operation and status monitoring of industrial processes. Designing a digital twin system for drones leverages the abundant computing resources of virtual space to transfer the high computational burden of real-time perception and planning during obstacle avoidance to virtual space, and reduces the hardware costs required for drones through high-precision mapping relationships.

[0004] Obstacle avoidance flight for drones consists of two tasks: trajectory planning and motion control. In obstacle avoidance, trajectory planning algorithms can output collision-free trajectories, but due to real-time and lightweight requirements in practical applications, the planning results are often not optimal. Mainstream trajectory planning frameworks use depth cameras to perceive obstacles and make real-time planning decisions. However, this vision-based framework is susceptible to lighting conditions in real-world tasks and cannot guarantee the stability of obstacle avoidance flight.

[0005] Completing high-quality obstacle avoidance tasks requires precise motion control. Model predictive control and other methods are commonly used to control UAVs. MPC is suitable for handling multiple inputs and outputs and various constraints in UAV systems, such as safety restrictions and dynamic constraints. Therefore, MPC can be applied to physical and virtual UAVs to enable them to accurately track trajectories. However, MPC has the limitation of being difficult to solve.

[0006] To address these limitations, a quadrotor UAV obstacle avoidance method based on a digital twin system is proposed to improve trajectory planning quality and control accuracy in obstacle avoidance tasks. Invention content:

[0007] The purpose of this invention is to design an obstacle avoidance method for quadrotor drones based on a digital twin system. By establishing a digital twin system for the drone and designing a trajectory planning framework and a feedforward-feedback control strategy, the method improves the quality of drone trajectory planning and the accuracy of motion control.

[0008] The technical solution of this invention is: an obstacle avoidance method for a quadcopter UAV based on a digital twin system, comprising the following steps:

[0009] Step 1: Establish a digital twin system for unmanned aerial vehicles (UAVs);

[0010] Step 2: Design a feedforward-feedback control strategy based on model predictive control in the physical space;

[0011] Step 3: Design and deploy an offline trajectory planning framework in the virtual space.

[0012] Furthermore, the process of establishing the UAV digital twin system is as follows:

[0013] The digital twin system is designed from four dimensions: physical space, virtual space, application services, and data processing.

[0014] Digital twins perform high-fidelity mapping of physical space in a simulation environment, creating a virtual space that is highly similar to the physical space in terms of mathematical and physical properties, thus realizing a twin mapping from physical space to virtual space.

[0015] The application service module provides trajectory planning, trajectory visualization, and evaluation for physical drones at specified target points based on the virtual-real mapping relationship;

[0016] The data processing module stores and forwards the optimal feasible trajectory information and feedforward control quantities obtained from the virtual drone offline process to the physical drone.

[0017] Furthermore, the feedforward-feedback control strategy based on model predictive control in physical space is designed as follows:

[0018] First, based on Newton's second law and Euler's equations, a modeling analysis of the virtual drone is performed. The dynamic model of the virtual drone is as follows:

[0019]

[0020] Where, x v (t), y v (t), z v (t) represents the virtual drone's position in space; p v q v r vI represents the angular velocity of the virtual drone rotating around the x, y, and z axes of the ground coordinate system. vx I vy I vz The moment of inertia of the virtual drone rotating about its axis; m v U is the mass of the virtual drone; g is the acceleration due to gravity; U v1 U v2 U v3 U v4 For control input;

[0021] The quadcopter model consists of a position loop system and an attitude loop system. The attitude loop system is implemented using a cascaded PID controller, which receives the required thrust, roll, and pitch commands and calculates the control commands for the four motors. The desired roll and pitch angles of the attitude controller are expressed as follows:

[0022]

[0023] in, and θ dv (t) represents the desired roll and pitch angles, u vx (t) and u vy (t) represent the control inputs in the X and Y directions, respectively; taking the X direction as an example, the tracking error is expressed as follows:

[0024]

[0025] Where, x r (t) is the reference position, x v (t) represents the real-time location, e v1 (t) represents the position tracking error, e v2 (t) represents the velocity tracking error; the tracking error system is represented as follows:

[0026]

[0027] Additionally, let the tracking error vector be e vx (t)=[e v1 (t),e v2 (t)] T The state vector in the X direction is With a sampling period of h = 0.02s, the tracking error system of the virtual UAV after discretization in the X direction is as follows:

[0028]

[0029] Where e vx (tk ) and u vx (t k ) represent the virtual drone at sampling time t. k The tracking error and control input; similarly, the discrete tracking error system of the physical UAV in the X direction is as follows:

[0030]

[0031] Where e px (t k ) and u px (t k ) represent the physical drone at sampling time t k The tracking error and control input; after obtaining the tracking error system of the virtual UAV, a model predictive controller is designed for the system, and the terminal controller is designed as follows:

[0032] u vx (t k )=κ1e v1 (t k )+κ2e v2 (t k )

[0033] Where K = [κ1, κ2] are the feedback weight coefficients, and the circular terminal set is given:

[0034] Ω:={e vx (t k )|‖e vx (t k )‖ 2 ≤r 2}

[0035] Where r is the radius of the terminal set, and T = nh is the prediction time domain, where N represents the prediction step size; for the tracking error system of a virtual UAV, the cost function is given as:

[0036]

[0037] in Let the state cost function be... Let P, Q, and R be the terminal state function; P, Q, and R are three positive definite matrices; for τ∈[t] k ,t k +T], the optimization problem is designed as follows:

[0038]

[0039] stε vx (t k|t k )=ε vx (t k )

[0040]

[0041] e vx (t k +T|t k )∈Ω

[0042] Optimal cost function J * (t k )for:

[0043]

[0044] After solving the optimization problem, the first control variable is applied only in the first sampling period, and the process is repeated at the beginning of the next sampling period to achieve rolling optimization; the model predictive controller design of the physical UAV is consistent with that of the virtual UAV, making... Where u v (t),u p (t) represent the control inputs for the virtual drone and the physical drone, respectively; u v (t) is stored in the data storage unit, and the feedback control input u p (t) is obtained online by the model predictive controller of the physical UAV; the feedforward control input is represented as:

[0045] u f (t)=δ v u v (t)

[0046] Where δ v Let δ represent the feedforward gain. p For feedback gain, if there is δ v +δ p =1, then the feedforward-feedback control input is normalized; the control output of the virtual UAV MPC is used as the feedforward input of the physical UAV, and a feedforward-feedback control strategy based on digital twin is constructed to provide prior knowledge for MPC to improve the solution speed.

[0047] Furthermore, an offline trajectory planning framework is designed and deployed in the virtual space, as detailed below:

[0048] Repeatedly plan and select the optimal feasible trajectory in virtual space;

[0049] EGO-Planner is deployed on a virtual drone to generate multiple reference trajectories for the virtual drone. MPC is used to track these reference trajectories, and the tracking results are multiple feasible trajectories for the virtual drone during offline operation.

[0050] For the trajectory passing through the obstacle area, the evaluation function is selected as follows:

[0051] G(PT i )=αGoal(PT i )+βObs(PT i )+γVel(PT i )

[0052] Where α, β, and γ are weighting coefficients, and Goal(PTi), Obs(PTi), and Vel(PTi) are the target evaluation function, obstacle evaluation function, and velocity evaluation function of PTi, respectively. The evaluation function is used to select the faster collision-free velocity toward the endpoint at the current time t. The optimal feasible trajectory is selected as the reference trajectory for the physical UAV.

[0053] Beneficial effects

[0054] Compared with the prior art, the present invention has the following advantages:

[0055] 1. This invention, through the design of a digital twin system for quadcopter drones, can improve the obstacle avoidance performance of drones in terms of both trajectory planning and motion control, while reducing the onboard computing burden and sensor costs of drones.

[0056] 2. This invention transfers the computational burden to the virtual space through the high-precision mapping of the digital twin system, enabling perception and planning tasks to be performed in the virtual space. This reduces the interference of light on the planning results in the real scene and improves the stability of trajectory planning results.

[0057] 3. This invention replaces the initial controller by combining EGO-Planner and MPC to generate high-quality collision-free feasible trajectories in virtual space; and designs an offline trajectory planning framework in the digital twin system to obtain the optimal feasible trajectory, thereby achieving optimal obstacle avoidance performance.

[0058] 4. This invention, through the design of a feedforward-feedback control strategy, makes full use of the prior knowledge provided by the virtual space in the digital twin system, and uses the control input of the virtual UAV as the feedforward input of the physical UAV, thereby improving control accuracy and shortening the MPC solution time. Figure description:

[0059] Figure 1 is a schematic diagram of the obstacle avoidance method for a quadcopter UAV according to the present invention;

[0060] Figure 2 is a structural diagram of the UAV digital twin system of the present invention;

[0061] Figure 3 is a structural diagram of the trajectory planning framework of the present invention;

[0062] Figure 4 is a block diagram of the feedforward-feedback control strategy of the present invention;

[0063] Figure 5 shows the UAV coordinate system used in this invention;

[0064] Figure 6 shows the trajectory planning results of the present invention;

[0065] Figure 7 shows the feedforward-feedback control results of the present invention;

[0066] Figure 8 is a process diagram of an example of the UAV digital twin system of the present invention;

[0067] Figure 9 is a diagram of the feedforward-feedback control input of the present invention;

[0068] Figure 10 shows the trajectory tracking error results of the present invention. Detailed implementation method:

[0069] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0070] The purpose of this invention is to improve the obstacle avoidance performance of quadcopter drones by designing a digital twin system as shown in Figure 1 for obstacle avoidance scenarios. Based on this system, a trajectory planning framework and a feedforward-feedback control strategy are proposed, which reduces the hardware cost of autonomous flight of the drone while achieving better planning and control performance.

[0071] In Figure 2, the digital twin system is designed from four dimensions: physical space, virtual space, application services, and data processing. The digital twin performs a high-fidelity mapping of the physical space in a simulation environment, creating a virtual space that closely approximates the physical space in mathematical and physical properties, achieving a twin mapping from physical to virtual space. Leveraging its powerful computing capabilities, the virtual space can simulate and optimize the trajectory planning process, thereby transferring the computational burden from the physical space to the virtual space. In the virtual space, the trajectory planning algorithm is designed as a repeatable offline process to generate multiple collision-free feasible trajectories, and the optimal feasible trajectory is selected as the planning result based on trajectory evaluation metrics. The physical space, as the object of the digital twin, includes the UAV and obstacle environment. Environmental information is acquired through methods such as 3D reconstruction, and a high-precision mapping is constructed in the virtual space. A twin of the UAV in the physical space is constructed, and the corresponding mechanical structure is imported into the simulation environment, maintaining the same physical parameters to ensure the accuracy of the twin. The application service module is a C++ project built in the ROS system, providing application services to the physical UAV based on the virtual-physical mapping relationship. The system performs trajectory planning for a specified target point and visualizes and evaluates multiple feasible trajectories. Combined with a data processing module, communication between the virtual and physical spaces is established via ROS topics. The optimal feasible trajectory information and feedforward control variables obtained from the virtual drone's offline process are stored and forwarded to the physical drone. Based on these four dimensions, a drone-based digital twin system is constructed.

[0072] Using the UAV coordinate system shown in Figure 3, the dynamic model of the virtual UAV is established and discretized as follows:

[0073] Based on Newton's second law and Euler's equations, the dynamic model of the virtual drone is as follows:

[0074]

[0075] Where, x v (t), y v (t), z v (t) represents the virtual drone's position in space; p v q v r v I represents the angular velocity of the virtual drone rotating around the x, y, and z axes of the ground coordinate system. vx I vy I vz The moment of inertia of the virtual drone rotating about its axis; m v U is the mass of the virtual drone; g is the acceleration due to gravity; U v1 U v2 U v3 Uv4 For control input;

[0076] The quadcopter model consists of a position loop system and an attitude loop system. The attitude loop system is implemented using a cascaded PID controller, which receives the required thrust, roll, and pitch commands and calculates the control commands for the four motors. The desired roll and pitch angles of the attitude controller are expressed as follows:

[0077]

[0078] in, and θ dv (t) represents the desired roll and pitch angles, u vx (t) and u vy (t) represent the control inputs in the X and Y directions, respectively; taking the X direction as an example, the tracking error is expressed as follows:

[0079]

[0080] Where, x r (t) is the reference position, x v (t) represents the real-time location, e v1 (t) represents the position tracking error, e v2 (t) represents the velocity tracking error; the tracking error system is represented as follows:

[0081]

[0082] Additionally, let the tracking error vector be e vx (t)=[e v1 (t),e v2 (t)] T The state vector in the X direction is With a sampling period of h = 0.02s, the tracking error system of the virtual UAV after discretization in the X direction is as follows:

[0083]

[0084] Where e vx (t k ) and u vx (t k ) represent the virtual drone at sampling time t. k The tracking error and control input; similarly, the discrete tracking error system of the physical UAV in the X direction is as follows:

[0085]

[0086] Where e px (t k ) and u px (t k ) represent the physical drone at sampling time t k Tracking error and control input;

[0087] After obtaining the tracking error system of the virtual drone, a model prediction controller was designed for the system, and the terminal controller was designed as follows:

[0088] u vx (t k )=κ1e v1 (t k )+κ2e v2 (t k )

[0089] Where K = [κ1, κ2] are the feedback weight coefficients, and the circular terminal set is given:

[0090] Ω:={e vx (t k )|‖e vx (t k )‖ 2 ≤r 2}

[0091] Where r is the radius of the terminal set, and T = nh is the prediction time domain, where N represents the prediction step size. For the tracking error system of a virtual UAV, the cost function is given as:

[0092]

[0093] in Let the state cost function be... Let P, Q, and R be the terminal state function; P, Q, and R are three positive definite matrices; for τ∈[t] k ,t k +T], the optimization problem is designed as follows:

[0094]

[0095] stε vx (t k |t k )=ε vx (t k )

[0096]

[0097] e vx (tk +T|t k )∈Ω

[0098] Optimal cost function J * (t k )for:

[0099]

[0100] After solving the optimization problem, the first control variable is applied only in the first sampling period, and the process is repeated at the beginning of the next sampling period to achieve rolling optimization. Based on the proposed digital twin system, the same controller is designed for both the physical and virtual UAVs, and a feedforward-feedback control strategy as shown in Figure 4 is designed to improve the control effect. Where u v (t),u p (t) represent the control inputs for the virtual drone and the physical drone, respectively; u v (t) is stored in the data storage unit, and the feedback control input u p (t) is obtained online by the model predictive controller of the physical UAV; the feedforward control input is represented as:

[0101] u f (t)=δ v u v (t)

[0102] Where δ v Let δ represent the feedforward gain. p For feedback gain, if there is δ v +δ p If the input is equal to 1, the feedforward-feedback control input is normalized. That is, during offline operation, the optimal feasible trajectory is selected as the reference trajectory for the physical UAV, and the control input of the virtual UAV is used as the feedforward input for the physical UAV. This feedforward-feedback control strategy is obtained from the high-fidelity mapping relationship of the digital twin system, reducing the complexity and difficulty of mathematical modeling and improving control performance.

[0103] As shown in Figure 5, a trajectory planning framework based on a digital system was designed and applied to a physical UAV. Multiple feasible trajectories were obtained in virtual space and the optimal feasible trajectory was selected. EGO-Planner was deployed on the virtual UAV to generate multiple reference trajectories for the virtual UAV. MPC was used to track these reference trajectories. The tracking result g is the multiple feasible trajectories of the virtual UAV during the offline process. Among these feasible trajectories, there are trajectories that pass through the obstacle area and those that bypass the obstacle area.

[0104] For the trajectory passing through the obstacle area, the evaluation function is selected as follows:

[0105] G(PTi )=αGoal(PT i )+βObs(PT i )+γVel(PT i )

[0106] Where α, β, and γ are weighting coefficients, and Goal(PTi), Obs(PTi), and Vel(PTi) are the target evaluation function, obstacle evaluation function, and velocity evaluation function of PTi, respectively. The evaluation function is used to select the faster collision-free velocity toward the endpoint at the current time t. The optimal feasible trajectory is selected as the reference trajectory for the physical UAV.

[0107] Example

[0108] This invention is based on a quadcopter drone and other equipment as a collision avoidance experimental platform. The platform mainly consists of a quadcopter drone, a remote controller, a PC host computer, a wireless router, obstacles, and a pair of wireless data transmission devices.

[0109] The digital twin system is deployed on a PC host computer, establishing communication between the twin system and the physical drone via a wireless LAN. This allows for high-precision mapping of obstacle positions, dimensions, and other information in virtual space. The digital twin system obtains the optimal feasible trajectory as a reference trajectory for the physical drone and acquires the corresponding feedforward input for obstacle avoidance flight. Within the digital twin system, based on the start and end points and obstacle information, the virtual drone performs multiple repeated trajectory planning steps and utilizes MPC tracking and control to obtain multiple feasible trajectories. The application service module evaluates these trajectories, resulting in the trajectory planning results shown in Figure 6.

[0110] In an obstacle-prone environment, with a flight altitude set at 1.5 meters, a collision avoidance experiment was conducted using a physical UAV to verify the effectiveness of the proposed feedforward-feedback control strategy. RTK positioning was used to obtain precise positioning information. A coordinate system was established with the initial position of the quadcopter UAV as the origin, east as the X-axis, and north as the Y-axis. After the quadcopter took off using a remote controller, the flight mode was switched from stationary mode to offboard mode. The quadcopter UAV flew from its initial position to the target position (9m, 0m) and received optimal feasible trajectory information and feedforward control input from the digital twin system. The final collision avoidance trajectory tracking diagram is shown in Figure 7, and the operation process of the digital twin system in the example is shown in Figure 8. Figure 9 shows the position error of the physical UAV during trajectory tracking. It can be seen that the feedforward-feedback control strategy proposed in this invention is effective and can maintain high-precision trajectory tracking control. The feedforward-feedback control input is shown in Figure 10. It can be seen that the control input remains within the constraints. The digital twin system designed in this invention can obtain high-quality trajectory planning results, and the proposed feedforward-feedback control strategy can improve control accuracy and transfer computational burden.

Claims

1. An obstacle avoidance method for a quadcopter UAV based on a digital twin system, comprising the following steps: Step 1: Establish a digital twin system for the UAV; Step 2: Design a feedforward-feedback control strategy based on model predictive control in the physical space; Step 3: Design and deploy an offline trajectory planning framework in the virtual space; The design of the feedforward-feedback control strategy based on model predictive control in the physical space is as follows: First, model and analyze the virtual UAV according to Newton's second law and Euler's equations. The dynamic model of the virtual UAV is as follows: in, 、 、 The virtual location of the drone in space; 、 、 For virtual drones orbiting the coordinate axes of the ground coordinate system 、 、 The rotational angular velocity of rotation; 、 、 The moment of inertia of the virtual drone rotating about its axis; It's about the quality of the virtual drone; It is gravitational acceleration; 、 、 、 For control inputs; the quadcopter model is divided into a position loop system and an attitude loop system; the attitude loop system is implemented using a cascade PID controller, which receives the required thrust, roll, and pitch commands, and calculates the control commands for the four motors; the desired roll and pitch angles of the attitude controller are expressed as: in, and These are the expected roll angle and pitch angle. and They represent and Direction control input; with Taking direction as an example, the tracking error is expressed as follows: in, This is a reference position. For real-time location, For position tracking error, The tracking error is represented as follows: Additionally, let the tracking error vector be... The state vector in the X direction is ,by For sampling period; virtual drone in The tracking error system after direction discretization is as follows: in and These represent the virtual drone at the sampling time. The tracking error and control input; similarly, the discrete tracking error system of the physical UAV in the X direction is as follows: in and These represent the physical drone sampling time. The tracking error and control input; after obtaining the tracking error system of the virtual UAV, a model predictive controller is designed for the system, and the terminal controller is designed as follows: in To provide feedback weighting coefficients, a circular terminal set is given: in Let be the radius of the terminal set, and let For predicting the time domain, where This represents the prediction step size; for the tracking error system of a virtual UAV, the cost function is given as: in Let the state cost function be... For terminal state functions; There are three positive definite matrices; for The optimization problem is designed as follows: Optimal cost function for: After solving the optimization problem, the first control variable is applied only in the first sampling period, and the process is repeated at the beginning of the next sampling period to achieve rolling optimization; the model predictive controller design of the physical UAV is consistent with that of the virtual UAV, making... ， ,in These are the control inputs for virtual drones and physical drones, respectively. Stored in the data storage unit, feedback control input The model predictive controller of the physical UAV is acquired online; the feedforward control input is represented as: in To represent the feedforward gain, let For feedback gain, if there is Then the feedforward-feedback control input is normalized; the control output of the virtual UAV MPC is used as the feedforward input of the physical UAV, and a feedforward-feedback control strategy based on digital twin is constructed to provide prior knowledge for MPC to improve the solution speed.

2. The obstacle avoidance method for a quadcopter UAV based on a digital twin system according to claim 1, characterized in that, The process of establishing a digital twin system for unmanned aerial vehicles (UAVs) is as follows: The digital twin system is designed from four dimensions: physical space, virtual space, application services, and data processing; the digital twin performs a high-fidelity mapping of the physical space in a simulation environment, creating a virtual space that is highly similar to the physical space in terms of mathematical and physical characteristics, thus realizing a twin mapping from the physical space to the virtual space. The application service module provides trajectory planning, trajectory visualization, and evaluation for physical drones at specified target points based on the virtual-real mapping relationship; The data processing module stores and forwards the optimal feasible trajectory information and feedforward control quantities obtained from the virtual drone offline process to the physical drone.

3. The obstacle avoidance method for a quadcopter UAV based on a digital twin system according to claim 1, characterized in that, An offline trajectory planning framework is designed and deployed in virtual space, specifically as follows: The optimal feasible trajectory is repeatedly planned and selected in virtual space; EGO-Planner is deployed on the virtual drone to generate multiple reference trajectories for the virtual drone, and MPC is used to track these reference trajectories. The tracking results are the multiple feasible trajectories of the virtual drone during the offline process; for trajectories that pass through obstacle areas, the evaluation function is selected as follows: Where α, β, and γ are weighting coefficients, and Goal(PTi), Obs(PTi), and Vel(PTi) are the target evaluation function, obstacle evaluation function, and velocity evaluation function of PTi, respectively. The evaluation function is used to select the faster collision-free velocity toward the endpoint at the current time t. The optimal feasible trajectory is selected as the reference trajectory for the physical UAV.

Images