Virtual unmanned aerial vehicle and real-mounted unmanned aerial vehicle fusion simulation method and system

Abstract

The invention discloses a virtual unmanned aerial vehicle and real-mounted unmanned aerial vehicle fusion simulation method and system, and relates to the technical field of unmanned aerial vehicle modeling and simulation, and the method comprises the steps: obtaining the kinetic parameter information of a real-mounted unmanned aerial vehicle, and the kinetic parameter information of the real-mounted unmanned aerial vehicle detected by a sensor module disposed on the real-mounted unmanned aerial vehicle; sending the dynamics parameters and kinematics parameters of the real-mounted unmanned aerial vehicle to a simulation platform through a communication module; establishing a virtual unmanned aerial vehicle model according to the dynamic parameters of the real-mounted unmanned aerial vehicle; performing sensor configuration on the virtual unmanned aerial vehicle model according to kinematics parameters of the real-mounted unmanned aerial vehicle; and selecting a simulation environment according to the application scene to realize virtual unmanned aerial vehicle fusion simulation. Through precise modeling, real sensor parameter configuration and multi-sensor fusion algorithm application, higher simulation precision can be provided, so that the flight behavior of the virtual unmanned aerial vehicle is closer to that of a real unmanned aerial vehicle, and the reliability of a simulation result is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle modeling and simulation, and in particular to a simulation method and system for integrating a virtual unmanned aerial vehicle with a real unmanned aerial vehicle. Background Art

[0002] With the rapid development of drone technology, drone simulation has become an indispensable part of the process of designing and verifying drone-related algorithms. At present, drone simulation has achieved remarkable results in many fields such as drone dynamics, obstacle avoidance technology and cluster mission planning, and has played an irreplaceable role.

[0003] Traditional drone operations rely on actual flight tests, which is not only costly but also time-consuming. Especially in complex environments, flight tests face safety hazards and high risks. Especially in complex environments, there are safety hazards; traditional drone operations rely on actual flight tests. Existing drone simulation systems are usually unable to accurately simulate the complex environment and dynamic changes of the real world, resulting in a large difference between simulation results and actual performance. In addition, in the prior art, there are often significant differences between the simulation of virtual drones and the operation of real drones, making it difficult to achieve effective coordination. In drone operations, data from different sensors need to be effectively fused to provide accurate environmental perception and flight status. To this end, advanced data processing and fusion algorithms must be used to improve the efficiency and accuracy of data processing and ensure real-time performance.

[0004] In summary, drone operation and drone simulation based on traditional methods still face some challenges. Existing simulation systems have limitations in simulating the complex environment and dynamic changes of the real world, resulting in a large gap between simulation results and actual performance, affecting the effectiveness and reliability of the algorithm. This is specifically reflected in the following aspects:

[0005] 1. There are challenges in building accurate virtual models of real drones, especially in modeling complex dynamic behaviors and sensor characteristics, which often requires developers to invest a lot of time and resources;

[0006] 2. In the construction of high-precision virtual drone models in traditional simulation systems, the models often find it difficult to accurately capture complex dynamic behaviors, and the parameter collection process of actual drones is cumbersome and easily affected by the environment, resulting in inaccurate data.

[0007] 3. In terms of fast sensor model configuration, the configuration process is complex and time-consuming, which affects the simulation efficiency. At the same time, in a dynamic environment, it is difficult to ensure real-time calibration and configuration, resulting in data consistency issues.

[0008] 4. The data processing and fusion algorithms are highly complex and require large amounts of computation, making it difficult to meet real-time processing requirements. They also lack adaptability when processing data from different sensors, which affects the fusion effect.

[0009] Therefore, there is an urgent need for a simulation method that integrates virtual drones with real drones. A virtual drone model is constructed based on the real drones to achieve the integration and unified deployment of real resources and virtual resources in the simulation environment. Summary of the invention

[0010] In view of the technical problems existing in the prior art, the present invention provides a method and system for simulating the fusion of virtual drones and real drones. The method and system can quickly and accurately establish a model of a virtual drone based on the actual configuration of the real drone. The simulation environment can be built according to the actual environment, and the fusion and unified allocation of real resources and virtual resources in the simulation environment can be achieved with low cost, high real-time performance and strong reliability.

[0011] The technical solution adopted by the present invention to solve the technical problem is:

[0012] In a first aspect, the present invention provides a method for fusion simulation of a virtual drone and a real drone, comprising:

[0013] Acquire dynamic parameter information of the actual UAV and kinematic parameter information of the actual UAV detected by a sensor module installed on the actual UAV;

[0014] The dynamic parameters and kinematic parameters of the actual UAV are sent to the simulation platform through the communication module;

[0015] Establish a virtual UAV model based on the dynamic parameters of the actual UAV;

[0016] Configure sensors of the virtual drone model according to the kinematic parameters of the actual drone;

[0017] Select the simulation environment according to the application scenario to realize virtual UAV fusion simulation.

[0018] As a further technical solution of the present invention, the actual UAV dynamic parameters include: actual UAV mass, propeller diameter, temperature, geometric torque, maximum flight altitude, number of UAV rotors, UAV battery capacity, UAV battery minimum capacity, battery voltage, battery resistance and UAV cross-sectional area parameters.

[0019] As a further technical solution of the present invention, the sensor module includes: a laser radar, a magnetometer, a barometer, an inertial measurement unit (IMU), a global positioning system (GPS) and a distance sensor.

[0020] As a further technical solution of the present invention, the dynamic parameters and kinematic parameters of the installed UAV are sent to the simulation platform through the communication module; specifically, the communication module realizes the communication between the installed UAV and the simulation platform through the UDP / MAVLink protocol.

[0021] As a further technical solution of the present invention, the virtual drone model is established according to the dynamic parameters of the actual drone; specifically, it includes: establishing a multi-rotor kinematic position dynamic model, an attitude dynamic model and a control efficiency model according to the dynamic parameters of the actual drone;

[0022] The multi-rotor kinematic position dynamics model is:

[0023]

[0024] in, is the acceleration component of the drone in the x, y, and z directions of the ground coordinate system, and g is the acceleration due to gravity, usually 9.81 m / s 2 , f b is the lift (or thrust) generated in the body coordinate system, m is the mass of the UAV, θ, φ, ψ are the attitude angles of the UAV, representing the pitch angle, roll angle, and yaw angle, respectively;

[0025] The posture dynamics model is:

[0026]

[0027] Among them, [τ x τ y τ z ] is the total torque of the drone in the x, y, and z directions; [I xx I yy I zz ] are the diagonal elements of the UAV’s moment of inertia matrix, representing the moments of inertia around the x, y, and z axes respectively; is the angular acceleration component of the UAV in the body coordinate system;

[0028] The control efficiency model is:

[0029] The lift force f of the machine system is obtained according to the angular velocity of the rotor b and the moment τ acting on the body; lift f b The torque τ has the following relationship with the angular velocity of the rotor:

[0030]

[0031] Among them, f b is the total thrust generated in the body coordinate system, [τ x τy τ z ] is the total torque of the drone in the x, y, and z directions, c T is the thrust coefficient, which represents the relationship between the thrust generated by each rotor and the input power; dc T is the change of thrust coefficient; c M is the torque coefficient, which expresses the relationship between the torque generated by the rotor and the input power; is the square of the angular velocity of the four rotors.

[0032] As a further technical solution of the present invention, the sensor configuration of the virtual drone model is performed according to the kinematic parameters of the actual drone; specifically, it includes:

[0033] Generate data output of virtual sensors based on actual sensor parameters of installed drones;

[0034] The performance of actual sensors in different flight states and environmental conditions is simulated in the simulation platform to configure the sensors of the virtual UAV;

[0035] The sensor configuration is adjusted according to the characteristics of the virtual environment by introducing noise and errors in the data generation process.

[0036] As a further technical solution of the present invention, the data output of the virtual sensor is generated according to the actual sensor parameters of the installed UAV; specifically, it includes:

[0037] By setting the flight path of each drone during simulation and setting the control algorithm used during flight, that is, each virtual drone has a path point and controller that can set each drone individually, independent control of each drone is achieved;

[0038] The control algorithm in the default mode is to make the drone pass through each path point in sequence at a fixed speed of 1m / s.

[0039] As a further technical solution of the present invention, the performance of actual sensors under different flight states and environmental conditions is simulated in the simulation platform to configure the sensors of the virtual UAV; specifically, the multi-sensor fusion is performed by adaptively and dynamically adjusting the sensor weights based on the extended Kalman filter algorithm, wherein the adaptively and dynamically adjusted sensor weights are based on the performance indicators of the actual sensors, including accuracy, response time, and stability of historical data, and dynamic score calculation is performed. The dynamic score calculation formula is as follows:

[0040]

[0041] Among them, Score i is the dynamic score of sensor i, which is used to dynamically adjust the weight of the sensor. The higher the score, the greater the weight.i The accuracy of sensor i, which indicates the correctness of the sensor in measurement or detection, is usually expressed as a percentage (such as 0.95 means 95% accuracy); MaxAccuracy refers to the highest accuracy among all sensors. ResponseTime i Refers to the response time of sensor i, which means the time required for the sensor to receive the signal and output the result. MinResponseTime refers to the lowest response time among all sensors. Stability i Refers to the stability of sensor i, expressed by standard deviation; α, β, γ refer to weight coefficients, which respectively control the influence of accuracy, response time and stability on dynamic score, and the value is between 0 and 1.

[0042] As a further technical solution of the present invention, multi-sensor fusion is performed by adaptively and dynamically adjusting sensor weights based on an extended Kalman filter algorithm; and further comprising: updating the sensor weights, and calculating the entropy values ​​of each sensor in the sensor configuration module of the virtual drone as follows:

[0043]

[0044] Among them, H i refers to the entropy value of sensor i, which indicates the uncertainty of the sensor output information; m refers to the total number of different states or categories of sensor i output, which is used to determine the number of states that need to be considered in the entropy calculation; P k Refers to the probability of sensor i outputting state k, indicating that the frequency of state k in all outputs is between 0 and 1; log(Pk) refers to the logarithmic value of the probability of state k, reflecting the information contribution of state k;

[0045] The sensor weights are updated using the Bayesian method. The specific formula is as follows:

[0046] P(Weight i |Data)∝P(Data|Weight i )·P(Weight i );

[0047] Among them, P(Weight i |Data) refers to the posterior probability of the weight of sensor i given the data; P(Data|Weight i ) represents the probability of observing specific data when the weight of sensor i is assumed to be a certain value; P(Weight i ) refers to the prior probability of the weight of sensor i;

[0048] Based on the dynamic score, the final weight of the sensor is determined. The specific formula is as follows:

[0049]

[0050] Among them, Weighti refers to the final weight of sensor i; Scorei refers to the dynamic score of sensor i, reflecting the comprehensive performance of the sensor. The higher the score, the better the performance of the sensor, and the weight should increase accordingly; Entropyi refers to the entropy value of sensor i. The higher the entropy value, the more uncertain the information output by the sensor and the lower the reliability.

[0051] In a second aspect, the present invention further provides a virtual drone and real drone fusion simulation system, comprising:

[0052] An actual UAV parameter acquisition unit is used to acquire dynamic parameter information of the actual UAV and kinematic parameter information of the actual UAV detected by a sensor module disposed on the actual UAV;

[0053] A communication unit, used to send the actual UAV dynamic parameters and the actual UAV kinematic parameters to the simulation platform;

[0054] A virtual drone modeling unit is used to establish a virtual drone model based on the dynamic parameters of the actual drone;

[0055] A virtual drone sensor configuration unit configures sensors for the virtual drone model according to kinematic parameters of the actual drone;

[0056] The virtual UAV simulation unit selects the simulation environment according to the application scenario to realize the virtual UAV fusion simulation.

[0057] Compared with the prior art, the advantages of the present invention are:

[0058] The present invention has significant advantages in terms of integration, accuracy, flexibility and user-friendliness of the UAV simulation system, and provides more powerful support for the development and application of UAV algorithms. Among them, the present invention has multiple modules including multi-UAV modeling, UAV actual sensor configuration, flight control algorithm selection, simulation scene selection, communication configuration and flight log, which simplifies the development and testing process of UAV algorithms and improves the overall efficiency of the system; at the same time, through accurate modeling and real sensor parameter configuration and multi-sensor fusion algorithm application, the present invention can provide higher simulation accuracy, making the flight behavior of the virtual UAV closer to the actual UAV, and enhancing the reliability of the simulation results; furthermore, the present invention supports the selection of multiple control algorithms and simulation scenes, and users can flexibly configure them according to specific needs, adapt to different application scenarios and task requirements, and have higher flexibility and scalability; in addition, the design of the communication module and the flight log module enables the system to process and record flight data in real time, detect and respond to abnormal situations in a timely manner, and improve flight safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 A virtual UAV and real UAV fusion simulation method and system flow chart proposed by the present invention;

[0060] Figure 2 A structural diagram of a specific embodiment of the present invention;

[0061] Figure 3 A block diagram for setting parameters for the general multi-rotor drone modeling module proposed in the present invention;

[0062] Figure 4 A block diagram for setting parameters for the virtual drone sensor configuration module proposed by the present invention;

[0063] Figure 5 A flowchart of the multi-sensor fusion steps proposed in the present invention;

[0064] Figure 6 This is a structural diagram of a simulation system integrating a virtual UAV and a real UAV proposed in the present invention. DETAILED DESCRIPTION

[0065] The specific implementation of the present invention is described below in conjunction with the accompanying drawings and embodiments:

[0066] It should be noted that the structures, colors, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification so that people familiar with this technology can understand and read them, and are not used to limit the conditions under which the present invention can be implemented. Any structural modification, change in proportional relationship or adjustment of size should still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0067] At the same time, the terms such as "upper", "lower", "left", "right", "middle" and "one" cited in this specification are only for the convenience of description and are not used to limit the scope of implementation of the present invention. Changes or adjustments to their relative relationships should be regarded as the scope of implementation of the present invention without substantially changing the technical content.

[0068] Embodiment 1

[0069] like Figure 1 As shown, the present invention proposes a simulation method for integrating a virtual UAV with a real UAV, comprising:

[0070] Step 101, obtaining dynamic parameters of the actual drone and kinematic parameters of the actual drone detected by a sensor module installed on the actual drone;

[0071] Step 102, sending the actual UAV dynamic parameters and the actual UAV kinematic parameters to the simulation platform through the communication module;

[0072] Step 103, establishing a virtual drone model according to the actual drone dynamics parameters;

[0073] Step 104, configuring sensors for the virtual drone model according to kinematic parameters of the actual drone;

[0074] Step 105, selecting a simulation environment according to the application scenario to implement virtual UAV fusion simulation.

[0075] In the embodiment of the present invention, the model of the virtual drone is quickly and accurately established according to the actual configuration of the actual drone, and the simulation environment can be built according to the actual environment to achieve low-cost / high real-time and high-reliability fusion simulation of the virtual drone and the actual drone.

[0076] See also Figure 2 and Figure 3 The actual UAV dynamic parameters are the dynamic parameters of the actual UAV itself, including: actual UAV mass, propeller diameter, temperature, geometric torque, maximum flight altitude, number of UAV rotors, UAV battery capacity, UAV battery minimum capacity, battery voltage, battery resistance and UAV cross-sectional area parameters. The dynamic characteristics and kinematic model of the real UAV are modeled by mathematical equations, the dynamic equations are converted into state space form, Euler angles are used to describe the attitude changes of the UAV, and the motion equations and control efficiency models are established in combination with the changes in position and speed, so as to accurately simulate its flight behavior and response characteristics in the simulation environment.

[0077] In an embodiment of the present invention, the dynamic parameters and kinematic parameters of the actual UAV are sent to the simulation platform through the communication module; specifically, the communication module realizes the communication between the actual UAV and the simulation platform through the UDP / MAVLink protocol.

[0078] This embodiment selects the virtual drone scene through the scene selection module on the simulation platform. By providing a variety of preset simulation scenes, such as urban environment, mountain terrain, rainy fog and sunlight weather conditions, users can choose according to actual needs. Furthermore, this embodiment also supports the creation of custom scenes, allowing users to design unique flight environments according to specific mission requirements. By reasonably selecting and configuring simulation scenes, the pertinence and effectiveness of simulation experiments can be effectively improved, providing real test conditions for drone performance evaluation and algorithm verification.

[0079] Among them, a virtual drone model is established according to the dynamic parameters of the actual drone; specifically, it includes: establishing a multi-rotor kinematic position dynamics model, an attitude dynamics model and a control efficiency model according to the dynamic parameters of the actual drone;

[0080] The kinematic position dynamics model of the multi-rotor is:

[0081]

[0082] in, is the acceleration component of the drone in the x, y, and z directions of the ground coordinate system, and g is the acceleration due to gravity, usually 9.81 m / s 2 , f b is the lift (or thrust) generated in the body coordinate system, m is the mass of the UAV, θ, φ, ψ are the attitude angles of the UAV, representing the pitch angle, roll angle, and yaw angle, respectively;

[0083] The posture dynamics model is:

[0084]

[0085] Among them, [τ x τ y τ z ] is the total torque of the drone in the x, y, and z directions; [I xx I yy I zz ] are the diagonal elements of the UAV’s moment of inertia matrix, representing the moments of inertia around the x, y, and z axes respectively; is the angular acceleration component of the UAV in the body coordinate system;

[0086] The control efficiency model is:

[0087] The lift force f of the machine system is obtained according to the angular velocity of the rotor b and the moment τ acting on the body; lift f b The torque τ has the following relationship with the angular velocity of the rotor:

[0088]

[0089] Among them, f b is the total thrust generated in the body coordinate system, [τ x τ y τ z ] is the total torque of the drone in the x, y, and z directions, c T is the thrust coefficient, which represents the relationship between the thrust generated by each rotor and the input power; dc T is the change of thrust coefficient; c Mis the torque coefficient, which expresses the relationship between the torque generated by the rotor and the input power; is the square of the angular velocity of the four rotors.

[0090] In this embodiment, the input comprehensively considers the mass of the multi-rotor drone, the temperature and altitude of the flight environment, the propeller diameter, the geometric torque, and the battery-related performance parameters. Through the calculation of a series of performance formulas, the maximum load, maximum flight speed and maximum flight distance of the drone are calculated in the forward flight mode; or the maximum endurance time of the drone and the propeller speed to maintain the hovering state are obtained in the hovering mode; so as to quickly build a general virtual drone model, so that the drone model in the simulation platform is consistent with the actual drone model in terms of parameters. The simulation platform can be used to set and add different parameters of multiple drones.

[0091] The present invention converts the kinematic parameters of the actual drone detected by the sensor module set on the actual drone into the sensor configuration parameters of the virtual drone, wherein the sensor module includes: laser radar, magnetometer, barometer, inertial measurement unit (IMU), global positioning system (GPS) and distance sensor. The corresponding virtual sensor can be selected based on the sensor configuration of the actual drone, and parameter mapping can be performed. For example, GPS is used in the actual drone, and GPS can be configured on the virtual drone to set parameters such as horizontal / vertical accuracy and update frequency.

[0092] In this embodiment, each sensor can be configured, including the type, enabled state, various parameters and their default values ​​of each sensor. Figure 4 shown.

[0093] In an embodiment of the present invention, sensors are configured for a virtual drone model according to kinematic parameters of an actual drone; specifically, the data output of a virtual sensor is generated according to actual sensor parameters of the actual drone; these data simulate the performance of actual sensors under different flight states and environmental conditions, and the sensor performance of the virtual drone is ensured to be consistent with that of the actual drone through sensor configuration, and noise and errors are introduced in the data generation process to make the output of the virtual sensor closer to the performance of the actual sensor, thereby ensuring that the virtual drone exhibits similar characteristics to the actual drone in the simulated environment, thereby improving the authenticity and reliability of the simulated drone and the simulated environment.

[0094] In the embodiment of the present invention, the data output of the virtual sensor is generated according to the actual sensor parameters of the installed UAV; specifically, it includes:

[0095] By setting the flight path of each drone during simulation and setting the control algorithm used during flight, that is, each virtual drone has a path point and controller that can set each drone individually, independent control of each drone is achieved;

[0096] The control algorithm in the default mode is to make the drone pass through each path point in sequence at a fixed speed of 1m / s. By providing Simulink and C++ programming interfaces, user-defined control algorithms can be accessed.

[0097] This embodiment uses the north-east coordinate system, and the positive direction of the z-axis points directly downward, so the z-axis coordinate data of the drone trajectory point are all negative. In addition, the coordinates of each path point of the drone represent the relative position of the path point and the initial point of the drone. For example, (0, 5, -6) means that the path point is 5 meters east of the initial position and 6 meters above it. The communication module is used to realize data transmission and information interaction between the actual drone and the ground control station, the virtual drone and related equipment.

[0098] The initial flight state of the virtual drone in the simulation environment is the real flight state of the drone obtained after the fusion of multiple sensors of the real drone, including position, speed and attitude. In the real drone, multiple sensors are used to obtain flight state information; the simulation platform acquires real sensor data in real time based on time synchronization; the initialization state vector includes position, speed and attitude; the extended Kalman filter method based on dynamic weight adjustment is applied to estimate the real state of the drone. For example, the accurate position of the drone is obtained by fusing the data of GPS and IMU. GPS provides global positioning, while IMU provides relative motion information; the acceleration data of IMU and the speed information of GPS are fused to obtain a more accurate speed estimate; the angular velocity of IMU and the data of magnetometer are combined to use four Euler angles to represent the attitude of the drone. After multi-sensor fusion, the real flight state of the drone is output, including position, speed and attitude; this information will be used as the initial flight state of the simulated drone.

[0099] In the embodiment of the present invention, the performance of actual sensors under different flight states and environmental conditions is simulated in the simulation platform to configure the sensors of the virtual UAV; specifically, the multi-sensor fusion is performed by adaptively and dynamically adjusting the sensor weights based on the extended Kalman filter algorithm, wherein the adaptively and dynamically adjusting the sensor weights is based on the performance indicators of the actual sensors, including accuracy, response time, and stability of historical data, and dynamic score calculation is performed. The dynamic score calculation formula is as follows:

[0100]

[0101] Among them, Score iis the dynamic score of sensor i, which is used to dynamically adjust the weight of the sensor. The higher the score, the greater the weight. i The accuracy of sensor i, which indicates the correctness of the sensor in measurement or detection, is usually expressed as a percentage (such as 0.95 means 95% accuracy); MaxAccuracy refers to the highest accuracy among all sensors. ResponseTime i Refers to the response time of sensor i, which means the time required for the sensor to receive the signal and output the result. MinResponseTime refers to the lowest response time among all sensors. Stability i Refers to the stability of sensor i, expressed by standard deviation; α, β, γ refer to weight coefficients, which respectively control the influence of accuracy, response time and stability on dynamic score, and the value is between 0 and 1.

[0102] Multi-sensor fusion is performed by adaptively and dynamically adjusting sensor weights based on the extended Kalman filter algorithm; it also includes: updating the sensor weights, and the entropy values ​​of each sensor in the sensor configuration module of the virtual drone are calculated as follows:

[0103]

[0104] Among them, H i refers to the entropy value of sensor i, which indicates the uncertainty of the sensor output information; m refers to the total number of different states or categories of sensor i output, which is used to determine the number of states that need to be considered in the entropy calculation; P k Refers to the probability of sensor i outputting state k, indicating that the frequency of state k in all outputs is between 0 and 1; log(Pk) refers to the logarithmic value of the probability of state k, reflecting the information contribution of state k;

[0105] The sensor weights are updated using the Bayesian method. The specific formula is as follows:

[0106] P(Weight i |Data)∝P(Data|Weight i )·P(Weight i );

[0107] Among them, P(Weight i |Data) refers to the posterior probability of the weight of sensor i given the data; P(Data|Weight i ) represents the probability of observing specific data when the weight of sensor i is assumed to be a certain value; P(Weight i ) refers to the prior probability of the weight of sensor i;

[0108] Based on the dynamic score, the final weight of the sensor is determined. The specific formula is as follows:

[0109]

[0110] Among them, Weighti refers to the final weight of sensor i; Scorei refers to the dynamic score of sensor i, reflecting the comprehensive performance of the sensor. The higher the score, the better the performance of the sensor, and the weight should increase accordingly; Entropyi refers to the entropy value of sensor i. The higher the entropy value, the more uncertain the information output by the sensor and the lower the reliability.

[0111] This embodiment uses the above method to monitor the performance changes of the sensor in real time during the simulation operation, and dynamically adjusts the weight according to the feedback.

[0112] In this embodiment, M5 obtains the real-time operation data of the UAV through the ground control station. The multi-sensor fusion steps are as follows: Figure 5 As shown, the steps are as follows: Initialize the state vector x = [position, velocity, posture (Euler angle )]

[0113] Initialize the covariance matrix P

[0114] Initialize sensor data: such as lidar data L, IMU data I, GPS data G, ranging sensor data D

[0115] Set the time step dt

[0116] Set the process noise covariance matrix Q and the measurement noise covariance matrix R

[0117] Set the initial sensor weights w_L, w_I, w_G, w_D

[0118] S1. Cycle start phase:

[0119] a. Update the state according to the control input (such as acceleration a and angular velocity ω):

[0120] x'=f(x, control input)

[0121] Among them, x' refers to the updated state vector, which contains the new position, velocity and attitude information; f(x, control input) is the state transfer function, which describes how to update the state based on the current state and control input.

[0122] Position update: x'{pos}=x{pos}+v_{x}*dt+0.5*a_{x}*dt^2

[0123] y'{pos}=x{pos}+v_{y}*dt+0.5*a_{y}*dt^2

[0124] z'{pos}=x{pos}+v_{z}*dt+0.5*a_{z}*dt^2

[0125] Among them, x′{pos}, y′{pos}, z′{pos} are the updated position coordinates, indicating the positions on the x, y, and z axes respectively; x{pos}, y{pos}, z{pos} refer to the position coordinates in the current state; v_{x}, v_{y}, v_{z} refer to the components of the current velocity of the drone on the x, y, and z axes; dt refers to the time step, indicating the time interval for state update; a_{x}, a_{y}, a_{z} refer to the acceleration components of the drone on the x, y, and z axes.

[0126] Speed ​​Update:

[0127] v'{x}=v{x}+a_{x}*dt

[0128] v'{y}=v{y}+a_{y}*dt

[0129] v'{z}=v{z}+a_{z}*dt

[0130] Among them, v'{x}, v'{y}, v'{z} refer to the updated velocity components, which represent the velocities on the x, y, and z axes respectively;

[0131] Posture update (Euler angles):

[0132]

[0133] θ'=θ+q*dt

[0134] ψ'=ψ+r*dt

[0135] in, θ′, ψ′ refer to the updated attitude (Euler angles), which represent the roll angle, pitch angle, and yaw angle, respectively; θ and ψ refer to the posture in the current state (Euler angles), and p, q, and r refer to the angular velocities around the x, y, and z axes, respectively.

[0136] b. Update the covariance matrix:

[0137] P'=F*P*F^T+Q

[0138] Among them, P' is the updated covariance matrix, F is the state transfer matrix, Q is the process noise covariance matrix, and F can be obtained by linearizing the state transfer equation.

[0139] S2. Update step phase:

[0140] a. Obtaining data from multiple sensors (using four sensors as an example):

[0141] L = Get LiDAR data()

[0142] I = Get IMU data ()

[0143] G = Get GPS data ()

[0144] D = Get distance sensor data ()

[0145] Among them, L is the lidar data, I is the IMU data, G is the GPS data, and D is the ranging sensor data.

[0146] b. Calculate the measurement value and measurement matrix of each sensor:

[0147] z_L=h_L(x)

[0148] z_I=h_I(x)

[0149] z_G=h_G(x)

[0150] z_D=h_D(x)

[0151] Among them, z_L, z_I, z_G, z_D represent the measurement values ​​of LiDAR, IMU, GPS and ranging sensor respectively; h_L(x), h_I(x), h_G(x) and h_D(x) refer to the measurement model functions, which describe how to calculate the corresponding measurement value from the state vector x.

[0152] c. Calculate the measurement error:

[0153] y_L=L-z_L

[0154] y_I=I-z_I

[0155] y_G=G-z_G

[0156] y_D=D-z_D

[0157] Among them, y_L, y_I, y_G and y_D represent the measurement errors of lidar, IMU, GPS and ranging sensor respectively.

[0158] d. Dynamically adjust sensor weights:

[0159] w_L=1 / (1+|y_L|)

[0160] w_I=1 / (1+|y_I|)

[0161] w_G=1 / (1+|y_G|)

[0162] w_D=1 / (1+|y_D|)

[0163] Among them, w_L, w_I, w_G and w_D represent the weights of LiDAR, IMU, GPS and ranging sensors respectively.

[0164] e. Calculate the measurement matrix H:

[0165] f. Calculate the Kalman gain:

[0166] K=P'*H^T*(H*P'*H^T+R)^(-1)

[0167] Where H is the measurement matrix and R is the measurement noise covariance matrix.

[0168] g. Update the state vector and covariance matrix:

[0169] x=x'+K*(w_L*y_L+w_I*y_I+w_G*y_G+w_D*y_D)

[0170] P=(IK*H)*P'

[0171] S3. Output the current state, output (x, P)

[0172] Among them, x is the current state vector of the drone; P is the current covariance matrix

[0173] Through the multi-sensor fusion step, more accurate drone status data, including position, speed and attitude, is continuously obtained and sent to the simulation platform through the UDP protocol.

[0174] Figure 5 The dynamic weight adjustment in is to dynamically adjust the weight according to the measurement error of each sensor to improve the accuracy of fusion, while the weighted fusion during state update uses the sensor's weight to weight the measurement error when updating the state vector to ensure a more reliable state estimation.

[0175] In this embodiment, the position, speed and posture of the virtual drone obtained after calculation by the control algorithm on the simulation platform can also be sent to the real drone through the ground control station via the UDP / MAVLink protocol, thereby realizing data transmission and information interaction between the real drone and the virtual drone.

[0176] This embodiment can record and store various data generated by the virtual drone during flight. The data in this embodiment includes flight status, sensor readings, communication information, and environmental parameters, and is stored in JSON format or CSV format for subsequent analysis and processing and other system integration.

[0177] The present invention has significant advantages in terms of integration, accuracy, flexibility and user-friendliness of the UAV simulation system, and provides more powerful support for the development and application of UAV algorithms. Among them, the present invention has multiple modules including multi-UAV modeling, UAV actual sensor configuration, flight control algorithm selection, simulation scene selection, communication configuration and flight log, which simplifies the development and testing process of UAV algorithms and improves the overall efficiency of the system; at the same time, through accurate modeling and real sensor parameter configuration and multi-sensor fusion algorithm application, the present invention can provide higher simulation accuracy, making the flight behavior of the virtual UAV closer to the actual UAV, and enhancing the reliability of the simulation results; furthermore, the present invention supports the selection of multiple control algorithms and simulation scenes, and users can flexibly configure them according to specific needs, adapt to different application scenarios and task requirements, and have higher flexibility and scalability; in addition, the design of the communication module and the flight log module enables the system to process and record flight data in real time, detect and respond to abnormal situations in a timely manner, and improve flight safety.

[0178] Embodiment 2

[0179] See also Figure 6 The present invention also provides a virtual UAV and real UAV fusion simulation system, including:

[0180] The actual drone parameter acquisition unit 201 acquires the actual drone dynamic parameter information and the actual drone kinematic parameter information detected by the sensor module installed on the actual drone;

[0181] The communication unit 202 is used to send the dynamic parameters and kinematic parameters of the actual UAV to the simulation platform;

[0182] A virtual drone modeling unit 203 is used to establish a virtual drone model according to the dynamic parameters of the actual drone;

[0183] A virtual drone sensor configuration unit 204 configures sensors for the virtual drone model according to kinematic parameters of the actual drone;

[0184] The virtual drone simulation unit 205 selects a simulation environment according to an application scenario to implement virtual drone fusion simulation.

[0185] The actual UAV parameter acquisition unit and the communication unit are specifically mounted on the actual UAV. The virtual UAV modeling unit, the virtual UAV sensor configuration unit and the virtual UAV simulation unit are all set on the simulation platform. The actual UAV parameters are received through the simulation platform to realize virtual UAV modeling / sensor configuration and simulation.

[0186] The various variations and specific examples of the methods in the aforementioned embodiments are also applicable to a virtual drone and real drone fusion simulation system in the present embodiment. Through the aforementioned detailed description of a virtual drone and real drone fusion simulation method, those skilled in the art can clearly know a virtual drone and real drone fusion simulation system in the present embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0187] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A method for fusion simulation of a virtual drone and a real drone, characterized in that: include: Acquire dynamic parameter information of the actual UAV and kinematic parameter information of the actual UAV detected by a sensor module installed on the actual UAV; The dynamic parameters and kinematic parameters of the actual UAV are sent to the simulation platform through the communication module; Establish a virtual UAV model based on the dynamic parameters of the actual UAV; Configure sensors of the virtual drone model according to the kinematic parameters of the actual drone; Select the simulation environment according to the application scenario to realize virtual UAV fusion simulation.

2. The method for fusion simulation of a virtual drone and a real drone according to claim 1, characterized in that: The actual UAV dynamic parameter information includes: actual UAV mass, propeller diameter, temperature, geometric torque, maximum flight altitude, number of UAV rotors, UAV battery capacity, UAV battery minimum capacity, battery voltage, battery resistance and UAV cross-sectional area parameters.

3. The method for fusion simulation of a virtual drone and a real drone according to claim 1, characterized in that: The sensor module includes: a laser radar, a magnetometer, a barometer, an inertial measurement unit, a global positioning system and a distance sensor.

4. The method for fusion simulation of a virtual drone and a real drone according to claim 1, characterized in that: The method sends the dynamic parameters and kinematic parameters of the installed UAV to the simulation platform through the communication module; specifically includes: the communication module realizes the communication between the installed UAV and the simulation platform through the UDP / MAVLink protocol.

5. The method for fusion simulation of a virtual drone and a real drone according to claim 1, characterized in that: The virtual drone model is established according to the actual drone dynamics parameters; specifically includes: establishing a multi-rotor kinematic position dynamics model, an attitude dynamics model and a control efficiency model according to the actual drone dynamics parameters; The multi-rotor kinematic position dynamics model is: in, is the acceleration component of the drone in the x, y, and z directions of the ground coordinate system, g is the gravitational acceleration, and f b is the lift generated in the body coordinate system, m is the mass of the UAV, θ, φ, ψ are the attitude angles of the UAV, representing the pitch angle, roll angle, and yaw angle, respectively; The posture dynamics model is: Among them, [τ x τ y τ z ] is the total torque of the drone in the x, y, and z directions; [I xx I yy I zz ] are the diagonal elements of the UAV's moment of inertia matrix, representing the moment of inertia around the x, y, and z axes respectively; is the angular acceleration component of the UAV in the body coordinate system; The control efficiency model is: The lift force f of the machine system is obtained according to the angular velocity of the rotor b and the moment τ acting on the body; lift f b The torque τ has the following relationship with the angular velocity of the rotor: Among them, f b is the total thrust generated in the body coordinate system, [τ x τ y τ z ] is the total torque of the drone in the x, y, and z directions, c T is the thrust coefficient, which represents the relationship between the thrust generated by each rotor and the input power; dc T is the change of thrust coefficient; c M is the torque coefficient, which expresses the relationship between the torque generated by the rotor and the input power; is the square of the angular velocity of the four rotors.

6. The method for fusion simulation of a virtual drone and a real drone according to claim 1, characterized in that: The sensor configuration of the virtual drone model according to the kinematic parameters of the actual drone specifically includes: Generate data output of virtual sensors based on actual sensor parameters of installed drones; The performance of actual sensors in different flight states and environmental conditions is simulated in the simulation platform to configure the sensors of the virtual UAV; The sensor configuration is adjusted according to the characteristics of the virtual environment by introducing noise and errors in the data generation process.

7. The method for fusion simulation of a virtual drone and a real drone according to claim 6, characterized in that: The method of generating data output of the virtual sensor according to the actual sensor parameters of the installed UAV specifically includes: By setting the flight path of each drone during simulation and setting the control algorithm used during flight, that is, each virtual drone has a path point and controller that can set each drone individually, independent control of each drone is achieved; The control algorithm in the default mode is to make the drone pass through each path point in sequence at a fixed speed of 1m / s.

8. The method for fusion simulation of a virtual drone and a real drone according to claim 6, characterized in that: The performance of actual sensors under different flight states and environmental conditions is simulated in the simulation platform to configure the sensors of the virtual UAV; specifically, the multi-sensor fusion is performed by adaptively and dynamically adjusting the sensor weights based on the extended Kalman filter algorithm, wherein the adaptively and dynamically adjusted sensor weights are based on the performance indicators of the actual sensors, including accuracy, response time, and stability of historical data, and dynamic score calculation is performed. The dynamic score calculation formula is as follows: Among them, Score i is the dynamic score of sensor i, which is used to dynamically adjust the weight of the sensor. The higher the score, the greater the weight. i is the accuracy of sensor i, indicating the correctness of the sensor in measurement or detection; MaxAccuracy refers to the highest accuracy among all sensors; ResponseTime i Refers to the response time of sensor i, which means the time required for the sensor to output the result from receiving the signal; MinResponseTime refers to the lowest response time among all sensors; Stability i It refers to the stability of sensor i, expressed by standard deviation; α, β, and γ are weight coefficients, which respectively control the influence of accuracy, response time, and stability on the dynamic score.

9. The method for fusion simulation of a virtual drone and a real drone according to claim 8, characterized in that: The method of adaptively and dynamically adjusting sensor weights based on the extended Kalman filter algorithm to perform multi-sensor fusion also includes: updating the sensor weights, and calculating the entropy value of each sensor in the sensor configuration module of the virtual drone as follows: Among them, H i refers to the entropy value of sensor i, which indicates the uncertainty of the sensor output information; m refers to the total number of different states or categories of sensor i output, which is used to determine the number of states that need to be considered in the entropy calculation; P k Refers to the probability of sensor i outputting state k, indicating that the frequency of state k in all outputs is between 0 and 1; log(Pk) refers to the logarithmic value of the probability of state k, reflecting the information contribution of state k; The sensor weights are updated using the Bayesian method. The specific formula is as follows: P(Weight i |Data)∝P(Data|Weight i )·P(Weight i ) Among them, P(Weight i |Data) refers to the posterior probability of the weight of sensor i given the data; P(Data|Weight i ) represents the probability of observing data when the weight of sensor i is assumed to be a certain value; P(Weight i ) refers to the prior probability of the weight of sensor i; Based on the dynamic score, the final weight of the sensor is determined. The specific formula is as follows: Among them, Weight i Refers to the final weight of sensor i; Score i Refers to the dynamic score of sensor i, reflecting the comprehensive performance of the sensor. The higher the score, the better the performance of the sensor, and the weight should be increased accordingly. i Refers to the entropy value of sensor i; the higher the entropy value, the more uncertain the information output by the sensor is and the lower the reliability is.

10. A virtual drone and real drone fusion simulation system, characterized in that: A method for fusion simulation of a virtual UAV and a real UAV as described in any one of claims 1 to 9 is adopted, comprising: An actual UAV parameter acquisition unit is used to acquire dynamic parameter information of the actual UAV and kinematic parameter information of the actual UAV detected by a sensor module disposed on the actual UAV; A communication unit, used to send the actual UAV dynamic parameters and the actual UAV kinematic parameters to the simulation platform; A virtual drone modeling unit is used to establish a virtual drone model based on the dynamic parameters of the actual drone; A virtual drone sensor configuration unit configures sensors for the virtual drone model according to kinematic parameters of the actual drone; The virtual UAV simulation unit selects the simulation environment according to the application scenario to realize the virtual UAV fusion simulation.

Images