Large model construction method for safe flight of low-altitude unmanned aerial vehicle

Abstract

The invention relates to the field of intelligent control of low-altitude unmanned aerial vehicles, and discloses a large model construction method serving safe flight of a low-altitude unmanned aerial vehicle, which utilizes a neural network large model to learn flight attitude control and flight route optimization methods under various complex flight environment conditions. The system is directly deployed and operates on the unmanned aerial vehicle, assists the control of the unmanned aerial vehicle in an airborne real-time operation mode, and achieves the autonomous and safe flight of the unmanned aerial vehicle under a low-altitude complex environment condition. In a traditional mode, the unmanned aerial vehicle is manually controlled through a remote control link, the occupied human resources are many, and the execution efficiency is low. According to the method for constructing the large model serving the safe flight of the low-altitude unmanned aerial vehicle, the large model for the safe flight of the low-altitude unmanned aerial vehicle is equivalent to a brain for controlling the unmanned aerial vehicle, the flight strategy can be adjusted in real time according to the flight environment and meteorological conditions, and the safe flight of the unmanned aerial vehicle is autonomously controlled.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control of low-altitude unmanned aerial vehicles, and specifically to a large model construction method serving the safe flight of low-altitude unmanned aerial vehicles. A large neural network model is used to learn flight attitude control and flight route optimization methods under various complex flight environment conditions. After the model is compressed to reduce the model scale, it is directly deployed and run on the unmanned aerial vehicle, and the unmanned aerial vehicle control is assisted in an airborne real-time operation mode, so as to realize autonomous and safe flight of the unmanned aerial vehicle under low-altitude complex environmental conditions. Background Art

[0002] Low-altitude economy is an emerging industry with broad market prospects. Drone transportation is an important means to help the development of low-altitude economy, and low-altitude drone safe flight technology escorts the development of low-altitude economy. Unlike medium and high-altitude airspace flights, low-altitude airspace flights face many special flight safety issues due to flying closer to the ground. Mountains, communication towers, high-voltage line towers, and buildings will block the flight route; strong winds, low clouds, fog and other low-altitude meteorological phenomena can easily cause flight control difficulties. In traditional methods, ground operators need to adjust the attitude and route of drones based on experience. Due to the influence of control experience and remote control communication link delay, sometimes the drone control action may be invalid, affecting flight safety, and sometimes forced to cancel the flight mission, thus affecting the development of low-altitude economy. Drones play an important role in the field of low-altitude economy, and the safe flight of drones in low-altitude environments is the basic guarantee for the stable development of low-altitude economy. In traditional methods, drones are controlled manually through remote control links, which takes up a lot of human resources and has low execution efficiency. Therefore, it is necessary to study the problem of autonomous and safe flight of drones and use technical means to achieve autonomous and safe low-altitude flight of drones. Summary of the invention

[0003] In order to effectively solve the problem of safe low-altitude flight of UAVs, the present invention proposes a large model construction method serving the safe low-altitude flight of UAVs. With the help of the large model, auxiliary flight decisions under various flight conditions are realized to ensure the autonomous and safe low-altitude flight of UAVs.

[0004] The technical solution adopted by the present invention to solve its technical problem is:

[0005] A method for constructing a large model serving the safe flight of a low-altitude unmanned aerial vehicle comprises the following steps:

[0006] Step 1: Use a six-degree-of-freedom aerodynamic model to link the mass, inertia, force, and generated acceleration and angular velocity of the drone, describe the translation and rotation of the drone, and establish a low-altitude drone simulation model; at the same time, construct natural and man-made objects, match and arrange them to establish a typical surface environment scene model; and construct a natural weather phenomenon and wind resistance model to simulate the common wind resistance and natural weather phenomena in the low-altitude flight process, and establish a typical low-altitude meteorological condition environment simulation model;

[0007] Step 2, using a typical surface environment scene model and a typical low-altitude meteorological condition environment simulation model to randomly generate a low-altitude flight environment;

[0008] Step 3: randomly set the flight starting point and end point in various low-altitude flight environments, generate a flight route using a route planning algorithm, and then select a certain type of drone to perform a simulated flight mission. During the flight, special events are randomly set to build a realistic low-altitude flight environment. A reinforcement learning algorithm is used to drive the low-altitude drone simulation model to face various environments and overcome various emergency events to complete the flight mission from the flight starting point to the end point. At the same time, the low-altitude flight surface environment data, meteorological conditions, drone control parameters, and drone attitude data of each flight are recorded;

[0009] Step 4: Collect low-altitude flight surface environment data, meteorological conditions, UAV control parameters, and UAV attitude data recorded in various typical low-altitude flight simulation application scenarios, perform data cleaning, data enhancement, and feature extraction, and align them according to time;

[0010] Step 5: Use a multimodal big data training mechanism to generate a low-altitude UAV flight safety model through training based on the time-aligned data;

[0011] Step 6: compress the large low-altitude UAV flight safety model generated by training, deploy the compressed model on the UAV, drive the UAV to fly in a real low-altitude airspace environment, collect the attitude data of the UAV during the actual flight, and use the attitude data for model evaluation and retraining.

[0012] Among them, the natural features constructed in step 1 include mountains, canyons, hills, plains, waters and trees, the man-made features include buildings, high-voltage towers, communication towers and bridges, and the surface environment scenes include high-rise buildings, rural fields, mountains and hills, alpine river valleys, forests and grasslands, and construction site cargo yards.

[0013] The advantages of the present invention compared with the prior art are:

[0014] In the traditional way, the drone is controlled manually through a remote control link, which takes up a lot of human resources and has low execution efficiency. The present invention proposes a method for constructing a large model for the safe flight of low-altitude drones. The large model for the safe flight of low-altitude drones is equivalent to a "brain" for controlling the drone, which can adjust the flight strategy in real time according to the flight environment and meteorological conditions, and autonomously control the safe flight of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a principle flow chart of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below in conjunction with the accompanying drawings.

[0017] like Figure 1 As shown, the present invention provides a large model construction method for serving the safe flight of low-altitude unmanned aerial vehicles, comprising the following steps:

[0018] Step 1: construct a low-altitude UAV simulation model and a typical low-altitude meteorological condition environment simulation model;

[0019] (1) A six-degree-of-freedom aerodynamic model is used to link the mass, inertia, force, acceleration and angular velocity of the UAV, describe the translation and rotation of the UAV, and establish a low-altitude UAV simulation model. The six-degree-of-freedom aerodynamic model is described by the Newton-Euler equations, including the translational motion equation and the rotational motion equation.

[0020] Translational motion equation:

[0021]

[0022] In the formula, m is the mass of the drone, is the horizontal position of the drone, is the horizontal component of the total external force acting on the drone, It is the horizontal component of the thrust generated by the UAV propulsion system;

[0023]

[0024] In the formula, is the vertical position of the drone, is the vertical component of the total external force acting on the drone, It is the component of thrust generated by the UAV propulsion system in the vertical direction;

[0025]

[0026] In the formula, is the lateral position of the drone, is the lateral component of the total external force acting on the drone, It is the lateral component of the thrust generated by the UAV propulsion system.

[0027] Rotational motion equations:

[0028]

[0029] In the formula, is the moment of inertia about the horizontal axis, is the moment of inertia about the longitudinal axis, is the roll angle, is the pitch angle, is the component of the total external moment acting on the aircraft in the horizontal axis;

[0030]

[0031] In the formula, is the moment of inertia about the vertical axis, is the pitch angle, is the yaw angle, is the component of the total external moment acting on the aircraft in the longitudinal axis;

[0032]

[0033] In the formula, It is the component of the total external torque acting on the aircraft about the vertical axis.

[0034] The forces and moments in the above equations are obtained by aerodynamic calculations, including lift, drag, side force and corresponding moments. In practical applications, these equations are solved using the Euler method.

[0035] (2) Construct natural and man-made objects, match and arrange them to establish a typical surface environment scene model; and construct a natural weather phenomenon and wind resistance model to simulate the common wind resistance and natural weather phenomena in low-altitude flight processes, and establish a typical low-altitude meteorological condition environment simulation model;

[0036] Among them, the typical surface environment scene models include models of natural objects such as mountains, canyons, hills, plains, waters, trees, and models of artificial objects such as buildings, high-voltage towers, communication towers, and bridges; the parameter settings of various models are as follows:

[0037] ① Mountain simulation model parameters include:

[0038] Altitude: The altitude of different locations on the mountain range;

[0039] Slope: the inclination of the ground;

[0040] Slope aspect: the direction of the ground's slope;

[0041] Curvature: the curvature of the terrain;

[0042] Terrain roughness: the curvature of the terrain;

[0043] Ground Horizon Outline: Ground Horizon Outline Curve of Mountain Area.

[0044] ②Canyon simulation model parameters:

[0045] Canyon depth: the vertical distance between the bottom of the canyon and the tops of the mountains on both sides;

[0046] Canyon width: the distance between the widest points of the two sides of the canyon;

[0047] Slope: The degree of inclination of the canyon wall;

[0048] Slope aspect: the direction of inclination of the canyon wall;

[0049] Topographic roughness: the degree of undulation and irregularity of the canyon topography;

[0050] Ground Horizon Outline: Ground horizon outline curve of the canyon area.

[0051] ③Hill simulation model parameters:

[0052] Altitude: average and maximum altitude of hilly terrain;

[0053] Slope: The degree of inclination of a hill surface;

[0054] Aspect: the direction of the hill surface;

[0055] Topographic relief: the fluctuation and changes in hilly terrain;

[0056] Terrain roughness: the degree of irregularity of the terrain;

[0057] Ground Horizon Outline: Ground horizon outline curve in hilly areas.

[0058] ④Plain simulation model parameters:

[0059] Plain Contour: Plain contour curve.

[0060] ⑤ Water area simulation model parameters:

[0061] Water Contour: Water contour curve.

[0062] ⑥Tree simulation model parameters:

[0063] Height: tree height;

[0064] Density: the number of trees per unit area;

[0065] Coverage: The contour curve of the ground covered by trees.

[0066] ⑦Building model parameters:

[0067] Geometry: The building is bounded by a 3D rectangular geometric curve.

[0068] ⑧High-voltage tower model parameters:

[0069] Geometric shape: circumscribed 3D rectangular geometric curve;

[0070] Line direction: the direction of the high-voltage line path.

[0071] ⑨Communication tower model parameters:

[0072] Geometric shape: circumscribed 3D rectangular geometric curve;

[0073] Line direction: the direction of the communication line path.

[0074] ⑩ Bridge model parameters:

[0075] Geometry: The geometric curve of the bridge profile.

[0076] Among them, the natural weather phenomenon models include wind, rain, cloud, fog and other models, which simulate the impact of weather conditions on drones and support the presentation of various weather phenomena as special events in a simulated low-altitude environment; the parameter settings of various models are as follows:

[0077] Wind field model:

[0078] The power spectral density (PSD) method is used to describe the variation of wind speed over time and space to ensure that the simulated wind field is statistically consistent with the measured wind field. The main steps are as follows:

[0079] ① The Davenport model is selected as the power spectral density (PSD) model to describe the wind speed and describe the energy distribution at different frequencies. The specific expression formula is as follows:

[0080]

[0081] In the formula, is the wind speed power spectral density (unit: ), is the frequency (unit: Hz), k is the surface roughness coefficient, is the average wind speed at a height of 10 meters (unit: m / s), .

[0082] ② Use Cholesky decomposition technology to generate wind speed time history with target PSD characteristics. This involves converting PSD into time domain signals, mainly through the inverse transform of Fourier transform. The basic process is as follows:

[0083] 1: Select a suitable frequency range, usually 0 to a certain cutoff frequency, and calculate the target PSD value within this frequency range;

[0084] 2: For each frequency point, generate an independent random phase angle, which will be used for subsequent Fourier transform;

[0085] 3: Perform the inverse Fourier transform and convert the PSD to the time domain using the following process:

[0086] The PSD value of each frequency point is squared to obtain the amplitude of the frequency point; the obtained amplitude is combined with the random phase angle to obtain the complex Fourier coefficient; these complex Fourier coefficients are inverse Fourier transformed to obtain the time domain signal.

[0087] 4: Cholesky decomposition, the specific process is:

[0088] (a) Construct a covariance matrix whose elements are determined by the PSD through the following relationship:

[0089] ,in is the PSD, N is the number of data points;

[0090] (b) Perform Cholesky decomposition on the covariance matrix to obtain a lower triangular matrix L;

[0091] (c) Generate a random vector whose elements are random numbers from a standard normal distribution;

[0092] (d) By multiplying the lower triangular matrix L with the random vector, the time domain signal of the specific target PSD characteristics is obtained.

[0093] ③ Accurately simulate the impact of ground terrain on the wind field, including wind speed changes and streamline distortion caused by complex terrain (buildings, hills, etc.). CFD numerical simulation is used to describe the impact of terrain on wind speed changes, and this impact is integrated into the above wind model.

[0094] Step 2: Use typical surface environment scene models and typical low-altitude meteorological condition environment simulation models to randomly construct low-altitude flight environments such as high-rise buildings, rural fields, mountainous hills, alpine river valleys, forests and grasslands, construction sites and cargo yards. In this process, the multimodal large model can be used to generate diverse and colorful application scenario designs based on its text-to-image capability.

[0095] Step 3: Randomly set the flight starting point and end point in various low-altitude flight environments, use the route planning algorithm to generate the flight route, and then select a certain type of UAV to perform the simulated flight mission. During the flight, special events are randomly set to build a realistic low-altitude flight environment. The reinforcement learning algorithm is used to drive the low-altitude UAV simulation model to face various environments and overcome various emergency events to complete the flight mission from the flight starting point to the end point. At the same time, the low-altitude flight surface environment data, meteorological conditions, UAV control parameters and UAV attitude data of each flight are recorded.

[0096] Step 4: Collect low-altitude flight surface environment data, meteorological conditions, UAV control parameters, and UAV attitude data recorded in various typical low-altitude flight simulation application scenarios, perform data cleaning, data enhancement, and feature extraction, and align them according to time;

[0097] Among them, data cleaning is to remove invalid, erroneous or duplicate low-altitude flight environment data, meteorological conditions, UAV control parameters and UAV attitude data to ensure data quality.

[0098] Data enhancement is as follows: according to the semantically reasonable value range of low-altitude flight environment data, meteorological conditions, UAV control parameters and UAV attitude data, rotation, scaling, cropping, random changes and other means are used to further increase the diversity of various types of data.

[0099] The feature extraction is as follows: for low-altitude flight environment data of video image type, ResNet is used to extract the features of video image data; for formatted numerical meteorological condition environment data, UAV control parameter data and UAV attitude data, features are selected according to their relevance to flight safety, and wind speed, wind direction, temperature, humidity, temperature moving average and humidity moving average are used as the feature set of meteorological condition environment; ascent, descent, forward, backward, left movement and right movement are used as the feature set of UAV control parameters; the roll, pitch, yaw, roll angular velocity, pitch angular velocity, yaw angular velocity, attitude angle, angular acceleration, linear acceleration, heading and altitude of the UAV are used as the feature set of UAV attitude data.

[0100] Time alignment is: low-altitude flight surface environment data is the video data of the drone's forward perspective. Low-altitude meteorological environment data, drone control parameters and drone attitude data are formatted numerical data. The four types of data are aligned in time and space dimensions to meet the data alignment requirements of what kind of control action causes what kind of flight attitude change under what terrain environment and meteorological conditions.

[0101] Step 5: Use a multimodal big data training mechanism to generate a low-altitude UAV flight safety model through training based on the time-aligned data;

[0102] The Transformer model is used to achieve unified expression of video image data and formatted numerical data and the fusion of different types of data, supporting training and learning based on cross-modal data.

[0103] The loss function in the model training process uses the Huber loss function to measure the difference between the probability distribution predicted by the model and the actual situation. The Huber loss function combines the advantages of the mean square error (MSE) and the mean absolute error (MAE) loss functions and is insensitive to outliers. Considering that in the actual low-altitude flight environment, both terrain data and meteorological conditions data may differ greatly from the same values ​​in the simulation environment, the Huber function is selected as the loss function, which is defined as follows.

[0104]

[0105] In the formula, is the predicted value, is the true value, is a threshold parameter that determines the transition of the loss function from a quadratic function to a linear function.

[0106] In addition, the Dropout technology is used to prevent the model from overfitting, and the gradient clipping technology is used to avoid the gradient explosion problem.

[0107] During model training, use pre-trained model initialization or random initialization to initialize parameters. Then, select Adam or SGD as the optimizer, set hyperparameters such as learning rate and momentum, determine the batch size based on the memory size and model complexity, and set the number of iterations based on the model convergence.

[0108] The flight success rate is used as the model performance evaluation indicator, and the cross-validation method is used to evaluate the generalization ability of the model. By analyzing the performance of the model on different types of data, the model weaknesses are found to point out the direction for model optimization. And according to the performance of the model on the validation set, the learning rate, batch size and other hyperparameters are adjusted for tuning. In addition, the model structure or feature fusion method is adjusted according to the evaluation results.

[0109] Step 6: compress the large low-altitude UAV flight safety model generated by training, deploy the compressed model on the UAV, drive the UAV to fly in a real low-altitude airspace environment, collect the attitude data of the UAV during the actual flight, and use the attitude data for model evaluation and retraining.

[0110] During the compression process, the model's weight matrix is ​​decomposed into a low-rank matrix using a low-rank decomposition technique based on the least squares method, reducing the model size and computational complexity to adapt to the airborne embedded operating environment with limited computing and storage resources.

Claims

1. A large model construction method serving the safe flight of low-altitude unmanned aerial vehicles, characterized in that: The following steps are involved: Step 1: Use a six-degree-of-freedom aerodynamic model to link the mass, inertia, force, acceleration and angular velocity of the drone, describe the translation and rotation of the drone, and establish a low-altitude drone simulation model; at the same time, construct natural and man-made objects, match and arrange them to establish a typical surface environment scene model; And build a natural weather phenomenon and wind resistance model to simulate the common wind resistance and natural weather phenomena in low-altitude flight, and establish a typical low-altitude meteorological condition environment simulation model; Step 2, using a typical surface environment scene model and a typical low-altitude meteorological condition environment simulation model to randomly generate a low-altitude flight environment; Step 3: randomly set the flight starting point and end point in various low-altitude flight environments, generate a flight route using a route planning algorithm, and then select a certain type of drone to perform a simulated flight mission. During the flight, special events are randomly set to build a realistic low-altitude flight environment. A reinforcement learning algorithm is used to drive the low-altitude drone simulation model to face various environments and overcome various emergency events to complete the flight mission from the flight starting point to the end point. At the same time, the low-altitude flight surface environment data, meteorological conditions, drone control parameters, and drone attitude data of each flight are recorded; Step 4: Collect low-altitude flight surface environment data, meteorological conditions, UAV control parameters, and UAV attitude data recorded in various typical low-altitude flight simulation application scenarios, perform data cleaning, data enhancement, and feature extraction, and align them according to time; Step 5: Use a multimodal big data training mechanism to generate a low-altitude UAV flight safety model through training based on the time-aligned data; Step 6: compress the large low-altitude UAV flight safety model generated by training, deploy the compressed model on the UAV, drive the UAV to fly in a real low-altitude airspace environment, collect the attitude data of the UAV during the actual flight, and use the attitude data for model evaluation and retraining.

2. A large model construction method serving the safe flight of low-altitude unmanned aerial vehicles according to claim 1, characterized in that: The natural features constructed in step 1 include mountains, canyons, hills, plains, waters and trees; the man-made features include buildings, high-voltage towers, communication towers and bridges; the surface environment scenes include high-rise buildings, rural fields, mountains and hills, alpine river valleys, forests and grasslands, and construction site cargo yards.

Images