End-to-end unmanned aerial vehicle autonomous control method based on environmental complexity

Abstract

The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, and discloses an end-to-end unmanned aerial vehicle autonomous control method based on environmental complexity, which comprises the following steps: carrying out analogue simulation to acquire image data acquired by a visual sensor in the flight process of an unmanned aerial vehicle, and preprocessing the image data; a convolutional neural network is trained based on the image data, and the convolutional neural network obtained after training is completed serves as an environment vector acquisition model; acquiring images continuously acquired in the flight process of the unmanned aerial vehicle in real time, inputting the images into the environment vector acquisition model to obtain corresponding environment vectors, calculating an environment change rate based on the environment vectors, and calculating environment complexity based on the environment change rate; according to the environment complexity obtained through calculation, the environment type is judged; and based on different environment types, different control models are adopted to control the unmanned aerial vehicle. The method can improve the flight safety and navigation precision of the unmanned aerial vehicle, and is especially suitable for variable and dense flight scenes.

Description

Technical Field

[0001] The present invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, and relates to an end-to-end unmanned aerial vehicle autonomous control method based on environmental complexity. Background Art

[0002] In the past decade, the demand for the use of autonomous UAV systems in surveying, search and rescue, etc. has grown exponentially. With the rise of these applications, the control algorithms designed need to be highly robust and real-time to disturbances such as motion blur, high dynamic range, model uncertainty, and aerodynamics of UAVs in complex environments. Traditional UAV navigation tasks are decomposed into three parts: perception, planning, and control. The perception module provides state estimation for the UAV through cameras and inertial navigation, but the camera is easily affected by lighting conditions and textureless environments, and the high-speed translation and rotation of the UAV will cause large optical flow, making it difficult to detect and track features in the image, resulting in a large amount of drift in the inertial navigation state estimate in a short period of time. Performing the perception, planning, and control subtasks in sequence will increase the signal processing delay and is not suitable for high-speed flight of UAVs.

[0003] Using neural networks to replace perception, planning, and control modules can directly process high-dimensional perception signals, but most learning-based perception networks are deployed in environments that have not been trained with data sets, and the generalization effect is poor. Independent modules will also cause compound errors and delays in the system. At present, many scholars directly map the observation information obtained from the sensor end to the control end of the autonomous driving vehicle. It has good robustness to modeling uncertainty and system delays, and the deployment process is simple. It has been a commonly used control method in the field of autonomous driving in recent years. Unlike autonomous driving, high-speed control of drones needs to adapt to different or unknown environments. Therefore, it is necessary to further improve the control method of drones to achieve the premise of ensuring flight efficiency and enhancing its safety and robustness in complex environments. Summary of the invention

[0004] The present invention aims to propose an end-to-end UAV autonomous control method based on environmental complexity. By using deep neural networks combined with reinforcement learning technology, appropriate end-to-end control models and strategies are selected according to the complexity of the environment in a complex dynamic environment to solve the problems of low efficiency in visual data processing, dynamic obstacle avoidance and path planning of existing UAVs when flying in complex environments.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is: an end-to-end UAV autonomous control method based on environmental complexity, comprising the following steps: Step 1: simulate the high-speed flight environment of the UAV, collect and pre-process the image data obtained by the visual sensor during the flight of the UAV; Step 2: training the first convolutional neural network based on the image data, and using the first convolutional neural network obtained after the training as an environment vector acquisition model; Step 3: Real-time acquisition of images continuously collected during the flight of the drone, and input into the environment vector acquisition model to obtain the corresponding environment vector, calculate the environment change rate based on the environment vector, and calculate the environment complexity based on the environment change rate. The calculation formula of the environment change rate is: ; in, is the environmental change rate, , , Respectively , and The environment vector at the moment; Step 4: Determine the environment type based on the calculated environment complexity; and use different control models to control the drone based on different environment types.

[0006] The calculation formula of environmental complexity is: ; in, , , is the weighting coefficient, is the number of obstacles in the current field of view, is the distribution of obstacles, is the rate of environmental change.

[0007] In the step 1, a high-speed flight environment of a UAV is simulated based on the Flightmare simulator; The preprocessing includes cropping, scaling, normalization, data enhancement operations, and labeling of the image.

[0008] In the step 2, feature extraction and feature fusion are performed on the image data to obtain a feature vector, and the first convolutional neural network is trained based on the feature vector. The obtained feature vector is: ,in, They respectively represent the number, distribution, size and dynamic properties of obstacles in the environment.

[0009] In step 4, each control model is implemented based on a second convolutional neural network and a recurrent neural network, which is used to extract the image time series information collected during the flight of the UAV and perform flight path planning.

[0010] In step 4, in each control model, the overall reward and penalty function of the second convolutional neural network is: ; in, , and They represent distance reward, obstacle avoidance penalty, and dynamic adaptation reward respectively, and their expressions are: ; ; ; in, is the gain factor, Indicates the current position of the drone. Indicates the location of the target point; is the weight coefficient of obstacle avoidance penalty, Indicates the position of the drone to the obstacle. Indicates safe distance. To dynamically adapt the reward weight coefficient, represents the velocity vector of the obstacle, Indicates the obstacle avoidance direction of the drone.

[0011] In step 4, the specific method for determining the environment type is: like , it is judged as a low-complexity environment; like , then it is judged as a medium complexity environment; like , it is judged as a high-complexity environment; in, and are the first empirical threshold and the second empirical threshold, < .

[0012] In step 4, the specific method of using different control models to control the drone based on different environment types is: If it is a low-complexity environment, a low-complexity control model is used, and the control formula is: ; in, For the control instructions of the drone, represents the control function of the low-complexity control model, The image input at the current moment; If the environment is of medium complexity, the medium complexity control model is adopted, and the control formula is: ; in, represents the control function of a medium-complexity control model, Represents the current environment feature vector; If it is a high-complexity environment, a high-complexity control model is used, and the control formula is: ; in, represents the control function of a high-complexity control model, represents an image sequence containing time series information, and Represent the angular velocity and acceleration of IMU data respectively.

[0013] In step 4, the control formula when the low complexity model switches to the medium complexity model is: , ; The control formula when the medium complexity model switches to the high complexity model is: , ; The control formula when the high complexity model switches to the medium complexity model is: , ; The control formula for switching from a medium complexity model to a low complexity model is: , ; in, , , , Represent the corresponding smoothing factors, which change dynamically over time.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention provides an end-to-end UAV autonomous control method based on environmental complexity. A large amount of UAV visual data is collected through a simulation platform to train a convolutional neural network, which can accurately extract environmental vectors. Then, the overall reward and punishment function including distance reward, obstacle avoidance penalty, and dynamic adaptation reward is used as the objective function of the convolutional neural network in the control model. The convolutional neural network can learn the optimal strategy for obstacle avoidance and trajectory tracking during the training process, and finally directly output control instructions (such as speed, angular velocity, thrust, and rudder angle) to achieve accurate obstacle avoidance and trajectory control, thereby significantly improving the response speed and decision-making accuracy of the UAV in a complex environment, so that the UAV can achieve end-to-end autonomous flight in various environments. Therefore, the present invention accurately evaluates environmental complexity through real-time visual data and convolutional neural networks, thereby improving the speed and accuracy of environmental complexity evaluation; 2. The present invention extracts environmental feature vector features based on convolutional neural networks and calculates the environmental change rate. It then divides the environmental complexity based on factors such as the number of obstacles, distribution density, and dynamic environmental change rate. Finally, different control models and control strategies are selected based on the environmental complexity, which greatly improves the processing speed of drone control and can take into account both the speed and accuracy of drone control. 3. When switching between different control models, the present invention adopts a soft switching method to ensure a smooth transition of the switching process, improve the smoothness of the UAV control, and enhance the safety and robustness of the UAV in complex environments while ensuring flight efficiency.

[0015] In summary, the present invention realizes efficient navigation in environments of different complexity, ensures the flight safety and navigation accuracy of the UAV, and is particularly suitable for changeable and intensive flight scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A schematic diagram of the workflow of an end-to-end UAV autonomous control method based on environmental complexity provided by an embodiment of the present invention; Figure 2 A closed-loop control architecture diagram of a learning system in an end-to-end UAV autonomous control method based on environmental complexity provided in an embodiment of the present invention; Figure 3 Schematic diagram of depth image preprocessing in an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0018] like Figure 1~2 As shown, an embodiment of the present invention provides an end-to-end UAV autonomous control method based on environmental complexity, comprising the following steps: Step 1: Simulate the high-speed flight environment of the UAV, collect and pre-process the image data obtained by the visual sensor during the flight of the UAV.

[0019] Specifically, in the step one, a high-speed flight environment of a UAV is simulated based on the Flightmare simulator; the preprocessing includes cropping, scaling, normalizing, data enhancement operations, and labeling of the image.

[0020] In this embodiment, scenes are added to the Flightmare simulation to train data, and virtual environments of different complexities are obtained by randomly placing numbers and dynamic obstacles on uneven open ground. Given the posture of the drone, the simulator Flightmare is used to collect data on the flight of the drone in a cluttered environment, and the simulated RGB image and real depth image collected by the drone in this posture can be obtained. Among them, the simulator Flightmare consists of a rendering engine and a physics engine based on Unity. The two modules are completely decoupled and can run independently of each other. The rendering module can provide visual information from low to high fidelity, and can also simulate sensor noise, such as motion blur and environmental momentum (such as wind).

[0021] In addition, the image dataset obtained in the simulation environment needs to be preprocessed and feature extracted to obtain detailed information about obstacles and provide decision support for the autonomous navigation of the drone. The preprocessing process includes image cropping, scaling, normalization and data enhancement operations to improve the generalization ability of the model. Next, the preprocessed dataset is annotated, and the annotation information includes the action and state of the obstacle to provide supervision information for subsequent training.

[0022] In this embodiment, the purpose of preprocessing is to enhance data diversity, reduce model overfitting, and provide standardized input for neural network feature extraction. The specific preprocessing operations are cropping, scaling, normalization, and data enhancement. The present invention selects a depth image resolution of 640×480, takes a feature vector size of 16×16, removes eight pixels from the edge of the image to locate the center of the image, and scales the depth image to finally obtain a processed image with a resolution of 39×29. The processing process is as follows: Figure 3 As shown in the figure, a single pixel corresponds to a 2°×2° field of view. In order to retain the information of the obstacle closest to the drone, the grayscale of each pixel takes the minimum value in the corresponding area before reduction.

[0023] Step 2: Based on the convolutional neural network (CNN), feature extraction and feature fusion are performed on the image data obtained in step 1 to obtain feature vectors, and the convolutional neural network is trained based on the extracted feature vectors. The convolutional neural network obtained after training is used as the environment feature vector acquisition model.

[0024] Specifically, in step 2, the feature vector obtained is: ,in, They respectively represent the number, distribution, size and dynamic properties of obstacles in the environment.

[0025] In this embodiment, the multi-layer structure of the convolutional neural network (CNN) is used to extract features from the preprocessed image data in step 1 to analyze the obstacles and terrain information in the environment. During the extraction, the features are gradually extracted from the edge and low-level features to the high-level semantic features. The specific feature extraction process is as follows: Low-level feature extraction: The initial convolution layer scans the image through a small convolution kernel to extract low-level features such as edges, corners, and textures. The convolution operation formula is: .in, represents the eigenvalue after the convolution operation, is the weight of the convolution, These low-level features are the pixel values ​​of the input image and provide the basis for further analysis of the shape and position of obstacles.

[0026] High-level feature extraction: Convolutional neural networks extract higher-level semantic features through high-level convolutional layers, such as the specific shape, type, and location of obstacles. These features help drones understand the distribution of obstacles in complex environments and assist in generating path planning decisions. The specific high-level convolution operation formula is: .in, For the Feature map after layer convolution; For the Layer convolution kernel; It is an activation function that introduces nonlinear capabilities into the model; is the bias term. By marking and counting the areas in the feature map that belong to obstacles, the number of obstacles can be obtained. .

[0027] Assume the position of the drone as the origin, and divide the space around the drone into multiple small areas with a certain spatial resolution (for example, divide the horizontal and vertical directions into m and n For each obstacle, the small area where it is located is determined according to its position information extracted from the high-level semantic features. Then, a The matrix , where each element Indicated in Row, No. The number of obstacles in the small area of ​​the column. This matrix can be used To express, that is .

[0028] It is calculated based on the shape information extracted from the high-level semantic features of the obstacle. Assuming that the shape of the obstacle can be approximated as a rectangle, by measuring its length and width in the image (in pixels), and combining the camera's intrinsic parameters and the drone's flight altitude, the pixel size can be converted to the actual physical size (for example, meters). The length and width of an obstacle in the image are and (pixels), the camera internal parameters are (focal length), the drone's flight altitude is (m), then The actual size of the obstacle (square meters) can be calculated by the formula Calculate. Combine all obstacle size information into a vector or matrix to get the obstacle size For example, if there is obstacles, you can build a Dimensional vector Indicates the size of the obstacle.

[0029] The dynamic attributes mainly include the speed and direction of movement of the obstacle. The position change of an obstacle between two adjacent frames is (in and are the position coordinates of the obstacle in the two frames respectively), the time interval between the two frames is , then The speed of the obstacle is: . Movement direction The expression of a dynamic property is a Dimensional vector .

[0030] Combine the high-level features extracted from the high-level convolutional layers of the convolutional neural network to form a feature vector ,The feature vector includes the number, distribution, size and dynamic properties of obstacles in the environment, which can describe the complexity of the current environment.

[0031] Step 3: Acquire the images continuously collected during the flight of the drone in real time, and input the environment vector acquisition model to obtain the corresponding environment vector, calculate the environment change rate based on the environment vector, and calculate the environment complexity based on the environment change rate.

[0032] According to the relevant knowledge of vectors, the result of the inner product operation of vectors is a scalar, which can represent the degree of deviation or closeness of two vectors. The larger the inner product, the closer or closer the two vectors are. At this moment, the environment vector With the previous moment The environment vector Do the inner product operation, At this moment, the environment vector With the previous moment The environment vector Perform inner product operation. The inner product of two vectors corresponding to two time intervals can be used to determine the degree of change of the environment vector at different times by the change of the inner product.

[0033] Taking the derivative of the above two inner product operations with respect to time, we can get the environmental change rate, and the time interval is , that is, the calculation formula of the environmental change rate is: ; (1) in, is the environmental change rate, , , Respectively , and The environment vector at the moment.

[0034] In this embodiment, the environment complexity is calculated by combining visual and IMU data using the following formula: ; (2) In the formula, , , It is a weighting coefficient used to adjust the impact of different complexity factors on the total complexity. is the number of obstacles in the current field of view, is the distribution of obstacles, is the environmental change rate. C represents the environmental complexity. In this embodiment, it is proposed to set the environmental complexity to a value between [0,1].

[0035] Step 4: Determine the environment type based on the calculated environment complexity; and use different control models to control the drone based on different environment types.

[0036] Specifically, in this embodiment, each control model combines convolutional neural network (CNN) and recurrent neural network (RNN) to extract time series information, which is used to capture the dynamic characteristics of obstacles and environment changing over time. This dynamic characteristic includes the direction of movement of obstacles, speed changes, and the distribution adjustment of dynamic obstacles in the environment, thereby introducing information of the time dimension in path planning and obstacle avoidance strategies, so that the drone can predict potential dangers or adjust the path in advance. In path planning, time series information helps the model dynamically update the flight trajectory to avoid the delayed response that may be caused by relying solely on static environmental information. For example, the dynamic obstacle avoidance weight in the control function will be adjusted in real time according to the speed and direction of the obstacle extracted from the time series to optimize the obstacle avoidance angle and speed of the drone. In addition, time series information can also accurately plan the flight path in an environment with dense and dynamically changing obstacles by optimizing the reward and punishment function, enhance the adaptive ability of the drone in a complex dynamic environment, and ensure the stability and safety of the flight.

[0037] Furthermore, in order to train the deep reinforcement learning neural network so that the drone can avoid obstacles and maintain smooth flight, in step 4, the overall reward and penalty function of the convolutional neural network used in the control model is: ; (3) in, , and They represent distance reward, obstacle avoidance penalty, and dynamic adaptation reward respectively.

[0038] The drone needs to move towards the target when flying, so a distance reward is designed to encourage the drone to approach the target point. To ensure that the drone can maintain a safe distance during obstacle avoidance, an obstacle avoidance penalty is designed to punish the drone for approaching obstacles. The drone needs to identify the dynamic properties of obstacles (such as speed and direction of movement) to optimize the obstacle avoidance trajectory, so a dynamic adaptation reward is designed to encourage the drone to adjust the obstacle avoidance direction to avoid collisions with dynamic obstacles. The expressions are: ; (4) ; (5) ; (6) in, is the gain factor, Indicates the current position of the drone. Indicates the location of the target point; is the weight coefficient of obstacle avoidance penalty, Indicates the position of the drone to the obstacle. Indicates the safe distance; when the distance between the drone and the obstacle is less than the safe distance, the obstacle avoidance penalty is negative, resulting in a negative penalty, otherwise it is zero. To dynamically adapt the reward weight coefficient, represents the velocity vector of the obstacle, Indicates the obstacle avoidance direction of the drone. If the obstacle's movement direction is consistent with the drone's obstacle avoidance direction, a penalty is generated to encourage the drone to adjust its direction.

[0039] Through the above-mentioned comprehensive reward and punishment function, in the control model of the present invention, the convolutional neural network can learn the optimal strategy for obstacle avoidance and trajectory tracking during the training process, and finally directly output control instructions (such as speed, angular velocity, thrust, and rudder angle) to achieve precise obstacle avoidance and trajectory control, thereby significantly improving the response speed and decision-making accuracy of the UAV in complex environments, enabling the UAV to achieve end-to-end autonomous flight in various environments.

[0040] Specifically, in this embodiment, a variety of control models are designed according to different complexity levels. Each model generates control instructions through an end-to-end deep learning strategy and can adapt to complex environments from sparse to dense and from static to dynamic.

[0041] In step 4, the control model is selected by the adaptive selection module. The adaptive selection module selects the control model that best suits the current environment in real time based on the output signal of the environmental complexity assessment module. The module first determines the applicable model based on the current complexity level, and then switches the model in real time according to the complexity change. The control logic of the adaptive selection module is as follows: ; (7) In this embodiment, it is proposed to set the environmental complexity to a value between [0, 1]. Environmental complexity in different intervals represents different environmental complexity levels. Environmental complexity can be divided into three levels according to the operating environment of the drone: : Low complexity, empty or obstacle-sparse environments.

[0042] : Medium complexity, there are a few obstacles in the environment, mainly static objects.

[0043] : Complex environments with high complexity, dense obstacles and dynamic changes.

[0044] in, and are the first empirical threshold and the second empirical threshold, which are used to classify the complexity level of the environment. < .

[0045] Based on different environment types, the specific methods of using different control models to control the drone are: If it is a low-complexity environment, a low-complexity control model is used, and the control formula is: ; (8) in, For the control instructions of the drone, represents the control function of the low-complexity control model, The image input at the current moment; the low-complexity model is designed to cope with simple environments. Its main feature is to focus on fast path generation to ensure that the drone can fly at high speed in an open area and has a fast response speed. The structure of this model is simplified, requiring only a basic path generation network, without complex obstacle detection and obstacle avoidance planning.

[0046] If the environment is of medium complexity, the medium complexity control model is adopted, and the control formula is: ; (9) in, represents the control function of a medium-complexity control model, Represents the current environment feature vector, which is obtained by acquiring the model through the environment vector; this model is suitable for environments with a small number of obstacles and has basic obstacle avoidance functions. The model adds an obstacle avoidance module on the basis of path generation, and generates the optimal path between static obstacles through visual information and neural network learning.

[0047] If it is a high-complexity environment, a high-complexity control model is used, and the control formula is: ; (10) in, represents the control function of a high-complexity control model, represents an image sequence containing time series information, and Represent the angular velocity and acceleration of IMU data respectively. The high-complexity control model is dedicated to dense obstacles and dynamically changing environments. Its structure is more complex, including sophisticated path planning, dynamic obstacle avoidance, and real-time attitude adjustment modules.

[0048] Referring to formulas (7) to (10), in this embodiment, control models of different complexity use different input parameters, and the final control instructions of the drone are obtained through convolutional neural networks (CNN) and recurrent neural networks (RNN). Therefore, the control method of the embodiment of the present invention can be more adaptable to complex environments from sparse to dense and from static to dynamic.

[0049] Furthermore, in order to avoid control discontinuity during the switching process, this embodiment adopts a "soft switching" strategy to achieve transition between models through time-weighted smooth conversion, thereby reducing sudden changes in control instructions.

[0050] Specifically, in step 4, in order to achieve soft switching, the control formula when the low complexity model switches to the medium complexity model is: , ; (11) The control formula when the medium complexity model switches to the high complexity model is: , ; (12) The control formula when the high complexity model switches to the medium complexity model is: , ; (13) The control formula for switching from a medium complexity model to a low complexity model is: , ; (14) in, , , , They represent the corresponding smoothing factors, which change dynamically over time to ensure a smooth transition of the switching process.

[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An end-to-end UAV autonomous control method based on environmental complexity, characterized in that: The following steps are involved: Step 1: simulate the high-speed flight environment of the UAV, collect and pre-process the image data obtained by the visual sensor during the flight of the UAV; Step 2: training the first convolutional neural network based on the image data, and using the first convolutional neural network obtained after the training as an environment vector acquisition model; Step 3: Real-time acquisition of images continuously collected during the flight of the drone, and input into the environment vector acquisition model to obtain the corresponding environment vector, calculate the environment change rate based on the environment vector, and calculate the environment complexity based on the environment change rate. The calculation formula of the environment change rate is: ； in, is the environmental change rate, , , Respectively , and The environment vector at the moment; Step 4: Determine the environment type based on the calculated environment complexity; and use different control models to control the drone based on different environment types.

2. The end-to-end UAV autonomous control method based on environmental complexity according to claim 1 is characterized in that: The calculation formula of environmental complexity is: ； in, , , is the weighting coefficient, is the number of obstacles in the current field of view, is the distribution of obstacles, is the rate of environmental change.

3. The end-to-end UAV autonomous control method based on environmental complexity according to claim 1 is characterized in that: In the step 1, a high-speed flight environment of a UAV is simulated based on the Flightmare simulator; The preprocessing includes cropping, scaling, normalization, data enhancement operations, and labeling of the image.

4. The end-to-end UAV autonomous control method based on environmental complexity according to claim 1 is characterized in that: In the step 2, feature extraction and feature fusion are performed on the image data to obtain a feature vector, and the first convolutional neural network is trained based on the feature vector. The obtained feature vector is: ,in, They respectively represent the number, distribution, size and dynamic properties of obstacles in the environment.

5. The end-to-end UAV autonomous control method based on environmental complexity according to claim 1 is characterized in that: In step 4, each control model is implemented based on a second convolutional neural network and a recurrent neural network, which is used to extract the image time series information collected during the flight of the UAV and perform flight path planning.

6. The end-to-end UAV autonomous control method based on environmental complexity according to claim 5 is characterized in that: In step 4, in each control model, the overall reward and penalty function of the second convolutional neural network is: ； in, , and They represent distance reward, obstacle avoidance penalty, and dynamic adaptation reward respectively, and their expressions are: ； ； ； in, is the gain factor, Indicates the current position of the drone. Indicates the location of the target point; is the weight coefficient of obstacle avoidance penalty, Indicates the position of the drone to the obstacle. Indicates safe distance. To dynamically adapt the reward weight coefficient, represents the velocity vector of the obstacle, Indicates the obstacle avoidance direction of the drone.

7. The end-to-end UAV autonomous control method based on environmental complexity according to claim 1 is characterized in that: In step 4, the specific method for determining the environment type is: like , it is judged as a low-complexity environment; like , then it is judged as a medium complexity environment; like , it is judged as a high-complexity environment; in, and are the first empirical threshold and the second empirical threshold, < .

8. The end-to-end UAV autonomous control method based on environmental complexity according to claim 7 is characterized in that: In step 4, the specific method of using different control models to control the drone based on different environment types is: If it is a low-complexity environment, a low-complexity control model is used, and the control formula is: ； in, For the control instructions of the drone, represents the control function of the low-complexity control model, The image input at the current moment; If the environment is of medium complexity, the medium complexity control model is adopted, and the control formula is: ； in, represents the control function of a medium-complexity control model, Represents the current environment feature vector; If it is a high-complexity environment, a high-complexity control model is used, and the control formula is: ； in, represents the control function of a high-complexity control model, represents an image sequence containing time series information, and Represent the angular velocity and acceleration of IMU data respectively.

9. The end-to-end UAV autonomous control method based on environmental complexity according to claim 7 is characterized in that: In step 4, the control formula when the low complexity model switches to the medium complexity model is: ， ； The control formula when the medium complexity model switches to the high complexity model is: ， ； The control formula when the high complexity model switches to the medium complexity model is: ， ； The control formula for switching from a medium complexity model to a low complexity model is: ， ； in, , , , Represent the corresponding smoothing factors, which change dynamically over time.

Images