Autonomous Navigation Methods for Aircraft Based on Monocular Depth Estimation and Reactive Strategy

By adopting a single-eye depth estimation and reactive strategy on the drone, the self-navigation algorithm of the drone is solved by using a convolutional neural network of quantization and pruning, combined with the control of the finite state machine. Limited deployment problems on the platform, the low-power consumption and efficient autonomous navigation function is achieved.

CN115686062BActive Publication Date: 2024-11-08TSINGHUA UNIVERSITY
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202211329504.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-10-27
Publication Date
2024-11-08
Estimated Expiration
2042-10-27

AI Technical Summary

Technical Problem

The existing autonomous navigation algorithms for drones are difficult to deploy on platforms with limited computing resources, especially the calculation complexity and memory occupancy of monocular depth estimation algorithms, which makes it difficult to achieve real-time operation on nano-type drones.

Method used

Using the aircraft autonomous navigation method based on monocular depth estimation and reactive strategies, the image depth features are extracted using convolutional neural network, and the network parameters are quantized and pruned, and deployed on an ultra-low power edge computing chip, combined with finite state The control method of the aircraft realizes autonomous navigation of the drone.

Benefits of technology

The autonomous navigation function of drones is realized on the ultra-low power computing platform, reducing computing complexity and power consumption, suitable for the low power consumption needs of nano-type drones, and can achieve autonomous obstacle avoidance and rapidity in complex environments flight.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115686062B_ABST
    Figure CN115686062B_ABST
Patent Text Reader

Abstract

The present invention discloses an autonomous navigation method for an aircraft based on monocular depth estimation and reactive strategy, the method comprising obtaining a training data set of an unmanned aerial vehicle image, and using the training data set to train a depth estimation network to obtain a trained depth estimation network; using multiple images in the training data set to quantize the network parameters of the trained depth estimation network to obtain a convolution tensor, and using the optimal tensor partitioning method to divide the convolution tensor to generate network code data; using real-time unmanned aerial vehicle images and network code data, a depth map of real-time unmanned aerial vehicle image estimation is obtained through network calculation of the trained depth estimation network; using a preset strategy formula to reactively calculate the depth map to generate an expected flight angular velocity of the unmanned aerial vehicle, and completing autonomous navigation of the unmanned aerial vehicle based on the expected flight angular velocity and the control method of a finite state machine. The present invention can effectively complete autonomous obstacle avoidance of unmanned aerial vehicles.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle navigation, and in particular to an autonomous navigation method for an aircraft based on monocular depth estimation and a reactive strategy. Background Art

[0002] Autonomous navigation is one of the most important and basic capabilities of UAVs. The traditional method of autonomous navigation of UAVs is to adopt the framework of "perception-localization-mapping-planning-obstacle avoidance-control". It includes using external sensors (such as GPS) or onboard sensors (such as visual inertial sensors) to estimate the motion of the UAV, build a local 3D map of the environment, and plan a safe trajectory that can be passed based on the UAV's posture and the environment map. However, such a method based on the "perception-localization-mapping-planning-obstacle avoidance-control" framework is too complex and difficult to implement for computing-constrained platforms. It is not yet possible to deploy such an algorithm on nano-UAVs. On the other hand, with the rise of artificial intelligence technology, recently published literature shows that breaking through the existing "perception-localization-mapping-planning-obstacle avoidance-control" paradigm and designing a lightweight "end-to-end" algorithm based on convolutional neural network (CNN) is expected to achieve autonomous navigation of nano-UAVs, and can run in real time on computing platforms with limited onboard resources with extremely low power consumption. These works provide new ideas for the research of nano-UAVs, that is, instead of using remote base stations as computing platforms, only relying on the resource-constrained platform of the UAV itself, and realizing the autonomous navigation of nano-UAVs through neural network algorithms.

[0003] Current research on monocular depth estimation mainly focuses on improving accuracy, but the computational complexity and memory usage of these monocular depth estimation algorithms are large, making them difficult to deploy on robotic systems, especially on platforms with limited computing resources and power consumption. Therefore, a key challenge is how to balance the accuracy of the algorithm with the requirements for computing resources. Summary of the invention

[0004] The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

[0005] To this end, the present invention proposes an autonomous navigation method for aircraft based on monocular depth estimation and reactive strategy. The method uses a convolutional neural network to extract image depth features for encoding and decoding to estimate the depth values ​​at different pixel points of the image, and reactively calculates the expected angular velocity of the current unmanned aerial vehicle based on the obtained depth map to complete autonomous obstacle avoidance. The overall algorithm has been successfully deployed on an ultra-low power edge computing chip, which requires quantizing the trained network parameters and decomposing each layer of convolution operations to be deployed on an ultra-low power edge computing chip.

[0006] Another object of the present invention is to provide an autonomous aircraft navigation system based on monocular depth estimation and reactive strategy.

[0007] To achieve the above object, the present invention proposes, on one hand, an autonomous navigation method for an aircraft based on monocular depth estimation and a reactive strategy, comprising:

[0008] Acquire a training data set of unmanned aerial vehicle images, and use the training data set to train a depth estimation network to obtain a trained depth estimation network;

[0009] Using multiple images in the training data set to quantize the network parameters of the trained depth estimation network to obtain a convolution tensor, and using an optimal tensor partitioning method to partition the convolution tensor to generate network code data;

[0010] Using the real-time unmanned aerial vehicle image and the network code data, a depth map of the real-time unmanned aerial vehicle image estimation is obtained through network calculation of the trained depth estimation network;

[0011] The depth map is reactively calculated using a preset strategy formula to generate an expected flight angular velocity of the unmanned aerial vehicle, and autonomous navigation of the unmanned aerial vehicle is completed based on the expected flight angular velocity and a finite state machine control method.

[0012] In addition, the autonomous aircraft navigation method based on monocular depth estimation and reactive strategy according to the above embodiment of the present invention may also have the following additional technical features:

[0013] Furthermore, in one embodiment of the present invention, after obtaining the trained depth estimation network, the method further includes: using the NetAdapt pruning algorithm to prune the trained depth estimation network to filter out a preset number of convolution kernels in the network.

[0014] Further, in one embodiment of the present invention, the using of a preset strategy formula to perform reactive calculation on the depth map to generate an expected flight angular velocity of the unmanned aerial vehicle includes:

[0015]

[0016]

[0017]

[0018] Among them, i is the angle corresponding to the pixel point, d i is the depth value of the pixel, φ is the current direction of the drone, Ψ goal is the angle corresponding to the target direction, λ goto, avoid ,c,σ,w b All are adjustable parameters;

[0019] The desired angular velocity is calculated by and the desired angular velocity towards the target direction Weighted to get the final desired flight angular velocity

[0020] Furthermore, in one embodiment of the present invention, the control method based on the expected flight angular velocity and the finite state machine to complete the autonomous navigation of the unmanned aerial vehicle includes: obtaining ranging data of sensors in the front, left and right directions of the unmanned aerial vehicle; and completing the autonomous navigation of the unmanned aerial vehicle based on the reactive algorithm and the wall-following strategy according to the ranging data and the expected flight angular velocity.

[0021] Furthermore, in one embodiment of the present invention, the method of dividing the convolution tensor using the optimal tensor division method to generate network code data includes: using the AutoTiler network tool to select the optimal tensor division method to divide the convolution tensor, so as to encapsulate the calculation process of the trained depth estimation network on the ultra-low power edge computing chip into code data expressed in C language.

[0022] To achieve the above object, the present invention further provides an autonomous aircraft navigation system based on monocular depth estimation and reactive strategy, comprising:

[0023] A network training module is used to obtain a training data set of unmanned aerial vehicle images, and use the training data set to train a depth estimation network to obtain a trained depth estimation network;

[0024] A tensor partitioning module, used to quantize the network parameters of the trained depth estimation network using multiple images in the training data set to obtain a convolution tensor, and to partition the convolution tensor using an optimal tensor partitioning method to generate network code data;

[0025] A depth estimation module, for obtaining a depth map of the real-time unmanned aerial vehicle image estimation by network calculation of the trained depth estimation network using the real-time unmanned aerial vehicle image and the network code data;

[0026] The strategy control module is used to perform reactive calculations on the depth map using a preset strategy formula to generate an expected flight angular velocity of the unmanned aerial vehicle, and complete autonomous navigation of the unmanned aerial vehicle based on the expected flight angular velocity and a finite state machine control method.

[0027] The autonomous navigation method and system of an aircraft based on monocular depth estimation and reactive strategy of the embodiment of the present invention takes the nano-sized unmanned aerial vehicle as the platform, and in view of its small size, light weight, low power consumption and other characteristics, only relies on the monocular camera as the sensor input, and designs a low-power neural network that can realize onboard computing, so as to realize the autonomous navigation function of the nano-sized unmanned aerial vehicle. The new navigation paradigm proposed by the present invention is based on the monocular depth estimation network + reactive strategy, that is, the expected angular velocity of navigation is calculated directly according to the input of the monocular camera sensor, so as to realize the autonomous flight and obstacle avoidance functions of the unmanned aerial vehicle.

[0028] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flowchart of an autonomous navigation method for an aircraft based on monocular depth estimation and reactive strategy according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the FastDepth network structure for reference of a monocular depth estimation neural network structure according to an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of a reactive algorithm according to an embodiment of the present invention;

[0033] FIG. 4( a ), FIG. 4( b ), FIG. 4( c ) and FIG. 4( d ) are schematic diagrams of four flight experiment scenarios and trajectories tested according to an embodiment of the present invention, respectively;

[0034] Figure 5 Schematic diagram of a reactive algorithm + wall-following strategy state machine according to an embodiment of the present invention;

[0035] Figure 6 Schematic diagram of the structure of an aircraft autonomous navigation system based on monocular depth estimation and reactive strategy according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0037] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0038] The following describes an autonomous navigation method and system for an aircraft based on monocular depth estimation and reactive strategy according to an embodiment of the present invention with reference to the accompanying drawings.

[0039] It is understandable that the embodiment of the present invention is deployed on the Crazyflie 2.0 nano-sized UAV platform. The flight platform is equipped with an optical flow board for positioning the aircraft and a TOF board for sensing the distance in front, behind, left and right. The AIdeck platform is used as the computing platform. AIdeck is mainly composed of the GAP8 chip and Himax camera of GreenWaves Technologies, which expands the computing power and will enable complex workloads based on artificial intelligence to run on nano-sized unmanned platforms. The GAP8 chip is a commercial embedded RISC-V multi-core processor derived from the PULP open source project, and its core consists of an advanced RISC-V MCU and a programmable eight-core processor. The implementation of the code part in the present invention is based on the Python language, and Pytorch is used as the deep learning framework.

[0040] Figure 1 It is a flow chart of an autonomous navigation method of an aircraft based on monocular depth estimation and reactive strategy according to an embodiment of the present invention.

[0041] like Figure 1 As shown, the method includes but is not limited to the following steps:

[0042] S1, obtaining a training data set of unmanned aerial vehicle images, and using the training data set to train a depth estimation network to obtain a trained depth estimation network.

[0043] Specifically, the present invention selects a suitable depth data set to train the parameters of the depth estimation network so that the estimation accuracy of the network meets the requirements, so as to obtain a trained depth estimation network. This embodiment uses the NYU Depth v2 data set. The NetAdapt pruning method is used to further lightweight the network.

[0044] S2, using multiple images in the training data set to quantize the network parameters of the trained depth estimation network to obtain a convolution tensor, and using the optimal tensor partitioning method to divide the convolution tensor to generate network code data.

[0045] Specifically, the network parameters are quantized and deployed. Quantization requires first selecting several images as input to obtain the reference range of the tensor t of each convolution layer [α t ,β t ), which is then mapped to an 8-bit pure integer tensor The quantization was done using the NNTOOL tool developed by GWT (GreenWaves Technologies). The deployment was done using the AutoTiler tool, which was used to find out how to maximize the parallel execution of all available cores on the GAP8 chip while minimizing the data transfer overhead and to package and generate C code for subsequent calls.

[0046] S3, using the real-time UAV image and network code data, obtains the depth map of the real-time UAV image estimation through network calculation of the trained depth estimation network.

[0047] Specifically, after step S2, the network's calculation process on the ultra-low power edge computing chip is encapsulated into a code expressed in C language. The real-time UAV image is obtained by calling the interface in gap_sdk and stored in an array, and then the image data is input, and the estimated depth information can be obtained through network calculation. The process is repeated to obtain the depth map estimated by the real-time camera image.

[0048] S4, using a preset strategy formula to perform reactive calculations on the depth map, generating an expected flight angular velocity of the UAV, and completing autonomous navigation of the UAV based on the expected flight angular velocity and the control method of the finite state machine.

[0049] Specifically, the depth map is reactively calculated to generate the expected angular velocity of the UAV. The main purpose is to make the UAV fly as far away from the direction with smaller depth as possible and fly towards the direction with larger depth. The specific strategy is shown in the following equations (1), (2), and (3):

[0050]

[0051]

[0052]

[0053] Among them, i is the angle corresponding to the pixel point, d i is the depth value of the pixel, φ is the current direction of the drone, which can also be understood as the angle corresponding to the horizontal center of the camera image, Ψ goal is the angle corresponding to the target direction (can be set in advance), λ goto , avoid ,c,σ,wb All are adjustable parameters. The expected angular velocity for obstacle avoidance is obtained by calculation. and the desired angular velocity towards the target direction Weight the two to get the final expected flight angular velocity

[0054] Furthermore, since the information provided by the depth estimation network is somewhat unreliable, it is difficult to provide reliable depth information, especially when the aircraft is facing a direction with sparse texture, such as a white wall. To address this problem, inspired by the exploration strategy given in the literature [9], we introduced a finite state machine control method. When the front of the drone is very close to an obstacle or wall, a follow the wall flight mode is triggered, which avoids the problem of the drone colliding or getting stuck in a corner or obstacle and not knowing how to fly.

[0055] Specifically, the distance measurement data of the TOF sensor in the front, left and right directions of the drone is used to complete the combination of the reactive algorithm + the wall-following strategy. The specific strategies are as follows: Figure 5 As shown, this ensures that the UAV’s flight can be guided when the results given by the reactive algorithm are unreliable.

[0056] According to the autonomous aircraft navigation method based on monocular depth estimation and reactive strategy of an embodiment of the present invention, a convolutional neural network is used to extract image depth features for encoding and decoding to estimate the depth values ​​at different pixel points of the image, and a reactive calculation is performed based on the obtained depth map to obtain the expected angular velocity of the current unmanned aerial vehicle to complete autonomous obstacle avoidance.

[0057] Furthermore, the embodiment of the present invention is deployed on the Crazyflie 2.0 nano-UAV platform. The flight platform is equipped with an optical flow board for positioning the aircraft and a TOF board for sensing the distance in front, behind, left and right. The AIdeck platform is used as the computing platform. AIdeck is mainly composed of the GAP8 chip and Himax camera of GreenWaves Technologies, which expands the computing power and will enable complex workloads based on artificial intelligence to run on nano-unmanned platforms. The GAP8 chip is a commercial embedded RISC-V multi-core processor derived from the PULP open source project, and its core consists of an advanced RISC-V MCU and a programmable eight-core processor. The implementation of the code part in the present invention is based on the Python language, and Pytorch is used as the deep learning framework.

[0058] This example uses the NYU Depth v2 dataset to train, trim the network, and verify the final deployment results. The network structure refers to the FastDepth network structure. The FastDepth network structure is as follows: Figure 2 As shown in the figure, the MobileNetv1 in the front-end Encoder part is replaced with a more lightweight MobileNetv2 network. The back-end multi-layer upsampling layers and cross-layer connections are also added, and the NetAdapt pruning method is used to perform up to 23 rounds of iterative pruning on the network.

[0059] Preferably, the NNTOOL tool is used to quantize the network parameters to 8 bits so that the chip can perform network operations faster. Autotiler is used to complete the deployment of the quantization model on GAP8 and encapsulate it into C code for direct calling. In this example, the parameter settings are to set the chip CL frequency to 175MHz, the FC frequency to 250MHz, the L1 allocated memory to 46736Bytes, and the L2 allocated memory to 250000Bytes.

[0060] Preferably, the specific strategy of the reactive algorithm is as follows: Figure 3 is a schematic diagram of the algorithm. The present invention uses a series of depth values ​​in the horizontal direction of the center of the depth map as the basis for obstacle avoidance. In this project, since the nano-UAV used lacks reliable autonomous positioning capability, a fixed target direction ψ is not set. goal , but let the target direction be equal to the current direction of the drone, that is, ψ goal =φ, so Since the information provided by the depth estimation network has a certain degree of unreliability, it is difficult to provide reliable depth information, especially when the aircraft is facing a direction with sparse texture, such as a white wall. To address this problem, a finite state machine control method is introduced. When the front of the drone is very close to an obstacle or a wall, a follow the wall flight mode is triggered to avoid the drone colliding or getting stuck in a corner or obstacle and not knowing how to fly next. On the basis of depth estimation + reactive algorithm, the present invention incorporates the ranging data of the TOF sensors in the front, left, and right directions of the drone to complete the combination of reactive algorithm + follow the wall strategy. The specific strategy is as follows: Figure 5 As shown. The angular velocity is the turning angular velocity of the drone calculated by the reactive algorithm. When the drone rotates to the left, When the UAV rotates to the right. front ,r left ,r right Respectively represent the distance between the front, left, and right sides of the drone and the obstacle. Th is the set distance threshold. When the distance is less than the threshold, the drone is judged to be close to the obstacle. When no wall or obstacle is detected, the drone moves at an angular velocity. Normally, once a wall is detected in front of the drone, the drone enters the state of following the wall. The drone stops moving and rotates until the nose is parallel to the wall. At this time, the drone moves in the direction of the wall and always keeps a certain distance from the wall. When the drone finds a wall in front of it again, it stops moving and rotates until the nose is parallel to the wall, and then continues to move along the wall. At any time in any state of following the wall, as long as the angular velocity is satisfied: If the distance on one side of the drone corresponding to the turning direction and the distance in front of the drone are both greater than the set threshold Th, the state of following the wall is jumped out and the drone flies forward according to the result given by the reactive algorithm.

[0061] Furthermore, in order to test the effectiveness of the present invention, the embodiments conducted flight tests in four flight experimental scenarios, FIG4 (a) an empty corridor environment; FIG4 (b) an office environment; FIG4 (c) a corridor environment with a chair as an obstacle; and FIG4 (d) a corridor environment with three corners. In the experiment, the nano-UAV can fly forward at a maximum speed of 0.5 m / s. In order to evaluate the effect of the experiment, the maximum flight distance of the UAV in the scene is used as the standard, and the baseline selects a simple direct flight strategy. The experimental results are shown in Table 1. It can be seen that the navigation method of the present invention has a significant effect on the navigation of nano-UAVs.

[0062] Table 1

[0063]

[0064] In summary, the present invention focuses on designing a low-computational complexity, lightweight autonomous navigation method, and deploys the trained depth estimation network and its weights to the GAP8 chip after 8-bit quantization, an ultra-low power edge computing chip. The present invention implements the deployment and operation of a monocular depth estimation network on an ultra-low power computing chip, which is of great significance for nano-type unmanned aerial vehicles that can only use a monocular camera as a perception unit. The reactive navigation algorithm based on depth estimation does not require a lot of computing resources, and is very friendly to low-power, lightweight nano-type unmanned aerial vehicles. After testing, the algorithm proposed by the present invention was successfully deployed on a nano-type unmanned aerial vehicle Crazyflie 2.0 weighing 27g and 10cm in size, realizing the full online autonomous navigation function and conducting actual flight experiments. The experimental results show that the autonomous nano-type unmanned aerial vehicle can (i) fly quickly at a maximum speed of 0.5m / s; (ii) autonomously avoid obstacles in a complex indoor environment; (iii) the computing power consumption for the autonomous navigation function is only 393mW, which is less than 7.2% of the total power of the unmanned aerial vehicle.

[0065] The autonomous navigation method of the aircraft based on monocular depth estimation and reactive strategy according to the embodiment of the present invention has an advantage over the traditional planning algorithm in that it only needs to consider the reaction made by the aircraft in the current frame, so that the unmanned aerial vehicle is as far away from the direction with smaller current depth as possible and faces the direction with larger depth, which greatly reduces the amount of calculation and is suitable for the ultra-low power consumption computing platform of the nano-type unmanned aerial vehicle used in the present invention. The expected angular velocity of navigation is calculated directly according to the input of the monocular camera sensor, realizing the autonomous flight and obstacle avoidance functions of the unmanned aerial vehicle.

[0066] In order to implement the above embodiment, Figure 6 As shown, this embodiment also provides an aircraft autonomous navigation system 10 based on monocular depth estimation and reactive strategy. The system 10 includes a network training module 100, a tensor partitioning module 200, a depth estimation module 300 and a strategy control module 400.

[0067] The network training module 100 is used to obtain a training data set of UAV images, and use the training data set to train a depth estimation network to obtain a trained depth estimation network;

[0068] A tensor partitioning module 200 is used to quantize the network parameters of the trained depth estimation network using multiple images in the training data set to obtain a convolution tensor, and to partition the convolution tensor using an optimal tensor partitioning method to generate network code data;

[0069] The depth estimation module 300 is used to obtain a depth map of the real-time UAV image estimation by network calculation of a trained depth estimation network using the real-time UAV image and network code data;

[0070] The strategy control module 400 is used to perform reactive calculations on the depth map using a preset strategy formula to generate an expected flight angular velocity of the UAV, and complete autonomous navigation of the UAV based on the expected flight angular velocity and the control method of the finite state machine.

[0071] Further, after the above network training module 100, the system 10 further includes:

[0072] The network pruning module is used to prune the trained depth estimation network using the NetAdapt pruning algorithm to filter out a preset number of convolution kernels in the network.

[0073] Furthermore, the policy control module 400 is also used for:

[0074]

[0075]

[0076]

[0077] Among them, i is the angle corresponding to the pixel point, d i is the depth value of the pixel, φ is the current direction of the drone, Ψ goal is the angle corresponding to the target direction, λ goto , avoid ,c,σ,w b are all adjustable parameters; the desired angular velocity calculated by and the desired angular velocity towards the target direction Weighted to get the final desired flight angular velocity

[0078] Furthermore, the policy control module 400 is also used for:

[0079] Obtain the ranging data of the sensors in the front, left and right directions of the UAV;

[0080] According to the ranging data and the expected flight angular velocity, the autonomous navigation of the unmanned aerial vehicle is completed based on the reactive algorithm and the wall-following strategy.

[0081] Furthermore, the tensor partitioning module 200 is also used for:

[0082] The AutoTiler network tool is used to select the optimal tensor partitioning method for convolution tensor partitioning, so as to encapsulate the calculation process of the trained depth estimation network on the ultra-low power edge computing chip into code data expressed in C language.

[0083] The autonomous aircraft navigation system based on monocular depth estimation and reactive strategy according to the embodiment of the present invention has an advantage over the traditional planning algorithm in that it only needs to consider the reaction made by the aircraft in the current frame, so that the unmanned aerial vehicle is as far away from the direction with smaller current depth as possible and faces the direction with larger depth, which greatly reduces the amount of calculation and is suitable for the ultra-low power consumption computing platform of nano-type unmanned aerial vehicles used in the present invention. The expected angular velocity of navigation is calculated directly according to the input of the monocular camera sensor, realizing the autonomous flight and obstacle avoidance functions of the unmanned aerial vehicle.

[0084] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0085] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

Claims

1. An autonomous navigation method for an aircraft based on monocular depth estimation and reactive strategy, characterized in that: The following steps are involved: Acquire a training data set of unmanned aerial vehicle images, and use the training data set to train a depth estimation network to obtain a trained depth estimation network; Using multiple images in the training data set to quantize the network parameters of the trained depth estimation network to obtain a convolution tensor, and using an optimal tensor partitioning method to partition the convolution tensor to generate network code data; Using the real-time unmanned aerial vehicle image and the network code data, a depth map of the real-time unmanned aerial vehicle image estimation is obtained through network calculation of the trained depth estimation network; Reactively calculating the depth map using a preset strategy formula to generate an expected flight angular velocity of the unmanned aerial vehicle, and completing autonomous navigation of the unmanned aerial vehicle based on the expected flight angular velocity and a control method of a finite state machine; The method of using a preset strategy formula to perform reactive calculation on the depth map to generate an expected flight angular velocity of the unmanned aerial vehicle includes: in, is the angle corresponding to the pixel point, is the depth value of the pixel, is the current direction of the drone, is the angle corresponding to the target direction, , , , , All are adjustable parameters; The desired angular velocity is calculated by and the desired angular velocity towards the target direction Weighted to get the final desired flight angular velocity .

2. The method according to claim 1, characterized in that After obtaining the trained depth estimation network, the method further includes: The NetAdapt pruning algorithm is used to prune the trained depth estimation network to filter out a preset number of convolution kernels in the network.

3. The method according to claim 1, characterized in that The control method based on the expected flight angular velocity and the finite state machine to complete the autonomous navigation of the unmanned aerial vehicle includes: Obtain the ranging data of the sensors in the front, left and right directions of the UAV; According to the distance measurement data and the expected flight angular velocity, autonomous navigation of the unmanned aerial vehicle is completed based on a reactive algorithm and a wall-following strategy.

4. The method according to claim 1, characterized in that: The step of dividing the convolution tensor by using an optimal tensor division method to generate network code data includes: The AutoTiler network tool is used to select the optimal tensor partitioning method to partition the convolution tensor, so as to encapsulate the calculation process of the trained depth estimation network on the ultra-low power edge computing chip into code data represented by C language.

5. An autonomous aircraft navigation system based on monocular depth estimation and reactive strategy, characterized in that: include: A network training module is used to obtain a training data set of unmanned aerial vehicle images, and use the training data set to train a depth estimation network to obtain a trained depth estimation network; A tensor partitioning module, used to quantize the network parameters of the trained depth estimation network using multiple images in the training data set to obtain a convolution tensor, and to partition the convolution tensor using an optimal tensor partitioning method to generate network code data; A depth estimation module, for obtaining a depth map of the real-time unmanned aerial vehicle image estimation by network calculation of the trained depth estimation network using the real-time unmanned aerial vehicle image and the network code data; A strategy control module, configured to perform reactive calculations on the depth map using a preset strategy formula to generate an expected flight angular velocity of the unmanned aerial vehicle, and complete autonomous navigation of the unmanned aerial vehicle based on the expected flight angular velocity and a control method of a finite state machine; The policy control module is further used for: in, is the angle corresponding to the pixel point, is the depth value of the pixel, is the current direction of the drone, is the angle corresponding to the target direction, , , , , All are adjustable parameters; The desired angular velocity is calculated by and the desired angular velocity towards the target direction Weighted to get the final desired flight angular velocity .

6. The system according to claim 5, characterized in that After the network training module, the system further includes: The network pruning module is used to prune the trained depth estimation network using the NetAdapt pruning algorithm to filter out a preset number of convolution kernels in the network.

7. The system according to claim 5, characterized in that The policy control module is further used for: Obtain the ranging data of the sensors in the front, left and right directions of the UAV; According to the distance measurement data and the expected flight angular velocity, autonomous navigation of the unmanned aerial vehicle is completed based on a reactive algorithm and a wall-following strategy.

8. The system according to claim 5, characterized in that The tensor partitioning module is further used for: The AutoTiler network tool is used to select the optimal tensor partitioning method to partition the convolution tensor, so as to encapsulate the calculation process of the trained depth estimation network on the ultra-low power edge computing chip into code data represented by C language.

Citation Information

Patent Citations

  • Flight device, and flight control system and method

    CN105346706A

  • Multi-rotor aircraft safety navigation control method based on depth image

    CN111638727A

  • Unmanned aerial vehicle control method for autonomous target tracking

    CN112506210A

  • End-side collaborative multi-unmanned aerial vehicle autonomous navigation method

    CN114061589A

  • Unmanned aerial vehicle autonomous obstacle avoidance system and method based on deep reinforcement learning

    CN114326821A