Closed space unmanned aerial vehicle intelligent navigation method and system based on three-dimensional reconstruction

Abstract

The invention discloses a closed space unmanned aerial vehicle intelligent navigation method and system based on three-dimensional reconstruction, and the method comprises the steps: S1, scanning a closed space based on a sensor, and obtaining environment data in the closed space; s2, constructing a three-dimensional model of the closed space by utilizing a three-dimensional reconstruction technology according to the acquired environment data; s3, according to the constructed three-dimensional model, planning a flight path of the unmanned aerial vehicle based on an A * algorithm; s4, monitoring the change of the environmental data in real time in the flight process, and adjusting the flight path according to the change; and S5, matching the real-time position of the unmanned aerial vehicle with the three-dimensional model to realize accurate navigation. According to the method, the three-dimensional model of the closed space can be constructed based on the three-dimensional reconstruction technology, the unmanned aerial vehicle can obtain more accurate space information, so that accurate navigation is realized, the change of environmental data is monitored in real time, the flight path is adjusted according to the change, an improved fuzzy obstacle avoidance algorithm and a backstepping flight control algorithm are combined, obstacles are effectively avoided, and the unmanned aerial vehicle is more accurate in navigation. And the flight safety is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) intelligent navigation, and more specifically to a method and system for unmanned aerial vehicle (UAV) intelligent navigation in a confined space based on three-dimensional reconstruction. Background Art

[0002] A drone is an unmanned aircraft that can navigate and fly autonomously. It has advanced sensor components and can be remotely controlled by a pilot on the ground or controlled by the flight control system on the drone. In recent years, with the development of science and technology and the advent of the Internet era, drone technology has been increasingly used in our lives, whether for military purposes at the national level or for personal entertainment purposes. And with the continuous development of drone technology, drone technology will be used in more occasions.

[0003] With the rapid development of drone technology, the application areas of drones are also expanding rapidly. For example, drones can be used not only for conventional high-altitude photography, but also for environmental surveys during disaster relief. However, the latter application scenarios are usually accompanied by complex environmental conditions, such as obstructions by trees and buildings.

[0004] The publication number is CN105865439A, and the name is a drone navigation method, which includes calling a first navigation map; determining a reference point on the first navigation map, and obtaining the first coordinates of the reference point in the first navigation map; using a drone to obtain the second coordinates of the corresponding real geographical location of the reference point; correcting multiple navigation coordinates of a preset area in the first navigation map according to the vector difference between the first coordinate and the second coordinate, and navigating the drone using the corrected multiple navigation coordinates.

[0005] The publication number is CN115164901A, and the name is a drone navigation method, which includes: after determining that the satellite navigation signal is interrupted, determining the last positioning data when the satellite navigation signal is not interrupted; continuously detecting whether there are new obstacles within a preset range of the drone; navigating the drone according to the positioning data and the detection result of whether there are new obstacles; and repeating the step of navigating the drone according to the positioning data and the detection result of whether there are new obstacles each time a preset condition is met until the target position is reached, or the satellite navigation signal is restored.

[0006] During flight, drones need to use satellite navigation signals to correct their position, attitude, and speed. Once the satellite navigation signals are blocked by obstructions, the drone’s attitude may shift and it may be unable to locate itself, causing it to crash.

[0007] Therefore, how to provide a drone navigation method that can autonomously and intelligently navigate and avoid obstacles in a confined space is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0008] In view of this, the present invention provides an intelligent navigation method and system for unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction, aiming to solve the technical problem that during the flight, the position, attitude and speed need to be corrected with the help of satellite navigation signals. Once the satellite navigation signal is blocked by an obstruction, it may cause the unmanned aerial vehicle attitude to shift and become unable to be positioned, resulting in a crash.

[0009] In order to achieve the above object, the present invention adopts the following technical solution:

[0010] A confined space UAV intelligent navigation method based on three-dimensional reconstruction includes the following steps:

[0011] S1. Scan the confined space based on the sensor to obtain environmental data in the confined space;

[0012] S2. Construct a three-dimensional model of the confined space using three-dimensional reconstruction technology based on the acquired environmental data;

[0013] S3, based on the constructed 3D model, plan the flight path of the UAV based on the A* algorithm;

[0014] S4. During the flight, monitor the changes in environmental data in real time and adjust the flight path according to the changes;

[0015] S5. Match the real-time position of the drone with the three-dimensional model to achieve precise navigation.

[0016] Furthermore, the step of scanning the confined space based on the sensor in S1 to obtain the environmental data in the space is as follows:

[0017] The acceleration force and tilt angle of the drone in a stationary state are obtained based on the accelerometer, the angular velocity of the three axes of the drone is obtained based on the gyroscope sensor to monitor the rate of change of the angle of the drone during pitch, roll and yaw, the flight altitude of the drone is obtained based on the barometer, and multi-perspective image data is obtained based on the camera device.

[0018] Furthermore, in S2, the three-dimensional model of the confined space is constructed using three-dimensional reconstruction technology according to the acquired environmental data, specifically:

[0019] Based on the feature detection algorithm, key points in the image data are detected, and descriptors are calculated for matching. Corresponding feature points are found in images of different perspectives. The feature point matching results are used to estimate the camera posture through the PnP algorithm to determine the position and direction of each camera device relative to the scene. The three-dimensional coordinates are calculated based on the matched feature points and camera posture using triangulation, and the preliminary three-dimensional structure of the scene is constructed. The Bundle Adjustment technology is applied to optimize the camera posture and the position of the three-dimensional points to improve the accuracy and robustness of the reconstruction.

[0020] Use stereo matching algorithm to obtain the disparity between the reference image and the candidate image, merge the depth information of multiple views through the aggregation strategy in COLMAP, integrate the disparity information of multiple views, improve the accuracy of depth estimation, apply filtering and optimization methods to improve the quality of depth map, and remove erroneous and discontinuous depth values;

[0021] Use point cloud fusion technology to fuse multiple depth maps into a dense point cloud, use Delaunay triangulation to build the point cloud into a triangular mesh model, use UV unfolding technology to map the texture to the triangular mesh, map the color information from the original image to the 3D model, apply the Laplacian Smoothing algorithm to remove noise, and use the MeshSimplification algorithm to simplify the 3D model.

[0022] Furthermore, in S3, the flight path of the drone is planned based on the constructed three-dimensional model and the A* algorithm as follows:

[0023] S31. Build A* algorithm model:

[0024] f(n)=g(n)+h(n)

[0025] Among them, f(n) is the heuristic function corresponding to each path node when searching, g(n) is the actual travel cost from the starting path node to the current path node, and h(n) is the estimated value of the travel cost from the current path node to the end point of the path;

[0026] S32, constructing a set of nodes P that have not been added to the optimal path tree and a set of nodes Q that have been added to the optimal path tree;

[0027] S33, set the P set and Q set to empty, add the starting path node S to the P set, set its g value to 0, set the parent node to empty, and set the g values ​​of other path nodes in the road network to infinity;

[0028] S34, if the P set is empty, the A* algorithm terminates. Otherwise, select the path node with the smallest f value in the P set, record it as BT, add it to the Q set, and determine whether BT is the end point T. If so, proceed to S35. Otherwise, find the NT of each adjacent node of BT based on the road network topology attributes and traffic rules, and perform the following steps:

[0029] Calculate the heuristic value of NT:

[0030] f(NT)=g(NT)+h(NT)

[0031] g(NT)=g(BT)+cost(BT,NT)

[0032] In the formula, cost(BT,NT) is the travel cost from BT to NT;

[0033] If NT is in the P set and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value and set the parent node of NT to BT;

[0034] If NT is in the Q set, and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value, set the parent node of NT to BT, and move NT into the P set;

[0035] If NT is neither in the P set nor in the Q set, set NT's parent node to BT and move NT to the P list;

[0036] loop S34;

[0037] S35. Backtrack from the end point of the path, find the parent nodes one by one, and add them to the optimized path until the path starting point S is reached to obtain the optimal path.

[0038] Furthermore, the real-time monitoring of changes in environmental data in S4 and adjusting the flight path according to the changes are specifically as follows:

[0039] Obstacle distance information and flying object angle information are collected during flight, and the obstacle distance information is pre-processed to determine whether there are obstacles. If so, local path planning is performed based on the improved fuzzy obstacle avoidance algorithm, and point-to-point flight is performed based on the backstepping flight control algorithm to avoid obstacles and return to the subsequent scheduled optimal flight path.

[0040] Furthermore, the local path planning based on the improved fuzzy obstacle avoidance algorithm is as follows:

[0041] Initialize the activity values ​​of all neurons in the neural network to zero, and propagate the positive neuron activity values ​​at the target point to the starting point through the local side connections between neurons based on the following formula:

[0042]

[0043] In the formula, x i (t) is the activity value of the ith neuron at time t, N is the number of neurons in the ith neuron area, ω is the degree to which the activity value of the ith neuron at the current moment is affected by its state at the previous moment, ω ij x j (t)+ωx i (t) is the stimulus input of the i-th neuron at time t, I i is the inhibitory input of the i-th neuron at time t;

[0044] ω ij is the connection weight from the jth neuron to the i-th neuron:

[0045]

[0046] In the formula, |q i -q j | is the vector q in the structure space i and vector q j The Euclidian distance between them, η and r are constants, ω ij =ω ji and;

[0047]

[0048] In the formula, I i is the external input of the i-th neuron, which is generated by mapping the position information of the target point and the obstacle in the neural network topology. The neurons corresponding to the obstacle area have negative external inputs, and the target point neurons have positive external inputs. The parameter E determines the external input values ​​of the neurons corresponding to the obstacle area and the target point.

[0049] In a dynamic environment, the external input of neurons is generated according to the mapping of the location information of the target point and obstacles in the neural network topology structure;

[0050] Using equations (1) and (3), the neuron activity values ​​at the target point are propagated throughout the entire workspace through the local side connections between neurons.

[0051] The hill climbing method is used to search for the neuron with the largest activity value in the neighborhood of the current position. If the activity values ​​of the neurons in the neighborhood are not greater than the activity value of the current neuron, the drone remains in its original position; otherwise, the neuron at the next position is the neuron with the largest activity value in the neighborhood.

[0052] If the UAV reaches the target point, the path planning process ends, otherwise the process is looped again.

[0053] An intelligent navigation system for unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction, comprising:

[0054] A three-dimensional reconstruction module, used for three-dimensional modeling of a confined space, the three-dimensional reconstruction module comprising a depth camera and an image processing unit, the depth camera is used to capture image data of the confined space, and the image processing unit is used to process the image data captured by the depth camera to generate a three-dimensional point cloud;

[0055] A path planning module, used to generate a navigation path for the UAV according to the three-dimensional model, the path planning module includes a path search algorithm and a path optimization algorithm, the path search algorithm is used to search for an optimal path in the three-dimensional model, and the path optimization algorithm is used to optimize the searched path according to the flight characteristics of the UAV;

[0056] A sensor module, used to obtain the position information of the drone in a confined space in real time, the sensor module includes an inertial measurement unit and a global positioning system receiver, the inertial measurement unit is used to measure the acceleration and angular velocity of the drone, and the global positioning system receiver is used to obtain the position information of the drone in an outdoor environment;

[0057] The control module is used to adjust the flight path of the UAV according to the position information obtained by the sensor module. The control module includes a flight control algorithm and a dynamic adjustment mechanism. The flight control algorithm is used to control the flight of the UAV according to the path generated by the path planning module. The dynamic adjustment mechanism is used to dynamically adjust the flight path of the UAV when encountering obstacles.

[0058] Furthermore, the 3D reconstruction module further includes a feature detection unit and a 3D reconstruction algorithm unit; wherein the feature detection unit is used to detect key points from the image data captured by the depth camera and calculate descriptors for matching, and the 3D reconstruction algorithm unit is used to calculate the 3D coordinates through triangulation based on the matched feature points and camera posture, construct a preliminary 3D structure of the scene, and apply the Bundle Adjustment technology to optimize the camera posture and the position of the 3D points.

[0059] Furthermore, the path search algorithm in the path planning module is algorithm A, which is used to evaluate the actual travel cost from the starting node to the current node and the estimated value of the travel cost from the current node to the end point; the path optimization algorithm dynamically adjusts the searched path based on the flight characteristics of the drone, including speed, acceleration and steering angle limits.

[0060] Furthermore, the sensor module also includes an obstacle detection sensor for monitoring obstacle information around the drone in real time; and the dynamic adjustment mechanism in the control module includes an improved fuzzy obstacle avoidance algorithm for performing local path planning when an obstacle is detected, and a backstepping flight control algorithm for performing point-to-point flight, avoiding obstacles, and returning to a predetermined optimal flight path.

[0061] The above technical solution includes at least the following technical effects:

[0062] Through the above technical solutions, it can be known that the present invention discloses a method and system for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction. Compared with the prior art, the present invention can construct a three-dimensional model of a confined space based on three-dimensional reconstruction technology, and the unmanned aerial vehicle can obtain more accurate spatial information, thereby realizing precise navigation, monitoring changes in environmental data in real time, and adjusting the flight path according to the changes. In combination with an improved fuzzy obstacle avoidance algorithm and a backstepping flight control algorithm, obstacles can be effectively avoided, and flight safety is improved. The flight path is planned using the A* algorithm, and the path is optimized according to the flight characteristics of the unmanned aerial vehicle, thereby improving the efficiency and adaptability of path planning. The camera posture and the position of three-dimensional points are optimized through the Bundle Adjustment technology, thereby improving the accuracy and robustness of three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0064] Figure 1 The present invention is a flow chart of a method for intelligent navigation of unmanned aerial vehicles in a confined space based on three-dimensional reconstruction. DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present invention are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] See attached Figure 1 The embodiment of the present invention discloses an intelligent navigation method for a UAV in a confined space based on three-dimensional reconstruction.

[0067] S1. Scan the confined space based on the sensor to obtain the environmental data in the confined space, obtain the acceleration force of the drone and the tilt angle when it is stationary based on the accelerometer, obtain the angular velocity of the three axes of the drone based on the gyroscope sensor to monitor the rate of change of the angle of the drone during pitch, roll and yaw, obtain the flight altitude of the drone based on the barometer, and obtain multi-view image data based on the camera device.

[0068] S2. Based on the acquired environmental data, a 3D model of the confined space is constructed using 3D reconstruction technology; key points in the image data are detected based on the feature detection algorithm, and descriptors are calculated for matching, and corresponding feature points are found in images of different perspectives. The camera posture is estimated through the PnP algorithm using the feature point matching results, and the position and direction of each camera device relative to the scene are determined. The 3D coordinates are calculated based on the matched feature points and camera posture using triangulation, and the preliminary 3D structure of the scene is constructed. The Bundle Adjustment technology is applied to optimize the camera posture and the position of the 3D points to improve the accuracy and robustness of the reconstruction;

[0069] Use stereo matching algorithm to obtain the disparity between the reference image and the candidate image, merge the depth information of multiple views through the aggregation strategy in COLMAP, integrate the disparity information of multiple views, improve the accuracy of depth estimation, apply filtering and optimization methods to improve the quality of depth map, and remove erroneous and discontinuous depth values;

[0070] Use point cloud fusion technology to fuse multiple depth maps into a dense point cloud, use Delaunay triangulation to build the point cloud into a triangular mesh model, use UV unfolding technology to map the texture to the triangular mesh, map the color information from the original image to the 3D model, apply the Laplacian Smoothing algorithm to remove noise, and use the MeshSimplification algorithm to simplify the 3D model.

[0071] S3, based on the constructed 3D model, plan the flight path of the UAV based on the A* algorithm;

[0072] The specific flight path of the drone based on the A* algorithm is:

[0073] S31. Build A* algorithm model:

[0074] f(n)=g(n)+h(n)

[0075] Among them, f(n) is the heuristic function corresponding to each path node when searching, g(n) is the actual travel cost from the starting path node to the current path node, and h(n) is the estimated value of the travel cost from the current path node to the end point of the path;

[0076] S32, constructing a set of nodes P that have not been added to the optimal path tree and a set of nodes Q that have been added to the optimal path tree;

[0077] S33, set the P set and Q set to empty, add the starting path node S to the P set, set its g value to 0, set the parent node to empty, and set the g values ​​of other path nodes in the road network to infinity;

[0078] S34, if the P set is empty, the A* algorithm terminates. Otherwise, select the path node with the smallest f value in the P set, record it as BT, add it to the Q set, and determine whether BT is the end point T. If so, proceed to S35. Otherwise, find the NT of each adjacent node of BT based on the road network topology attributes and traffic rules, and perform the following steps:

[0079] Calculate the heuristic value of NT:

[0080] f(NT)=g(NT)+h(NT)

[0081] g(NT)=g(BT)+cost(BT,NT)

[0082] In the formula, cost(BT,NT) is the travel cost from BT to NT;

[0083] If NT is in the P set and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value and set the parent node of NT to BT;

[0084] If NT is in the Q set, and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value, set the parent node of NT to BT, and move NT into the P set;

[0085] If NT is neither in the P set nor in the Q set, set NT's parent node to BT and move NT to the P list;

[0086] loop S34;

[0087] S35. Backtrack from the end point of the path, find the parent nodes one by one, and add them to the optimized path until the path starting point S is reached to obtain the optimal path.

[0088] S4. During the flight, monitor the changes in environmental data in real time and adjust the flight path according to the changes;

[0089] Obstacle distance information and flying object angle information are collected during flight, and the obstacle distance information is pre-processed to determine whether there are obstacles. If so, local path planning is performed based on the improved fuzzy obstacle avoidance algorithm, and point-to-point flight is performed based on the backstepping flight control algorithm to avoid obstacles and return to the subsequent scheduled optimal flight path.

[0090] Initialize the activity values ​​of all neurons in the neural network to zero, and propagate the positive neuron activity values ​​at the target point to the starting point through the local side connections between neurons based on the following formula:

[0091]

[0092] In the formula, x i (t) is the activity value of the ith neuron at time t, N is the number of neurons in the ith neuron area, ω is the degree to which the activity value of the ith neuron at the current moment is affected by its state at the previous moment, ω ij x j (t)+ωx i (t) is the stimulus input of the i-th neuron at time t, I i is the inhibitory input of the i-th neuron at time t;

[0093] ω ij is the connection weight from the jth neuron to the i-th neuron:

[0094]

[0095] In the formula, |q i -q j | is the vector q in the structure space i and vector q j The Euclidian distance between them, η and r are constants, ω ij =ω ji and;

[0096]

[0097] In the formula, I i is the external input of the i-th neuron, which is generated by mapping the position information of the target point and the obstacle in the neural network topology. The neurons corresponding to the obstacle area have negative external inputs, and the target point neurons have positive external inputs. The parameter E determines the external input values ​​of the neurons corresponding to the obstacle area and the target point.

[0098] In a dynamic environment, the external input of neurons is generated according to the mapping of the location information of the target point and obstacles in the neural network topology structure;

[0099] Using equations (1) and (3), the neuron activity values ​​at the target point are propagated throughout the entire workspace through the local side connections between neurons.

[0100] The hill climbing method is used to search for the neuron with the largest activity value in the neighborhood of the current position. If the activity values ​​of the neurons in the neighborhood are not greater than the activity value of the current neuron, the drone remains in its original position; otherwise, the neuron at the next position is the neuron with the largest activity value in the neighborhood.

[0101] If the UAV reaches the target point, the path planning process ends, otherwise the process is looped again.

[0102] S5. Match the real-time position of the drone with the three-dimensional model to achieve precise navigation.

[0103] An intelligent navigation system for unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction, comprising:

[0104] The 3D reconstruction module is used for performing 3D modeling of a confined space. The 3D reconstruction module of the invention includes a depth camera and an image processing unit. The depth camera of the invention is used to capture image data of the confined space, and the image processing unit of the invention is used to process the image data captured by the depth camera to generate a 3D point cloud; the 3D reconstruction module of the invention further includes a feature detection unit and a 3D reconstruction algorithm unit; wherein the feature detection unit is used to detect key points from the image data captured by the depth camera and calculate descriptors for matching, and the 3D reconstruction algorithm unit is used to calculate the 3D coordinates by triangulation based on the matched feature points and camera pose, to construct a preliminary 3D structure of the scene, and to optimize the camera pose and the position of the 3D points using the Bundle Adjustment technology.

[0105] The path planning module is used to generate a navigation path for the UAV based on a three-dimensional model. The path planning module of the invention includes a path search algorithm and a path optimization algorithm. The path search algorithm of the invention is used to search for the optimal path in the three-dimensional model, and the path optimization algorithm of the invention is used to optimize the searched path according to the flight characteristics of the UAV; the path search algorithm in the path planning module of the invention is an A algorithm, which is used to evaluate the actual travel cost from the starting node to the current node and the estimated value of the travel cost from the current node to the end point; the path optimization algorithm of the invention dynamically adjusts the searched path based on the flight characteristics of the UAV, including speed, acceleration and steering angle limitations.

[0106] The invention discloses a sensor module for obtaining the position information of the UAV in a confined space in real time. The sensor module includes an inertial measurement unit and a global positioning system receiver. The inertial measurement unit is used to measure the acceleration and angular velocity of the UAV, and the global positioning system receiver is used to obtain the position information of the UAV in an outdoor environment. The invention sensor module also includes an obstacle detection sensor for monitoring the obstacle information around the UAV in real time. Moreover, the dynamic adjustment mechanism in the invention control module includes an improved fuzzy obstacle avoidance algorithm for performing local path planning when an obstacle is detected, and a backstepping flight control algorithm for performing point-to-point flight, avoiding obstacles, and returning to a predetermined optimal flight path.

[0107] The invention discloses a control module, which is used to adjust the flight path of the UAV according to the position information obtained by the sensor module. The invention discloses a control module, which includes a flight control algorithm and a dynamic adjustment mechanism. The invention discloses a flight control algorithm, which is used to control the flight of the UAV according to the path generated by the path planning module. The invention discloses a dynamic adjustment mechanism, which is used to dynamically adjust the flight path of the UAV when encountering obstacles.

[0108] Step 1: Obtain environmental data

[0109] Sensor Configuration:

[0110] Accelerometer: It is placed near the center of gravity of the drone to reduce the impact of vibration on the measurement.

[0111] Gyro sensor: Installed on the drone's stable platform to obtain precise angular velocity data.

[0112] Barometer: Installed on the bottom of the drone to obtain flight altitude data.

[0113] Camera device: Equipped with multiple high-resolution cameras, evenly distributed around the drone to obtain all-round image data.

[0114] Data synchronization: Ensure that the timestamps of all sensor data are synchronized so that they can be accurately matched during subsequent processing.

[0115] Step 2: 3D model construction

[0116] Feature detection and matching:

[0117] Detect keypoints in the image using algorithms like SIFT, SURF or ORB.

[0118] Compute descriptors of key points and perform feature matching in images from different viewpoints.

[0119] Camera pose estimation:

[0120] Using the matched feature points, the pose of each camera is estimated using the PnP algorithm.

[0121] 3D coordinate calculation:

[0122] Based on the matched feature points and camera pose, the 3D coordinates of each feature point are calculated using triangulation.

[0123] Bundle Adjustment Technology:

[0124] Bundle Adjustment technology is applied to optimize the camera posture and the position of 3D points to improve the accuracy of reconstruction.

[0125] Deep Information Fusion:

[0126] The disparity map is calculated using a stereo matching algorithm, and the depth information of multiple views is merged through an aggregation strategy.

[0127] Point cloud fusion and triangulation:

[0128] Multiple depth maps are fused into a dense point cloud and a triangular mesh model is constructed using Delaunay triangulation.

[0129] Texture Mapping and Simplification:

[0130] Map the color information from the original image onto the triangle mesh using UV unwrapping techniques.

[0131] The Laplacian Smoothing algorithm was applied to remove noise, and the Mesh Simplification algorithm was used to simplify the model.

[0132] Step 3: Flight Path Planning

[0133] A* algorithm model construction:

[0134] Define a heuristic function to evaluate the actual cost from the start node to the current node and the estimated cost from the current node to the end node.

[0135] Path search and optimization:

[0136] Use the A* algorithm to search for the optimal path in a 3D model.

[0137] The path is dynamically adjusted based on the drone's flight characteristics, such as speed, acceleration, and steering angle limits.

[0138] Step 4: Real-time monitoring and adjustment

[0139] Environmental data monitoring:

[0140] Collect obstacle distance information and flying object angle information in real time.

[0141] Pre-process the obstacle distance information to determine whether there is an obstacle.

[0142] Obstacle avoidance algorithm application:

[0143] If an obstacle is detected, local path planning is performed based on the improved fuzzy obstacle avoidance algorithm.

[0144] Use a backstepping flight control algorithm to perform point-to-point flight, avoid obstacles, and return to a predetermined optimal flight path.

[0145] Step 5: Precise Navigation

[0146] Real-time location matching:

[0147] Match the drone’s real-time position with the 3D model for precise navigation.

[0148] Use sensor modules such as inertial measurement units and global positioning system receivers to obtain real-time location information of the drone.

[0149] According to the invention, a confined space UAV intelligent navigation method and system based on 3D reconstruction is designed. By constructing a 3D model of the confined space based on 3D reconstruction technology, the UAV can obtain more accurate spatial information, thereby achieving precise navigation, real-time monitoring of changes in environmental data, and adjusting the flight path according to the changes. By combining the improved fuzzy obstacle avoidance algorithm and the backstepping flight control algorithm, obstacles can be effectively avoided, flight safety is improved, the flight path is planned using the A* algorithm, and the path is optimized according to the flight characteristics of the UAV, the efficiency and adaptability of path planning are improved, and the camera posture and the position of the 3D points are optimized by the Bundle Adjustment technology, and the accuracy and robustness of 3D reconstruction are improved. .

[0150] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0151] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction, characterized in that: The following steps are involved: S1. Scan the confined space based on the sensor to obtain environmental data in the confined space; S2. Construct a three-dimensional model of the confined space using three-dimensional reconstruction technology based on the acquired environmental data; S3, based on the constructed 3D model, plan the flight path of the UAV based on the A* algorithm; S4. During the flight, monitor the changes in environmental data in real time and adjust the flight path according to the changes; S5. Match the real-time position of the drone with the three-dimensional model to achieve precise navigation.

2. The method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction according to claim 1, characterized in that: In S1, the confined space is scanned based on the sensor to obtain the environmental data in the space: The acceleration force and tilt angle of the drone in a stationary state are obtained based on the accelerometer, the angular velocity of the three axes of the drone is obtained based on the gyroscope sensor to monitor the rate of change of the angle of the drone during pitch, roll and yaw, the flight altitude of the drone is obtained based on the barometer, and multi-perspective image data is obtained based on the camera device.

3. The method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction according to claim 1, characterized in that: In S2, based on the acquired environmental data, a three-dimensional model of the confined space is constructed using three-dimensional reconstruction technology, specifically: Based on the feature detection algorithm, key points in the image data are detected, and descriptors are calculated for matching. Corresponding feature points are found in images of different perspectives. The feature point matching results are used to estimate the camera posture through the PnP algorithm to determine the position and direction of each camera device relative to the scene. The three-dimensional coordinates are calculated based on the matched feature points and camera posture using triangulation, and the preliminary three-dimensional structure of the scene is constructed. The Bundle Adjustment technology is applied to optimize the camera posture and the position of the three-dimensional points. Use stereo matching algorithm to obtain the disparity between the reference image and the candidate image, merge the depth information of multiple views through the aggregation strategy in COLMAP, integrate the disparity information of multiple views to improve the accuracy of depth estimation, and apply filtering and optimization methods to remove erroneous and discontinuous depth values; Use point cloud fusion technology to fuse multiple depth maps into a dense point cloud, use Delaunay triangulation to build the point cloud into a triangular mesh model, use UV unfolding technology to map the texture to the triangular mesh, map the color information from the original image to the 3D model, apply the Laplacian Smoothing algorithm to remove noise, and use the MeshSimplification algorithm to simplify the 3D model.

4. The method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction according to claim 1, characterized in that: In S3, the flight path of the drone is planned based on the constructed three-dimensional model and the A* algorithm as follows: S31. Build A* algorithm model: f(n)=g(n)+h(n) Among them, f(n) is the heuristic function corresponding to each path node when searching, g(n) is the actual travel cost from the starting path node to the current path node, and h(n) is the estimated value of the travel cost from the current path node to the end point of the path; S32, constructing a set of nodes P that have not been added to the optimal path tree and a set of nodes Q that have been added to the optimal path tree; S33, set the P set and Q set to empty, add the starting path node S to the P set, set its g value to 0, set the parent node to empty, and set the g values ​​of other path nodes in the road network to infinity; S34, if the P set is empty, the A* algorithm terminates. Otherwise, select the path node with the smallest f value in the P set, record it as BT, add it to the Q set, and determine whether BT is the end point T. If so, proceed to S35. Otherwise, find each adjacent node NT of BT based on the road network topology attributes and traffic rules, and perform the following steps: Calculate the heuristic value of NT: f(NT)=g(NT)+h(NT) g(NT)=g(BT)+cost(BT,NT) In the formula, cost(BT,NT) is the travel cost from BT to NT; If NT is in the P set and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value and set the parent node of NT to BT; If NT is in the Q set, and the calculated g value is smaller than the original g value of NT, then update the g value of NT to the current value, set the parent node of NT to BT, and move NT into the P set; If NT is neither in the P set nor in the Q set, set NT's parent node to BT and move NT to the P list; loop S34; S35. Backtrack from the end point of the path, find the parent nodes one by one, and add them to the optimized path until the path starting point S is reached to obtain the optimal path.

5. The method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction according to claim 1, characterized in that: The real-time monitoring of changes in environmental data in S4 and adjusting the flight path according to the changes are specifically as follows: Obstacle distance information and flying object angle information are collected during flight, and the obstacle distance information is pre-processed to determine whether there are obstacles. If so, local path planning is performed based on the improved fuzzy obstacle avoidance algorithm, and point-to-point flight is performed based on the backstepping flight control algorithm to avoid obstacles and return to the subsequent scheduled optimal flight path.

6. The method for intelligent navigation of unmanned aerial vehicles in confined spaces based on three-dimensional reconstruction according to claim 5, characterized in that: The local path planning based on the improved fuzzy obstacle avoidance algorithm is as follows: Initialize the activity values ​​of all neurons in the neural network to zero, and propagate the positive neuron activity values ​​at the target point to the starting point through the local side connections between neurons based on the following formula: In the formula, x i (t) is the activity value of the ith neuron at time t, N is the number of neurons in the ith neuron area, ω is the degree to which the activity value of the ith neuron at the current moment is affected by its state at the previous moment, ω ij x j (t)+ωx i (t) is the stimulus input of the i-th neuron at time t, I i is the inhibitory input of the i-th neuron at time t; ω ij is the connection weight from the jth neuron to the i-th neuron: In the formula, |q i -q j | is the vector q in the structure space i and vector q j The Euclidian distance between them, η and r are constants, ω ij =ω ji and; In the formula, I i is the external input of the i-th neuron, which is generated by mapping the position information of the target point and the obstacle in the neural network topology. The neurons corresponding to the obstacle area have negative external inputs, and the target point neurons have positive external inputs. The parameter E determines the external input values ​​of the neurons corresponding to the obstacle area and the target point. In a dynamic environment, the external input of neurons is generated according to the mapping of the location information of the target point and obstacles in the neural network topology structure; Using equations (1) and (3), the neuron activity values ​​at the target point are propagated throughout the entire workspace through the local side connections between neurons. The hill climbing method is used to search for the neuron with the largest activity value in the neighborhood of the current position. If the activity values ​​of the neurons in the neighborhood are not greater than the activity value of the current neuron, the drone remains in its original position; otherwise, the neuron at the next position is the neuron with the largest activity value in the neighborhood. If the UAV reaches the target point, the path planning process ends, otherwise the process is looped again.

7. A confined space UAV intelligent navigation system based on three-dimensional reconstruction, based on any one of claims 1-6 of the confined space UAV intelligent navigation method based on three-dimensional reconstruction, characterized in that: include: A three-dimensional reconstruction module, used for three-dimensional modeling of a confined space, the three-dimensional reconstruction module comprising a depth camera and an image processing unit, the depth camera is used to capture image data of the confined space, and the image processing unit is used to process the image data captured by the depth camera to generate a three-dimensional point cloud; A path planning module, used to generate a navigation path for the UAV according to the three-dimensional model, the path planning module includes a path search algorithm and a path optimization algorithm, the path search algorithm is used to search for an optimal path in the three-dimensional model, and the path optimization algorithm is used to optimize the searched path according to the flight characteristics of the UAV; A sensor module, used to obtain the position information of the drone in a confined space in real time, the sensor module includes an inertial measurement unit and a global positioning system receiver, the inertial measurement unit is used to measure the acceleration and angular velocity of the drone, and the global positioning system receiver is used to obtain the position information of the drone in an outdoor environment; The control module is used to adjust the flight path of the UAV according to the position information obtained by the sensor module. The control module includes a flight control algorithm and a dynamic adjustment mechanism. The flight control algorithm is used to control the flight of the UAV according to the path generated by the path planning module. The dynamic adjustment mechanism is used to dynamically adjust the flight path of the UAV when encountering obstacles.

8. The confined space UAV intelligent navigation system based on three-dimensional reconstruction according to claim 7 is characterized in that: The 3D reconstruction module further includes a feature detection unit and a 3D reconstruction algorithm unit; wherein the feature detection unit is used to detect key points from the image data captured by the depth camera and calculate descriptors for matching, and the 3D reconstruction algorithm unit is used to calculate the 3D coordinates by triangulation based on the matched feature points and camera posture, construct the preliminary 3D structure of the scene, and apply the Bundle Adjustment technology to optimize the camera posture and the position of the 3D points.

9. The confined space UAV intelligent navigation system based on three-dimensional reconstruction according to claim 7, characterized in that: The path search algorithm in the path planning module is the A algorithm, which is used to evaluate the actual travel cost from the starting node to the current node and the estimated value of the travel cost from the current node to the end point; the path optimization algorithm dynamically adjusts the searched path based on the flight characteristics of the drone, including speed, acceleration and steering angle limits.

10. The confined space UAV intelligent navigation system based on three-dimensional reconstruction according to claim 7, characterized in that: The sensor module also includes an obstacle detection sensor for monitoring obstacle information around the drone in real time; and the dynamic adjustment mechanism in the control module includes an improved fuzzy obstacle avoidance algorithm for performing local path planning when an obstacle is detected, and a backstepping flight control algorithm for performing point-to-point flight, avoiding obstacles, and returning to a predetermined optimal flight path.

Images