Wall tracking flight method for unmanned aerial vehicle in closed environment based on visual inertia fusion and RANSAC algorithm

Abstract

The invention discloses a wall tracking flight method for an unmanned aerial vehicle in a closed environment based on visual inertia fusion and an RANSAC algorithm, relates to the technical field of unmanned aerial vehicle autonomous flight, and aims to solve many limitations of an existing unmanned aerial vehicle autonomous obstacle avoidance technology in the aspects of cost, calculation complexity and environmental adaptability. In order to solve the problems that in the prior art, an unmanned aerial vehicle autonomous obstacle avoidance algorithm based on visual inertia fusion and a boundary tracking algorithm based on RANSAC work cooperatively, accurate pose estimation and environment perception are provided through visual inertia fusion, basic data are provided for the boundary tracking algorithm, and the requirement of autonomous flight in a complex indoor environment is difficult to meet. And meanwhile, target point information output by the boundary tracking algorithm is fed back to a path planning module of the autonomous obstacle avoidance system to form closed-loop control. The method solves the problems of high calculation complexity and poor environmental adaptability in the prior art, has the advantages of low cost, high robustness, real-time performance and the like, and is suitable for routing inspection and surveying and mapping tasks of complicated indoor environments such as warehouses, factories and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous flight of unmanned aerial vehicles (UAVs), and in particular to a wall tracking flight method of an UAV in a closed environment based on visual-inertial fusion and a RANSAC algorithm. Background Art

[0002] With the rapid development of drone technology and the continuous expansion of its application areas, the ability of drones to fly autonomously and avoid obstacles in indoor environments has become particularly important. Especially in complex indoor environments such as warehouses, factories, and construction sites, drones need to possess high-precision autonomous obstacle avoidance and path planning capabilities to complete tasks such as inspection, mapping, and logistics. However, existing autonomous obstacle avoidance technologies for drones mostly rely on traditional sensor fusion methods, such as laser radar (LiDAR) and ultrasonic sensors. While these methods can achieve obstacle avoidance functions to a certain extent, they have limitations such as high cost, high computational complexity, and sensitivity to environmental changes. Furthermore, these methods often exhibit low robustness and adaptability when dealing with dynamic obstacles in complex environments.

[0003] In recent years, vision-based autonomous obstacle avoidance technology has become a research hotspot. Visual sensors offer advantages such as low cost, rich information, and strong environmental adaptability, providing drones with a wealth of environmental information. However, the application of visual sensors in indoor environments faces numerous challenges, such as fluctuating lighting conditions, lack of texture, and dynamic obstacles. To overcome these challenges, researchers have proposed a variety of vision-based autonomous obstacle avoidance algorithms. However, most of these algorithms rely on complex image processing and feature extraction techniques, resulting in high computational complexity and difficulty meeting real-time requirements.

[0004] Against this backdrop, research on autonomous wall-following algorithms for drones is particularly important and urgent. These algorithms specifically address the challenge of drones flying close to or navigating along walls in indoor environments. By efficiently processing and accurately analyzing visual information, they enable drones to overcome multiple challenges in complex indoor environments, such as lighting, textures, and dynamic obstacles, achieving stable and reliable autonomous wall-following. This, in turn, lays a solid foundation for the widespread adoption of drones in specific scenarios, such as indoor space mapping and confined area inspections.

[0005] In summary, existing autonomous obstacle avoidance technologies for drones have numerous limitations in terms of cost, computational complexity, and environmental adaptability, making them difficult to meet the demands of autonomous flight in complex indoor environments. Therefore, developing an efficient, low-cost, and adaptable autonomous obstacle avoidance and boundary tracking method for drones has become a pressing challenge in the field of drone technology. Summary of the Invention

[0006] In order to solve the problem that the existing UAV autonomous obstacle avoidance technology has many limitations in cost, computational complexity and environmental adaptability, and is difficult to meet the needs of autonomous flight in complex indoor environments, the present invention proposes a UAV wall tracking flight method in a closed environment based on visual-inertial fusion and RANSAC algorithm.

[0007] The technical solution adopted by the present invention to solve the above problems is:

[0008] The present invention discloses a method for tracking a UAV wall in a closed environment based on visual-inertial fusion and the RANSAC algorithm, comprising: an autonomous obstacle avoidance algorithm for the UAV based on visual-inertial fusion, for realizing autonomous positioning and obstacle avoidance of the UAV in a closed environment; and a boundary tracking algorithm based on the RANSAC algorithm, for realizing precise tracking flight of the UAV along a wall.

[0009] The autonomous obstacle avoidance algorithm for UAV based on visual-inertial fusion includes the following steps:

[0010] Step 1: Deploy visual sensors and IMUs. Install a binocular camera on the drone, ensuring that the camera covers the entire operating area as much as possible. Also install an inertial measurement unit to ensure that the IMU can obtain real-time acceleration and angular velocity information of the drone.

[0011] Step 2: Based on the indoor environment to be positioned, select the coordinate origin, establish a rectangular coordinate system, and initialize the visual-inertial fusion system. Complete the calibration of the visual and inertial sensors, and calibrate the binocular camera and IMU to ensure that the system can accurately obtain environmental information. The calibration process includes camera intrinsic parameter calibration, distortion parameter calibration, IMU-to-camera transformation matrix calibration, and time offset calibration between the IMU and camera.

[0012] Step 3: Extract feature points from the binocular image, record their pixel coordinates and descriptors, and track the feature points. Simultaneously, pre-integrate the IMU data, calculate the velocity and angle changes, and obtain the UAV's motion information. Fusion the visual feature points with the IMU pre-integrated data, using nonlinear optimization methods to solve and optimize the pose. Update the UAV's pose estimation results in real time to ensure accuracy and real-time performance.

[0013] Step 4: Divide the indoor environment into uniformly sized grids, with a fixed distance between two adjacent reference points. Convert the real-world coordinates to grid map coordinates. Initialize the occupied grid map, with all grids initially unoccupied.

[0014] Step 5: Combine the current position, grid occupancy information and the given target point, and use the path planning algorithm to search, optimize and adjust the path. During the path planning process, obstacles on the path are detected in real time, and the path is adjusted to avoid obstacles to ensure the safe flight of the UAV.

[0015] The RANSAC-based boundary tracking algorithm includes the following steps:

[0016] Step 1: Obtain the current position of the drone from the odometer, initialize the view of interest, and set the field of view direction to suit the mission requirements;

[0017] First, according to the virtual focal length f of the field of interest, the width and height ΔX and ΔY of the field of interest FOV, and the desired depth Z, the corresponding number and offset of the length and width directions of the discrete sampling points are calculated:

[0018]

[0019] Among them, f is the virtual focal length of the field of interest relative to the actual scene, ΔX and ΔY represent the width and height of the field of interest FOV respectively, and Z represents the desired depth of field of view. The specific physical quantities are as follows: Figure 2 As shown. w and h represent the number of discrete sampling points in length and width respectively, c x and c y Indicates the sampling offset of discrete sampling points in width and height.

[0020] Then by traversing the discrete sampling points (u,v,1) T Perform projection to traverse the raycasting endpoint (X, Y, Z) T :

[0021]

[0022] And transform the ray projection end point from the interested perspective coordinate system to the world coordinate system.

[0023] Step 2: Use raycasting to traverse the grid map within the perspective of interest to determine if there are any boundaries. First, determine the minimum number of data points required to calculate the model; for plane fitting, this is three points. Then, calculate the number of sampling rounds, N, based on the probability k of a single sampling being an inlier and the probability P of at least one round of multiple sampling being a full inlier.

[0024] Perform a round of random sampling and use the data points from that round to calculate the plane model. The three points can be used to construct a system of equations, which can be solved to obtain the parameters of the plane model.

[0025] Calculate the degree of deviation d between the remaining unsampled data points and the model. If the difference is less than a certain threshold, it is considered an inlier and the number of inliers is counted. Compare the number of inliers in the current model with the best previously tested model, and record the model parameters and number of inliers with the largest number of inliers.

[0026] Repeat the above steps until the iteration ends or the current model is good enough, that is, the number of inliers is greater than a certain number. After plane fitting, the plane normal vector is obtained, that is, the direction of the wall.

[0027] Transform all inner points from the world coordinate system {W} to the body coordinate system {B}, and select the point with the largest x coordinate as the starting point of the plane normal vector.

[0028] If the drone does not detect a wall, it will not perform wall fitting. At this time, the drone is in the state of searching for a wall. There are many strategies for generating target points. For example, the target point can be at the current position of the drone, that is, the drone is expected to hover. Alternatively, the target point can be generated in front of the right or left of the drone to conduct a tentative flight and search for the wall.

[0029] Step 3: When the number of occupied grids is greater than a given threshold, the RANSAC algorithm is used to perform plane fitting to obtain the direction and position of the boundary;

[0030] Traverse all occupied grids in the plane, find the occupied grid in front of the drone, attach the normal vector to this point, and move the given distance from the wall in the direction of the normal vector to obtain the coordinates of the next target point:

[0031]

[0032] s=SIGN((tp)·v) (2)

[0033]

[0034] in,(·) X Represents the translation transformation part of the rigid body transformation matrix (three-dimensional vector), represents the rigid body transformation matrix of the drone's body coordinate system relative to the world coordinate system, t represents the position of the drone relative to the world coordinate system, g represents the next target point the drone needs to go to, and SIGN(·) represents the sign function. represents the plane normal vector obtained after plane fitting, p represents the coordinate position of the frontmost point of the drone's view on the plane, s is a scalar of -1 or +1, d w The desired distance between the drone and the wall can be adjusted according to the specific situation.

[0035] Perform an occupancy check on the grid where the target point is located. If it is occupied, it means that the target point is within the obstacle. Therefore, the new target point is adjusted to the front of the occupied grid closest to the direction of the drone and the original target point to ensure that the adjusted target point is not within the obstacle.

[0036] Step 4: Based on the fitted plane normal vector and the given safe distance, the next target point is generated, ensuring that the drone maintains a safe distance from the boundary. Specifically, EGO-Planner is used to plan a path to the target point, ensuring that the path can be adjusted in real time to avoid static or slow-moving obstacles. During the path planning process, obstacles along the path are detected in real time and the path is adjusted to avoid them, ensuring safe flight.

[0037] Step 5: If no boundary is detected, generate target points according to a preset strategy, such as hovering or exploratory flight, to find the boundary.

[0038] The beneficial effects of the present invention are:

[0039] 1. This invention achieves high-precision pose estimation and environmental perception by fusing binocular vision and inertial measurement unit (IMU) data. Combined with occupancy grid maps and path planning algorithms, this method can achieve efficient autonomous obstacle avoidance and path planning in complex indoor environments, improving the autonomous flight capabilities of drones in complex indoor environments.

[0040] 2. The present invention also proposes a boundary tracking algorithm, which uses ray casting and RANSAC plane fitting technology to achieve effective tracking and obstacle avoidance of boundaries such as walls, ensuring that the drone can fly safely along the boundaries, reducing collision risks, and improving the efficiency and safety of mission execution.

[0041] 3. The present invention solves the problems of high computational complexity and poor environmental adaptability in the existing technology. It has the advantages of low cost, high robustness and real-time performance, and is suitable for inspection and mapping tasks in complex indoor environments such as warehouses and factories. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a framework diagram of the boundary tracking algorithm based on RANSAC in the present invention.

[0043] Figure 2 Schematic diagram of the boundary tracking algorithm of the present invention.

[0044] Figure 3 This is the autonomous flight test of the dark corridor of the present invention.

[0045] Figure 4 This is the boundary (wall) tracking simulation result of the present invention.

[0046] Figure 5This is the actual flight test result of the wall tracking algorithm of the present invention. DETAILED DESCRIPTION

[0047] Specific embodiment 1: This embodiment provides a method for tracking a UAV wall in a closed environment based on visual-inertial fusion and RANSAC algorithm, including:

[0048] The autonomous obstacle avoidance algorithm for UAVs based on visual-inertial fusion is used to achieve autonomous positioning and obstacle avoidance for UAVs in closed environments;

[0049] The RANSAC-based boundary tracking algorithm is used to achieve accurate tracking flight of drones along walls.

[0050] Among them, the UAV autonomous obstacle avoidance algorithm based on visual-inertial fusion and the boundary tracking algorithm based on RANSAC work together to provide accurate pose estimation and environmental perception through visual-inertial fusion, providing basic data for the boundary tracking algorithm. At the same time, the target point information output by the boundary tracking algorithm is fed back to the path planning module of the autonomous obstacle avoidance system to form a closed-loop control.

[0051] Specific embodiment 2: This embodiment provides a UAV autonomous obstacle avoidance algorithm based on visual-inertial fusion. The framework of the UAV autonomous obstacle avoidance algorithm based on visual-inertial fusion consists of a camera, a flight controller, a pose estimation algorithm based on visual-inertial, a map construction algorithm based on occupancy grid map, a path planning algorithm based on EGO-Planner, and a ground station. The specific process is as follows:

[0052] Step 1: Deploy visual sensors and IMU

[0053] 1. Deploy visual sensors: Install binocular cameras on the drone, ensuring that the cameras cover the entire operating area as much as possible.

[0054] 2. Deploy IMU: Install an inertial measurement unit on the drone to ensure that the IMU can obtain the drone's acceleration and angular velocity information in real time.

[0055] Step 2: Initialize the visual-inertial fusion system

[0056] 1. Select the coordinate origin: According to the indoor environment to be positioned, select the coordinate origin and establish a rectangular coordinate system.

[0057] 2. Calibrate visual and inertial sensors: Calibrate the binocular camera and IMU to ensure the system can accurately capture environmental information. The calibration process includes camera intrinsic parameter calibration, distortion parameter calibration, IMU-to-camera transformation matrix calibration, and time offset calibration between the IMU and camera.

[0058] Step 3: Drone pose estimation

[0059] 1. Feature point extraction and tracking: Extract feature points from the binocular image, record their pixel coordinates and descriptors, and track the feature points.

[0060] 2. IMU pre-integration: Pre-integrate the IMU data to calculate the speed and angle changes and obtain the motion information of the drone.

[0061] 3. Data fusion: Fuse visual feature points and IMU pre-integrated data, and use nonlinear optimization methods to solve and optimize the pose.

[0062] 4. Pose estimation results: Update the drone's pose estimation results in real time to ensure the accuracy and real-time performance of pose estimation.

[0063] Step 4: Build an occupancy grid map

[0064] 1. Grid division: Divide the indoor environment into grids of uniform size, with the distance between two adjacent reference points being a fixed value, and convert the coordinates in the real world into coordinates in the grid map.

[0065] 2. Initialize the map: Initialize the occupied grid map. All grids are initially unoccupied.

[0066] Step 5: Path Planning and Obstacle Avoidance

[0067] 1. Path planning: Combine the current position, grid occupancy information, and the given target point to use the path planning algorithm to perform path search, optimization, and time adjustment.

[0068] 2. Obstacle avoidance: During the path planning process, obstacles on the path are detected in real time, and the path is adjusted to avoid obstacles to ensure the safe flight of the drone.

[0069] Specific implementation method three: This implementation method provides a RANSAC-based boundary tracking algorithm. The boundary tracking algorithm framework is as follows: Figure 1 As shown in the figure. To enable the drone to use its onboard sensors to perform maintenance tasks along a wall while maintaining a certain distance from the wall, a boundary tracking algorithm was designed and verified through simulation. The overall framework of the boundary tracking algorithm includes odometry, occupancy grid maps, the core components of the boundary tracking algorithm, and path planning. The specific process is as follows:

[0070] Step 1: Initialize the perspective of interest

[0071] This function slices the data collected by the camera's full FOV. If the mission requires flying along the right wall, set the viewpoint to the right front. {C} represents the virtual imaging plane coordinate system, {V} represents the field of interest coordinate system, {B} represents the drone's body coordinate system, and {W} represents the world coordinate system.

[0072] 1. Calculation of the number of discrete sampling points in length and width: First, based on the virtual focal length f of the field of interest, the width ΔX and height ΔY of the regional field of view FOV, and the desired depth of view Z, calculate the number of sampling points corresponding to the width and height of the region and their offset:

[0073]

[0074] Where f is the virtual focal length of the field of interest, which can be selected so that the calculated w and h are not less than 1, ΔX and ΔY represent the width and height of the field of interest FOV respectively, and Z represents the desired depth of field of view. The specific physical quantities are as follows: Figure 2 (a). w and h represent the number of discrete sampling points in length and width, respectively. x and c y Indicates the sampling offset of discrete sampling points in width and height.

[0075] 2. Sampling point projection: Then traverse all discrete sampling points (u, v, 1) T And project, thus traversing all the ray projection end points (X, Y, Z) T :

[0076]

[0077] 3. Sampling point coordinate system transformation: project the ray to the end point (X, Y, Z) T Transform from the field of interest coordinate system to the world coordinate system.

[0078] Step 2: RANSAC algorithm plane fitting

[0079] 1. Determine the number of sampling points per round and the number of sampling rounds: First, determine the minimum number of data points required to calculate the model. For plane fitting, this is three points. Then, calculate the number of sampling rounds N based on the probability k of a single sampling being an inlier and the probability P of at least one round being fully inlier in multiple sampling rounds.

[0080] P=1-(1-k 3 ) N (6)

[0081]

[0082] 2. Calculate the plane model: Perform a round of random sampling and use the data points obtained from the single round of sampling to calculate the plane model. The three points can construct a system of equations, such as Equation (6), which can be solved to obtain the parameters of the plane model.

[0083] Ax+By+Cz=1 (8)

[0084]

[0085] 3. Calculate the number of inliers: Calculate the degree of deviation d between the remaining unextracted data points and the model. If the difference is less than a certain threshold, it is determined to be an inlier and the number of inliers is counted.

[0086]

[0087] 4. Number comparison: Compare the number of inliers between the current model and the best model previously introduced, and record the model parameters and number of inliers with the largest number of inliers.

[0088] 5. Model iteration: Repeat the above steps until the iteration ends or the current model is good enough, that is, the number of inliers is greater than a certain number.

[0089] 6. Plane normal vector: After plane fitting, the plane normal vector, that is, the direction of the wall, is obtained.

[0090] 7. Normal vector attachment point: Transform all internal points from the world coordinate system {W} to the body coordinate system {B}, and select the point with the largest x coordinate as the starting point of the plane normal vector.

[0091] Step 3: Target point generation

[0092] 1. Target point calculation: Traverse all occupied grids in the plane, find the occupied grid in front of the drone, attach the normal vector to this point, and move the given distance from the wall in the direction of the normal vector to obtain the coordinates of the next target point:

[0093]

[0094] s=SIGN((tp)·v) (12)

[0095]

[0096] in,(·) X Represents the translation transformation part of the rigid body transformation matrix (three-dimensional vector), represents the rigid body transformation matrix of the drone's body coordinate system relative to the world coordinate system, t represents the position of the drone relative to the world coordinate system, g represents the next target point the drone needs to go to, and SIGN(·) represents the sign function. represents the plane normal vector obtained after plane fitting, p represents the coordinate position of the frontmost point of the drone's view on the plane, s is a scalar of -1 or +1, d w The desired distance between the drone and the wall can be adjusted according to the specific situation. Figure 2 shown.

[0097] 2. Target point adjustment: Check the grid where the target point is located. If it is occupied, it means that the target point is within the obstacle. Therefore, the new target point is adjusted to the occupied grid p closest to the direction of the drone and the original target point. occ In front of the target point, ensure that the adjusted target point is not within the obstacle, where k is a scalar parameter that can be adjusted by itself, satisfying 0 <k<1。

[0098] g=t+k·(p occ -t) (14)

[0099] Step 4: Path Planning and Obstacle Avoidance

[0100] 1. Path Planning: Use EGO-Planner to plan the path to the target point, ensuring that the path can be adjusted in real time to avoid static or low-speed obstacles.

[0101] 2. Obstacle avoidance: During the path planning process, obstacles on the path are detected in real time, and the path is adjusted to avoid obstacles to ensure the safe flight of the drone.

[0102] Step 5: Search processing when no boundary is detected

[0103] Target point generation strategy: If the drone does not detect a wall, wall fitting is not performed. The drone is now searching for a wall. There are several strategies for generating target points. For example, the target point can be at the drone's current position, meaning the drone is expected to hover. Alternatively, a target point can be generated in front of the right or left of the drone for a tentative flight and search for a wall.

[0104] The entire boundary tracking algorithm is as follows Figure 2 As shown in the figure, the blue field of view represents the view of interest, the green plane represents the wall within that view, and the yellow vector represents the fitted plane normal. Extending the normal vector outward a certain distance yields the next waypoint, the blue dot. The red dashed line represents the path to the waypoint mapped by the planner, which can be adjusted in real time to avoid static or slow-moving obstacles.

[0105] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Although the present invention has been disclosed as a preferred embodiment as above, it is not intended to limit the present invention. Any technician familiar with the present profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical content disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent replacement and improvement of the above embodiments made according to the technical essence of the present invention, within the spirit and principles of the present invention, without departing from the content of the technical solution of the present invention, shall still fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for tracking the wall of a UAV in a closed environment based on visual-inertial fusion and RANSAC algorithm, characterized by: include: The autonomous obstacle avoidance algorithm for UAVs based on visual-inertial fusion is used to achieve autonomous positioning and obstacle avoidance for UAVs in closed environments; The RANSAC-based boundary tracking algorithm is used to achieve accurate tracking flight of drones along walls.

2. The method for tracking a wall of a UAV in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 1 is characterized in that: The autonomous obstacle avoidance algorithm for UAV based on visual-inertial fusion includes the following steps: Step 1: Deploy visual sensors and IMUs. Install a binocular camera on the drone to ensure that the camera covers the entire operating area as much as possible. Also install an inertial measurement unit to ensure that the IMU can obtain the drone's acceleration and angular velocity information in real time. Step 2: Based on the indoor environment to be positioned, select the coordinate origin, establish a rectangular coordinate system, and initialize the visual-inertial fusion system; complete the calibration of the visual and inertial sensors, and calibrate the binocular camera and IMU to ensure that the system can accurately obtain environmental information; Step 3: Extract feature points from the binocular image, record their pixel coordinates and descriptors, and track the feature points; at the same time, pre-integrate the IMU data, calculate the speed and angle changes, and obtain the motion information of the drone; The visual feature points and IMU pre-integrated data are integrated, and the pose is solved and optimized using nonlinear optimization methods. The pose estimation results of the UAV are updated in real time to ensure the accuracy and real-time performance of the pose estimation. Step 4: Divide the indoor environment into uniformly sized grids, with the distance between two adjacent reference points being a fixed value. Convert the real-world coordinates into grid map coordinates. Initialize the occupied grid map, with all grids initially unoccupied. Step 5: Combine the current position, grid occupancy information and the given target point, and use the path planning algorithm to perform path search, optimization and time adjustment; During the path planning process, obstacles on the path are detected in real time, and the path is adjusted to avoid obstacles to ensure the safe flight of the drone.

3. The method for tracking a UAV wall in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 1 is characterized in that: In step 2 of the UAV autonomous obstacle avoidance algorithm based on visual-inertial fusion, the calibration process includes camera intrinsic parameter calibration, distortion parameter calibration, IMU to camera transformation matrix calibration, and time offset calibration between IMU and camera.

4. The method for tracking a wall of a UAV in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 1 is characterized in that: The RANSAC-based boundary tracking algorithm includes the following steps: Step 1: Get the current position of the drone from the odometer, initialize the view of interest, and set the field of view to suit the mission requirements; Step 2: Use ray casting to traverse the occupancy of the grid map within the perspective of interest to determine whether there is a boundary; Step 3: When the number of occupied grids is greater than a given threshold, the RANSAC algorithm is used to perform plane fitting to obtain the direction and position of the boundary; Step 4: Generate the next target point based on the fitted plane normal vector and the given safety distance to ensure that the drone maintains a safe distance from the boundary; Step 5: If no boundary is detected, generate target points according to the preset strategy.

5. The method for tracking a wall of a UAV in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 4 is characterized in that: The specific operations of step 1 of the RANSAC-based boundary tracking algorithm are as follows: First, according to the virtual focal length f of the field of interest, the width and height ΔX and ΔY of the field of interest FOV, and the desired depth Z, the corresponding number and offset of the length and width directions of the discrete sampling points are calculated: Where f is the virtual focal length of the field of interest relative to the actual scene, ΔX and ΔY represent the width and height of the field of interest FOV, respectively, and Z represents the desired depth of field of view. The specific physical quantities are shown in Figure 2; w and h represent the number of discrete sampling points in length and width, respectively, and c x and c y Indicates the sampling offset of discrete sampling points in width and height; Then by traversing the discrete sampling points (u,v,1) T Perform projection to traverse the raycasting end point (X, Y, Z) T : And transform the ray projection end point from the interested perspective coordinate system to the world coordinate system.

6. The method for tracking a UAV wall in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 4 is characterized in that: The specific operation of step 2 of the RANSAC-based boundary tracking algorithm is as follows: First, determine the minimum number of data points to calculate the model, which is three points for plane fitting. Then calculate the number of sampling rounds N based on the probability k that a single sampling is an inlier and the probability P that at least one round of multi-round sampling is full of inliers. Perform a round of random sampling and use the data points obtained from the single round of sampling to calculate the plane model; the three points can construct a system of equations, and solving the system of equations can obtain the parameters of the plane model; Calculate the degree of deviation d between the remaining unsampled data points and the model. If the difference is less than a certain threshold, it is determined to be an inlier and the number of inliers is counted. Compare the number of inliers of the current model with that of the best model previously tested, and record the model parameters and number of inliers with the largest number of inliers. Repeat the above steps until the iteration ends or the current model is good enough, that is, the number of inliers is greater than a certain number; after plane fitting, the plane normal vector, that is, the direction of the wall, is obtained; Transform all inner points from the world coordinate system {W} to the body coordinate system {B}, and select the point with the largest x coordinate as the starting point of the plane normal vector; If the drone does not detect a wall, it will not perform wall fitting. At this time, the drone is in the state of searching for a wall. There are many strategies for generating target points. For example, the target point can be at the current position of the drone, that is, the drone is expected to hover. Alternatively, the target point can be generated in front of the right or left of the drone to conduct a tentative flight and search for the wall.

7. The method for tracking a wall of an unmanned aerial vehicle in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 4 is characterized in that: The specific operation of step 3 of the RANSAC-based boundary tracking algorithm is as follows: traverse all occupied grids in the plane, find the occupied grid in front of the drone, attach the normal vector to this point, and move the given distance from the wall in the direction of the normal vector to obtain the coordinates of the next target point: s=SIGN((tp)·v) (2) in,(·) X Represents the translation transformation part of the rigid body transformation matrix, represents the rigid body transformation matrix of the drone's body coordinate system relative to the world coordinate system, t represents the position of the drone relative to the world coordinate system, g represents the next target point the drone needs to go to, and SIGN(·) represents the sign function. represents the plane normal vector obtained after plane fitting, p represents the coordinate position of the frontmost point of the drone's view on the plane, s is a scalar of -1 or +1, d w The desired distance between the drone and the wall can be adjusted according to the specific situation; Perform an occupancy check on the grid where the target point is located. If it is occupied, it means that the target point is within the obstacle. Therefore, the new target point is adjusted to the front of the occupied grid closest to the direction of the drone and the original target point to ensure that the adjusted target point is not within the obstacle.

8. The method for tracking a UAV wall in a closed environment based on visual-inertial fusion and RANSAC algorithm according to claim 4 is characterized in that: The specific operations of step 4 of the RANSAC-based boundary tracking algorithm are as follows: using EGO-Planner to plan the path to the target point, ensuring that the path can be adjusted in real time to avoid static or low-speed obstacles; during the path planning process, obstacles on the path are detected in real time, and the path is adjusted to avoid obstacles to ensure the safe flight of the drone.

Images