A method, device, equipment and storage medium for avoiding obstacles by a UAV

Abstract

The application discloses a method, device and equipment for avoiding obstacles by a UAV and a storage medium, the method comprising: reconstructing a three-dimensional map based on collected images of obstacles and a monocular Visual SLAM framework, and acquiring the position of the UAV and the axis distance between the UAV and the obstacles; projecting the UAV according to the flight direction of the UAV, and judging whether the projection of the UAV in the flight direction intersects with the plane where the obstacles are located; when the projection of the UAV intersects with the obstacles, acquiring an edge point located on the obstacles and having the shortest distance to the projection center of the UAV; forming a collision circle with a preset safety radius with the edge point as the center; calculating the deflection angle of the UAV according to the relative position of the UAV and the obstacles and the driving speed of the UAV; and performing route planning according to the collision circle, the deflection angle, the axis distance and the edge point, so that the UAV can bypass the obstacles via the boundary of the collision circle. The method can realize the detection of obstacles and the planning of the UAV obstacle avoidance path.

Description

Technical Field

[0001] This invention relates to the field of drone protection, and more specifically to a method, apparatus, device, and storage medium for drones to avoid obstacles. Background Technology

[0002] With the continuous development of drone technology, the application fields of drones are also expanding. However, drones face safety threats from obstacles such as mountains, buildings, and dangerous areas during flight. Automatic obstacle avoidance systems are a crucial safety guarantee for drones to successfully complete their flight missions, largely reflecting the intelligence and safety of drones. Therefore, researching drone obstacle avoidance systems is of great significance.

[0003] Existing UAV obstacle avoidance technologies include infrared and laser ranging. However, infrared and laser obstacle avoidance are easily affected and interfered with by the external environment, leading to inaccurate distance measurements. They suffer from limitations in accurately autonomous obstacle avoidance. Furthermore, commonly used trajectory planning methods in current technologies include artificial potential field methods, A* algorithms, and RRT algorithms. When multiple minima exist, the A* algorithm cannot guarantee the optimal search path. RRT is a purely random search algorithm that is insensitive to environmental type; however, when the space contains many obstacles or narrow passage constraints, the algorithm's convergence speed is slow, and its efficiency drops significantly. Artificial potential field methods are prone to getting trapped in local optima and exhibit trajectory jitter in narrow passages. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and storage medium for an unmanned aerial vehicle (UAV) to avoid obstacles. The invention achieves obstacle detection and obstacle path planning during flight by reconstructing three-dimensional information in real time and accurately.

[0005] In a first aspect, embodiments of the present invention provide a method for an unmanned aerial vehicle (UAV) to avoid obstacles, the method comprising:

[0006] Based on the images of obstacles captured by the drone's camera and the monocular Visual SLAM framework, a 3D map is reconstructed to obtain the position of the drone and the axial distance between the drone and the obstacles.

[0007] The drone is projected according to its flight direction to determine whether the projection of the drone in the flight direction intersects with the plane where the obstacle is located.

[0008] When it is determined that the projection of the drone intersects with the obstacle, obtain the edge point on the obstacle that has the shortest distance to the center of the drone's projection;

[0009] A collision circle with a preset safety radius is formed with the edge point as the center;

[0010] The relative position between the drone and the obstacle is determined based on the position of the drone and the position of the obstacle;

[0011] The drone's speed is obtained, and the drone's deflection angle is calculated based on the relative position and the drone's speed.

[0012] Route planning is performed based on the collision circle, the drone's deflection angle, the axial distance, and the edge point, so that the drone can bypass the obstacle by passing through the boundary of the collision circle.

[0013] Furthermore, the safety radius d = 2M; where M is X times the larger of the length and width of the UAV; where X is greater than 2.

[0014] Furthermore, the route planning based on the collision circle, the axial distance, and the edge point, so that the drone can bypass the obstacle via the boundary of the collision circle, specifically involves:

[0015] Based on the collision circle and the center of the drone, determine the tangent from the center of the drone to the collision circle, and obtain the intersection point of the tangent and the collision circle;

[0016] Control the drone to fly to the intersection point in the direction of the tangent;

[0017] After reaching the intersection point, according to the safety radius, fly along the arc of the collision circle to the first target point where the drone's positive direction is parallel to the side of the obstacle;

[0018] After reaching the first destination, fly along the flight direction from the first destination to the second destination; wherein, the second destination is the position where the side of the obstacle is no longer visible in the three-dimensional map;

[0019] After reaching the second objective point, continue flying in the same direction as at the second objective point for at least a predetermined distance to reach the third objective point, thus completing the drone obstacle avoidance.

[0020] Furthermore, based on the collision circle and the center of the drone, the tangent line from the center of the drone to the collision circle is determined, and the intersection point of the tangent line and the collision circle is obtained as follows:

[0021] Calculate the distance from the center of the drone to the center of the circle, based on the center of the circle.

[0022] Based on the distance between the center of the drone and the center of the circle, and the axial distance between the drone and the obstacle, the first angle between the optical axis direction of the drone and the line connecting the drone to the center of the collision circle is obtained.

[0023] By establishing the radius of the collision circle, the second included angle between the UAV and the tangent of the collision circle is obtained;

[0024] The actual deflection angle of the UAV is obtained based on the first included angle and the second included angle, and the intersection point is determined based on the actual deflection angle and the collision circle.

[0025] Furthermore, the predetermined distance Where, β ω Let d be the horizontal field of view of the 3D map, d be the safety radius, and R be the predetermined distance. Furthermore, the target point of the UAV is acquired, and the UAV is controlled to fly along the direction of the target point after reaching the third target point.

[0026] In a second aspect, embodiments of the present invention provide a device for an unmanned aerial vehicle (UAV) to avoid obstacles, comprising:

[0027] The reconstruction module is used to reconstruct a 3D map based on the images of obstacles captured by the drone's camera and a monocular Visual SLAM framework, so as to obtain the position of the drone and the axial distance between the drone and the obstacles;

[0028] The judgment module is used to project the drone according to the drone's flight direction in order to determine whether the projection of the drone in the flight direction intersects with the plane where the obstacle is located.

[0029] The acquisition module is used to acquire the edge point on the obstacle that has the shortest distance to the center of the drone's projection when it is determined that the drone's projection intersects with the obstacle;

[0030] A forming module is used to form a collision circle with a preset safety radius, centered on the edge point;

[0031] A relative position acquisition module is used to determine the relative position between the drone and the obstacle based on the position of the drone and the position of the obstacle;

[0032] The calculation module is used to obtain the drone's travel speed and calculate the drone's deflection angle based on the relative position and the drone's travel speed.

[0033] The obstacle bypass module is used to perform route planning based on the collision circle, the UAV deflection angle, the axial distance, and the edge point, so that the UAV can bypass the obstacle via the boundary of the collision circle.

[0034] Thirdly, embodiments of the present invention provide a terminal device, characterized in that it includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor executes the computer program to implement the method for an unmanned aerial vehicle to avoid obstacles as described in the first aspect.

[0035] Fourthly, the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the obstacle avoidance method of the unmanned aerial vehicle as described in the first aspect.

[0036] Implementing the embodiments of the present invention has the following beneficial effects:

[0037] This invention utilizes images of obstacles captured by a drone's camera and a monocular Visual SLAM framework to achieve real-time and accurate reconstruction of 3D information. Based on this, it enables obstacle detection, acquisition of relative position information between the drone and obstacles, replanning of the drone's flight path, and autonomous obstacle avoidance. Attached Figure Description

[0038] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the method for an unmanned aerial vehicle (UAV) to avoid obstacles provided in the first embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the structure of the method for an unmanned aerial vehicle (UAV) to avoid obstacles provided in the first embodiment of the present invention.

[0041] Figure 3 This is a flowchart illustrating the monocular Visual SLAM framework provided in the first embodiment of the present invention.

[0042] Figure 4 This is a threat cone vector diagram showing the relative positions of obstacles and drones, provided in the first embodiment of the present invention.

[0043] Figure 5 This is a schematic diagram of the structure of the UAV projection center within the obstacle, provided in the first embodiment of the present invention.

[0044] Figure 6 This is a schematic diagram of the structure where the projection center of the UAV is outside the obstacle, as provided in the first embodiment of the present invention.

[0045] Figure 7 This is a schematic diagram of the structure of the device for avoiding obstacles by a drone provided in the second embodiment of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] First embodiment of the present invention:

[0048] See Figure 1 and Figure 2 This invention provides a method for an unmanned aerial vehicle (UAV) to avoid obstacles, the method comprising:

[0049] S10: Based on the images of obstacles captured by the drone's camera and the monocular Visual SLAM framework, a three-dimensional map is reconstructed to obtain the drone's position and the axial distance between the drone and the obstacles.

[0050] In this embodiment, a monocular camera and a satellite positioning receiver are pre-installed on the drone. The monocular camera captures images of the drone's surroundings, including obstacle images, while the satellite positioning receiver provides global anchoring and position information in world coordinates to provide bit information to the ground-based collaborative computer for drone path planning.

[0051] See Figure 3In this embodiment, the monocular Visual SLAM framework comprises sensor data, a front-end, a back-end, loop closure detection, and map building. Specifically, the sensor data preprocesses the obstacle image to remove noise. It should be noted that the sensors include a camera and an inertial measurement unit (IMU). The front-end is also called a visual odometry (VO). It quantitatively estimates the inter-frame camera motion based on adjacent frame images. By stringing together the motion trajectories of adjacent frames, the motion trajectory of the camera carrier is constructed, solving the localization problem. Then, based on the estimated camera position at each moment, the spatial position of each pixel is calculated, resulting in the map. In this embodiment, feature points of each frame are first extracted, and coarse matching of feature points is performed on adjacent frames. Then, the RANSAC (Random Sample Consensus) algorithm is used to remove unreasonable matching pairs, thus obtaining position and pose information. The back-end mainly optimizes the results of the front-end to obtain the optimal pose estimate. A nonlinear optimization (graph optimization) method is used. It treats the optimized variables as nodes of the graph and the error terms as edges, and iteratively optimizes and updates the graph after giving initial values. Due to the sparsity of graph optimization, computational complexity can be reduced while maintaining accuracy. The primary purpose of loop closure detection is to enable the UAV to recognize previously visited locations, thus addressing the problem of position drift over time. Visual loop closure detection is generally performed by judging the similarity between images, similar to how humans use their eyes to identify two identical locations. Because images are rich in information, VSLAM has a significant advantage in loop closure detection. Once a loop closure is successfully detected, a correspondence is established between the current image and previously seen images. The backend optimization algorithm can then readjust the trajectory and map based on this information, thereby minimizing accumulated errors. Finally, a 3D map is created.

[0052] S20: Project the drone onto the plane of the obstacle based on the drone's flight direction to determine whether the drone's projection in the flight direction intersects with the plane of the obstacle.

[0053] In this embodiment, the projection intersection refers to the process of framing obstacles when processing images acquired by the UAV, and simultaneously generating a projection of the UAV onto the obstacle plane on the image. The projection center can be compared with the UAV's projection by framing the obstacle to determine whether the UAV's projection intersects with the obstacle.

[0054] S30, when it is determined that the projection of the UAV intersects with the obstacle, obtain the edge point on the obstacle that has the shortest distance to the center of the UAV's projection.

[0055] S40 forms a collision circle with a preset safety radius, centered on the edge point.

[0056] S50: The relative position between the drone and the obstacle is determined based on the position of the drone and the position of the obstacle.

[0057] S60, acquire the drone's travel speed, and calculate the drone's deflection angle based on the relative position and the drone's travel speed;

[0058] S70, route planning is performed based on the collision circle, the drone's deflection angle, the axial distance, and the edge point, so that the drone can bypass the obstacle via the boundary of the collision circle.

[0059] In this embodiment, the safety radius d = 2M; where M is X times the larger of the length and width of the UAV; where X is greater than 2.

[0060] See Figure 2 In this embodiment, the optical center is the projection center of the UAV. The optical axis is a ray perpendicular to the collision center, originating from the collision center; this ray is called the optical axis. L is the distance between the UAV and the obstacle. γ a γ is the angle between the optical axis of the drone and the line connecting the drone to the center of the collision circle, which is the first angle in this embodiment. b The angle between the drone and the tangent of the collision circle is the second angle in this embodiment. d is the radius of the obstacle circle, and the minimum distance between the drone and the obstacle. P ou Let V be the position vector of the UAV. ou P represents the speed at which the drone travels. u P o P represents the location of the drone and the obstacle, respectively. ou =P o -P u Let P1 and P2 be the relative position vectors of the drone and the obstacle. P1 and P2 are the left and right tangents (r) through the collision region of the drone, respectively. u r o d represents the radius of the drone and the obstacle, respectively. c =r u +r o Points A, B, C, and D are all waypoints during flight, and the lines connecting them form the flight path. Specifically, A is the intersection of the tangent line and the collision circle in this embodiment; B is the first destination point; C is the second destination point; and D is the third destination point.

[0061] In this embodiment, the route planning based on the collision circle, the drone's deflection angle, the axial distance, and the edge point, so that the drone can bypass the obstacle via the boundary of the collision circle, specifically involves:

[0062] Based on the collision circle and the center of the drone, a tangent line from the center of the drone to the collision circle is determined, and the intersection of the tangent line and the collision circle is obtained. The drone is controlled to fly in the direction of the tangent line to the intersection point. After reaching the intersection point, according to the safety radius, the drone flies along the arc of the collision circle to a first target point parallel to the side of the obstacle in the forward direction. After reaching the first target point, the drone flies along the flight direction at the first target point to a second target point. The second target point is the position where the side of the obstacle is not visible in the 3D map for the first time. After reaching the second target point, the drone continues to fly along the flight direction at the second target point for at least a predetermined distance to reach a third target point, completing the drone obstacle avoidance.

[0063] Specifically, in this embodiment, when intersecting with an obstacle, the distance between the UAV and the collision center is determined geometrically by establishing the collision center. The collision cone is precisely determined by two vectors (P1, P2) derived from the UAV's position and perpendicular to the boundary circle, such as... Figure 2 As shown: P1 and P2 are the relative position vectors P of the left and right tangents through the collision area of ​​the UAV. ou The angle between the left tangent P1 and the left tangent P1 is α. + (α + >0), the angle between the right tangent P2 and the right tangent P2 is α. - (α - <0), α - =-α + φ is the relative velocity vector V ou and relative position vector P ou The included angle, V ou In P ou When on the left, φ is positive; when on the right, φ is negative. δi is the distance between the drone and the closest point to the obstacle. Based on the geometry of the collision cone, if the drone's position P... u For the vertex along the relative velocity vector V ou If a ray in the direction of the collision passes through the collision area P, then there is a potential collision hazard between the UAV and the obstacle. Equation (1) is used as the potential collision detection condition:

[0064] P ou ·V ou ≥0, δi<d C (1); Equation (1) means that only if the relative position vector P ou With relative velocity vector V ouA potential collision between the obstacle and the drone is considered to exist only when the angle is within ±π / 2 and the distance between the drone and the closest point is less than the collision radius. This collision detection condition is used to accelerate planning. If multiple obstacles exist, the obstacle with the shortest arrival time is selected as the most urgent obstacle. After identifying the obstacle requiring emergency collision avoidance, the drone must perform an emergency maneuver. This allows us to determine the angle γ between the drone's optical axis and the line connecting the drone to the center of the collision circle. a Then, by establishing the radius of the collision circle, the angle γ between the UAV and the tangent of the collision circle can be derived from the given relationship. b Therefore, the actual deflection angle γ of the UAV can be determined. p =γ a +γ b Deflection angle γ of UAV maneuver p It is usually no greater than π / 2, so the calculated deflection angle is: Considering that the information collected by the drone is affected by drone noise, the drone's deflection angle should be slightly larger than the calculated γ above, that is, a safety factor c should be added to γ: γ′=c γ In the formula, c is a constant greater than 1.

[0065] In this embodiment, based on the collision circle and the center of the drone, the tangent from the center of the drone to the collision circle is determined, and the intersection of the tangent and the collision circle is obtained as follows:

[0066] Based on the center of the circle, calculate the distance from the center of the drone to the center of the circle. Based on the distance from the center of the drone to the center of the circle and the axial distance between the drone and the obstacle, obtain the first angle between the optical axis direction of the drone and the line connecting the drone to the center of the collision circle. By establishing the radius of the collision circle, obtain the second angle between the drone and the tangent of the collision circle. Based on the first angle and the second angle, obtain the actual deflection angle of the drone, and determine the intersection point based on the actual deflection angle and the collision circle.

[0067] In this embodiment, the predetermined distance Where, β ω Let d be the horizontal field of view of the 3D map, d be the safety radius, and R be the predetermined distance.

[0068] In this embodiment, based on the characteristics of the drone, this application only describes a forward obstacle as an example. The minimum distance between the drone and the obstacle is d, which is the collision safety limit radius. A cone is constructed with the center of the obstacle and the location of the drone as the center. Let the position vector of the drone at time T be P. u The location of the obstacle is P. o So, what is the relative position P between the obstacle and the drone? ou =Po -P u , see Figure 4 :

[0069] This application constructs a threat cone with the collision intersection point as the center. An axis coordinate system is constructed with the projection center of the UAV as the origin. The axis will intersect with the edge of the obstacle. Since there are multiple intersection points between the axis and the obstacle, the edge point on the obstacle with the shortest distance from the projection center of the UAV is taken as the center. After determining the center, the minimum distance between the UAV and the obstacle is determined according to the center. The collision safety boundary is considered to be a circle with a radius of d. Set the collision center radius d = 2M or d = 2N, where M and N are 5 times the length and width respectively. This can ensure that the UAV can safely bypass the obstacle during flight. Because the flight process is not stable, setting M and N larger can reduce the collision between the UAV and the obstacle. If the collision center radius is too small (generally, M and N are not less than 2 times the length and width of the UAV respectively), it is easy to collide with the obstacle.

[0070] According to the relative position between the collision center and the projection center of the UAV, the position of the obstacle can be determined. Let the coordinates of the UAV projection center (optical center coordinates) be (x0, y0), and the coordinates of the collision center be (x p y p ).

[0071] See Figure 5 , when the UAV projection center is outside the obstacle: when x [[ID=​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​This invention utilizes images of obstacles captured by a drone's camera and a monocular Visual SLAM framework to achieve real-time and accurate reconstruction of 3D information. Based on this, it enables obstacle detection, acquisition of relative position information between the drone and obstacles, replanning of the drone's flight path, and autonomous obstacle avoidance.

[0074] Based on the first embodiment, in a preferred embodiment of this embodiment, the target point of the UAV is obtained, and the UAV is controlled to fly along the direction of the target point after reaching the third target point.

[0075] See Figure 2 In this embodiment, the final target point E of the UAV is known. After passing point D, the UAV flies directly to target point E to complete the obstacle avoidance task. After the obstacle avoidance strategy is determined, the ground-based collaborative computer sends dronekit-based code via the MAVLink protocol to communicate with the UAV flight controller and send commands to control the UAV to move horizontally left or right or vertically upward. The GPS performs repositioning, the ground station updates the waypoint information, and replans the waypoints.

[0076] Second embodiment of the present invention:

[0077] A second embodiment of the present invention provides a device for an unmanned aerial vehicle (UAV) to avoid obstacles, comprising:

[0078] The reconstruction module 100 is used to reconstruct a 3D map based on the images of obstacles collected by the drone's camera and a monocular Visual SLAM framework, so as to obtain the position of the drone and the axial distance between the drone and the obstacles.

[0079] The judgment module 200 is used to project the UAV according to the UAV's flight direction to determine whether the UAV's projection in the flight direction intersects with the plane where the obstacle is located.

[0080] The acquisition module 300 is used to acquire the edge point on the obstacle that has the shortest distance to the center of the drone's projection when it is determined that the drone's projection intersects with the obstacle.

[0081] The forming module 400 is used to form a collision circle with a preset safety radius, centered on the edge point.

[0082] The relative position acquisition module 500 is used to determine the relative position between the drone and the obstacle based on the position of the drone and the position of the obstacle;

[0083] The calculation module 600 is used to obtain the travel speed of the drone and calculate the deflection angle of the drone based on the relative position and the travel speed of the drone.

[0084] The obstacle bypass module 700 is used to perform route planning based on the collision circle, the drone's deflection angle, the axial distance, and the edge point, so that the drone can bypass the obstacle via the boundary of the collision circle.

[0085] Preferably, the safety radius d = 2M; where M is X times the larger of the length and width of the UAV; where X is greater than 2.

[0086] Preferably, the obstacle bypass module 700 specifically comprises:

[0087] The intersection point determination module is used to determine the tangent from the center of the drone to the collision circle based on the collision circle and the center of the drone, and to obtain the intersection point of the tangent and the collision circle.

[0088] An obstacle avoidance module controls the drone to fly towards the intersection point along the tangent. Upon reaching the intersection point, it flies along the arc of the collision circle to a first target point parallel to the side of the obstacle, based on the safety radius. After reaching the first target point, it flies along the flight direction from the first target point to a second target point; wherein the second target point is the position where the side of the obstacle is first no longer visible on the 3D map. After reaching the second target point, it continues flying along the flight direction from the second target point for at least a predetermined distance to reach a third target point, completing the drone obstacle avoidance.

[0089] Preferably, the intersection point determination module specifically comprises:

[0090] Based on the center of the circle, calculate the distance from the center of the drone to the center of the circle. Based on the distance from the center of the drone to the center of the circle and the axial distance between the drone and the obstacle, obtain the first angle between the optical axis direction of the drone and the line connecting the drone to the center of the collision circle. By establishing the radius of the collision circle, obtain the second angle between the drone and the tangent of the collision circle. Based on the first angle and the second angle, obtain the actual deflection angle of the drone, and determine the intersection point based on the actual deflection angle and the collision circle.

[0091] Preferably, the predetermined distance Where, β ω Let d be the horizontal field of view of the 3D map, d be the safety radius, and R be the predetermined distance.

[0092] Preferably, the drone target point is acquired, and the drone is controlled to fly along the direction of the target point after reaching the third target point.

[0093] The third embodiment of the present invention:

[0094] A third embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the steps of any of the preceding embodiments of a drone obstacle avoidance method, for example... Figure 1 Step S10 is shown. Alternatively, the processor executes the computer program to implement the functions in the above-described device examples.

[0095] Fourth embodiment of the present invention:

[0096] A fourth embodiment of the present invention provides a computer-readable storage medium comprising a stored computer program, such as a program for a method of drone obstacle avoidance. The computer program, when executed, controls the device containing the computer-readable storage medium to perform the drone obstacle avoidance method described in any of the above embodiments.

[0097] Exemplary examples show that the computer program described in the third and fourth embodiments of the present invention can be divided into one or more modules. These modules are stored in the memory and executed by the processor to complete the present invention. The one or more modules can be a series of computer program instruction segments capable of performing specific functions. These instruction segments describe the execution process of the computer program in implementing the obstacle avoidance device for unmanned aerial vehicles, specifically including the following steps. For example, the apparatus described in Embodiment 2 of the present invention.

[0098] The processor referred to can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the UAV obstacle avoidance method, connecting all parts of the UAV obstacle avoidance method through various interfaces and lines.

[0099] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the UAV obstacle avoidance method by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, text conversion function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, text message data, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0100] If the module implementing the obstacle avoidance device for the drone is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0101] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0102] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for an unmanned aerial vehicle (UAV) to avoid obstacles, characterized in that, The method includes: A 3D map is reconstructed based on obstacle images captured by a drone's camera and a monocular Visual SLAM framework. Specifically, the process involves first extracting feature points from each frame of the image, performing coarse matching of feature points between adjacent frames, and then using a random sampling consensus algorithm to remove unreasonable matching pairs, thereby obtaining position and pose information. The monocular Visual SLAM framework includes sensor data, a front-end, a back-end, loop closure detection, and map building. The back-end primarily optimizes the results from the front-end to obtain the optimal pose estimation, using a nonlinear optimization method. It treats the optimized variables as nodes in the graph and the error terms as edges, iteratively optimizing and updating after initial values ​​are given. Due to the sparsity of graph optimization, it can reduce computational load while maintaining accuracy. The main purpose of loop closure detection is to enable the UAV to recognize places it has visited, thereby solving the problem of position drift over time. Loop closure detection is accomplished by judging the similarity between images, which is the same principle as humans using their eyes to judge two identical locations. Because the images are rich in information, VSLAM has a significant advantage in loop closure detection. When a loop closure is successfully detected, a correspondence is established between the current image and previously seen images. The back-end optimization algorithm readjusts the trajectory and map based on this information, thereby minimizing accumulated errors. Finally, a 3D map is built. To obtain the position of the drone and the axial distance between the drone and the obstacle; The drone is projected according to its flight direction to determine whether the projection of the drone in the flight direction intersects with the plane where the obstacle is located. When it is determined that the projection of the drone intersects with the obstacle, obtain the edge point on the obstacle that has the shortest distance to the center of the drone's projection; A collision circle with a preset safety radius is formed with the edge point as the center; The relative position between the drone and the obstacle is determined based on the position of the drone and the position of the obstacle; The drone's speed is obtained, and the drone's deflection angle is calculated based on the relative position and the drone's speed. Route planning is performed based on the collision circle, the drone's deflection angle, the axial distance, and the edge point, so that the drone can bypass the obstacle by passing through the boundary of the collision circle; Based on the collision circle and the center of the drone, determine the tangent from the center of the drone to the collision circle, and obtain the intersection point of the tangent and the collision circle; Control the drone to fly to the intersection point in the direction of the tangent; After reaching the intersection point, according to the safety radius, fly along the arc of the collision circle to the first target point where the drone's positive direction is parallel to the side of the obstacle; After reaching the first destination, fly along the flight direction from the first destination to the second destination; wherein, the second destination is the position where the side of the obstacle is no longer visible in the three-dimensional map; After reaching the second objective point, continue flying in the same direction as when you reached the second objective point to reach the third objective point at least a predetermined distance, thus completing the drone obstacle avoidance; Based on the collision circle and the center of the drone, the tangent line from the center of the drone to the collision circle is determined, and the intersection point of the tangent line and the collision circle is obtained as follows: Calculate the distance from the center of the drone to the center of the circle, based on the center of the circle. Based on the distance between the center of the drone and the center of the circle, and the axial distance between the drone and the obstacle, the first angle between the optical axis direction of the drone and the line connecting the drone to the center of the collision circle is obtained. By establishing the radius of the collision circle, the second included angle between the UAV and the tangent of the collision circle is obtained; The actual deflection angle of the UAV is obtained based on the first included angle and the second included angle, and the intersection point is determined based on the actual deflection angle and the collision circle.

2. The method for an unmanned aerial vehicle (UAV) to avoid obstacles according to claim 1, characterized in that, The safety radius d = 2M; where M is X times the larger of the length and width of the UAV; where X is greater than 2.

3. The method for an unmanned aerial vehicle (UAV) to avoid obstacles according to claim 1, characterized in that, In the predetermined distance, βω is the horizontal field of view of the three-dimensional map, d is the safety radius, and R is the predetermined distance.

4. The method for an unmanned aerial vehicle (UAV) to avoid obstacles according to claim 1, characterized in that, Acquire the target point of the drone and control the drone to fly in the direction of the target point after reaching the third target point.

5. A device for an unmanned aerial vehicle (UAV) to avoid obstacles, characterized in that, include: The reconstruction module is used to reconstruct a 3D map based on obstacle images captured by the drone's camera and the monocular VisualSLAM framework. Specifically, the process involves first extracting feature points from each frame of the image, performing coarse matching of feature points between adjacent frames, and then using a random sampling consensus algorithm to remove unreasonable matching pairs, thereby obtaining position and pose information. The monocular Visual SLAM framework includes sensor data, a front-end, a back-end, loop closure detection, and map building. The back-end primarily optimizes the results from the front-end to obtain the optimal pose estimation, using a nonlinear optimization method. It treats the optimized variables as nodes in the graph and the error terms as edges, iteratively optimizing and updating after initial values ​​are given. Due to the sparsity of graph optimization, it can reduce computational load while maintaining accuracy. The main purpose of loop closure detection is to enable the UAV to recognize places it has visited, thereby solving the problem of position drift over time. Loop closure detection is accomplished by judging the similarity between images, which is the same principle as humans using their eyes to judge two identical locations. Because the images are rich in information, VSLAM has a significant advantage in loop closure detection. When a loop closure is successfully detected, a correspondence is established between the current image and previously seen images. The back-end optimization algorithm readjusts the trajectory and map based on this information, thereby minimizing accumulated errors. Finally, a 3D map is built. To obtain the position of the drone and the axial distance between the drone and the obstacle; The judgment module is used to project the drone according to the drone's flight direction in order to determine whether the projection of the drone in the flight direction intersects with the plane where the obstacle is located. The acquisition module is used to acquire the edge point on the obstacle that has the shortest distance to the center of the drone's projection when it is determined that the drone's projection intersects with the obstacle; A forming module is used to form a collision circle with a preset safety radius, centered on the edge point; A relative position acquisition module is used to determine the relative position between the drone and the obstacle based on the position of the drone and the position of the obstacle; The calculation module is used to obtain the drone's travel speed and calculate the drone's deflection angle based on the relative position and the drone's travel speed. The obstacle bypass module is used to perform route planning based on the collision circle, the UAV deflection angle, the axial distance, and the edge point, so that the UAV can bypass the obstacle via the boundary of the collision circle; Based on the collision circle and the center of the drone, determine the tangent from the center of the drone to the collision circle, and obtain the intersection point of the tangent and the collision circle; Control the drone to fly to the intersection point in the direction of the tangent; After reaching the intersection point, according to the safety radius, fly along the arc of the collision circle to the first target point where the drone's positive direction is parallel to the side of the obstacle; After reaching the first destination, fly along the flight direction from the first destination to the second destination; wherein, the second destination is the position where the side of the obstacle is no longer visible in the three-dimensional map; After reaching the second objective point, continue flying in the same direction as when you reached the second objective point to reach the third objective point at least a predetermined distance, thus completing the drone obstacle avoidance; Based on the collision circle and the center of the drone, the tangent line from the center of the drone to the collision circle is determined, and the intersection point of the tangent line and the collision circle is obtained as follows: Calculate the distance from the center of the drone to the center of the circle, based on the center of the circle. Based on the distance between the center of the drone and the center of the circle, and the axial distance between the drone and the obstacle, the first angle between the optical axis direction of the drone and the line connecting the drone to the center of the collision circle is obtained. By establishing the radius of the collision circle, the second included angle between the UAV and the tangent of the collision circle is obtained; The actual deflection angle of the UAV is obtained based on the first included angle and the second included angle, and the intersection point is determined based on the actual deflection angle and the collision circle.

6. A terminal device, characterized in that, The device includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the method for an unmanned aerial vehicle to avoid obstacles as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device on which the computer-readable storage medium is located to perform the method of drone obstacle avoidance as described in any one of claims 1 to 4.

Images