Visual obstacle avoidance method and device for unmanned aerial vehicle

Abstract

The invention is suitable for the technical field of image processing, and provides a visual obstacle avoidance method and device for an unmanned aerial vehicle, and the method comprises the steps: collecting a plurality of real-time depth images based on a preset sampling frequency, and determining whether a dynamic obstacle exists or not according to the real-time depth images; when the existence of the dynamic obstacle is determined, constructing a current obstacle avoidance space sphere according to the depth value in the real-time depth image, the speed of the unmanned aerial vehicle and the actual estimated length of the obstacle; when the flight direction of the unmanned aerial vehicle intersects with the current obstacle avoidance space sphere, calculating an obstacle avoidance bypassing route based on the surface edge of the current obstacle avoidance space sphere; and controlling the unmanned aerial vehicle to perform obstacle avoidance flight based on the obstacle avoidance bypassing route. Compared with a traditional obstacle avoidance method, the method can more intelligently plan the detour route of the unmanned aerial vehicle, and guarantees the stability and safety of the flight process.

Description

Technical Field

[0001] This invention belongs to the technical field of image processing, and particularly relates to a visual obstacle avoidance method and device for unmanned aerial vehicles (UAVs). Background Technology

[0002] With the rapid development of drone technology, drones are increasingly being used in various fields, including but not limited to agriculture, logistics, security, film and television production, and entertainment. However, whether in complex urban environments or vast wilderness areas, drones may encounter various obstacles, which pose a significant threat to their safe flight. Therefore, how to effectively achieve obstacle avoidance for drones has become an urgent problem to be solved.

[0003] Traditional obstacle avoidance methods for drones primarily rely on devices such as LiDAR, ultrasonic sensors, or infrared sensors. While these devices can detect obstacles to some extent, they also have limitations. LiDAR, although providing high-precision distance information, is expensive and its performance is affected by adverse weather conditions such as fog and haze. Ultrasonic and infrared sensors are relatively weaker in terms of detection range and accuracy, and are easily interfered with in complex environments.

[0004] In recent years, with the development of computer vision technology, vision-based obstacle avoidance methods have gradually become a research hotspot. Visual obstacle avoidance methods acquire environmental images through cameras and use image processing algorithms to identify and locate obstacles, offering advantages such as low cost and wide applicability. However, existing visual obstacle avoidance methods still face certain challenges when dealing with dynamic obstacles, particularly in terms of real-time performance and obstacle avoidance path planning; a solution that can efficiently and reliably handle dynamic obstacles has yet to be found. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a visual obstacle avoidance method and apparatus for unmanned aerial vehicles (UAVs) to solve the technical problem that there is no solution that can efficiently and reliably handle dynamic obstacles in terms of real-time performance and obstacle avoidance path planning.

[0006] A first aspect of this invention provides a visual obstacle avoidance method for an unmanned aerial vehicle (UAV), the method comprising:

[0007] Multiple real-time depth images are acquired based on a preset sampling frequency, and the presence of dynamic obstacles is determined based on the real-time depth images.

[0008] When a dynamic obstacle is identified, a current obstacle avoidance space sphere is constructed based on the depth value in the real-time depth image, the drone speed, and the actual estimated length of the obstacle; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle;

[0009] When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, the obstacle avoidance detour route is calculated based on the surface edge of the current obstacle avoidance space sphere;

[0010] Based on the obstacle avoidance and detour route, the drone is controlled to perform obstacle avoidance flight.

[0011] Furthermore, the step of acquiring multiple real-time depth images based on a preset sampling frequency and determining the presence of dynamic obstacles based on the real-time depth images includes:

[0012] Multiple real-time depth images are acquired based on a preset sampling frequency, and the first spatial location data of the multiple real-time depth images acquired by the UAV is obtained; wherein, the first spatial location data includes longitude, latitude and altitude;

[0013] Edge detection is performed on the real-time depth image to obtain multiple image regions;

[0014] Based on the first spatial location data and the depth value corresponding to the image region, calculate the second spatial location data corresponding to the center of the image region;

[0015] If the distance between the second spatial location data corresponding to multiple real-time depth images exceeds a first threshold, then the image region is determined to be a dynamic obstacle.

[0016] If the distance between the second spatial location data corresponding to multiple real-time depth images does not exceed the first threshold, then the image region is determined to be a static obstacle.

[0017] Further, the step of calculating the second spatial location data corresponding to the center of the image region based on the depth value corresponding to the image region, the first spatial location data, and the depth value corresponding to the image region includes:

[0018] Obtain the camera's intrinsic parameter matrix, the UAV's rotation matrix, the UAV's translation vector, and the rotation matrix of the world coordinate system;

[0019] The coordinates of the center of the image region in the image coordinate system are converted to the first coordinates in the camera coordinate system using the intrinsic parameter matrix.

[0020] Based on the rotation matrix and translation vector of the UAV, the first coordinates are converted into second coordinates in the UAV coordinate system;

[0021] Based on the rotation matrix of the world coordinate system and the spatial position data, the second coordinates are converted into second spatial position data in the world coordinate system.

[0022] Furthermore, the step of constructing the current obstacle avoidance space sphere based on the depth value in the real-time depth image, the drone speed, and the actual estimated length of the obstacle when a dynamic obstacle is determined to exist includes:

[0023] When a dynamic obstacle is identified, the longest pixel distance between all edge pixels in the image region is extracted; wherein, the longest pixel distance is represented by the number of pixels.

[0024] Based on the first depth value corresponding to the center of the image region, the longest pixel distance, and camera parameters, the actual estimated length corresponding to the longest pixel distance is calculated; wherein, the camera parameters include focal length and sensor size;

[0025] Multiply the actual estimated length by the preset conversion coefficient to obtain the expansion coefficient;

[0026] The estimated impact time is obtained by dividing the minimum depth value among multiple depth values ​​corresponding to the image region by the current speed of the drone.

[0027] Multiply the estimated impact time by the preset obstacle avoidance coefficient to obtain the first value;

[0028] Obtain the radius of the obstacle avoidance space sphere corresponding to each of the multiple numerical ranges, and match the radius of the target obstacle avoidance space sphere corresponding to the numerical range in which the first value is located; wherein, the obstacle avoidance space sphere refers to a three-dimensional space sphere centered on the second spatial position data of the image region;

[0029] Multiply the radius of the target obstacle avoidance space sphere by the expansion coefficient to obtain the obstacle avoidance radius;

[0030] The spatial sphere corresponding to the obstacle avoidance radius is taken as the current obstacle avoidance spatial sphere.

[0031] Further, the step of calculating the actual estimated length corresponding to the longest pixel distance based on the first depth value corresponding to the center of the image region, the longest pixel distance, and camera parameters includes:

[0032] Calculate the camera angle of view based on the focal length and sensor size;

[0033] Pixel projection values ​​are calculated based on the camera viewpoint, the depth value corresponding to the image region, and the sensor size; wherein, the pixel projection values ​​are used to represent the actual length corresponding to each pixel;

[0034] Multiply the pixel projection value by the longest pixel distance and divide by the focal length to obtain the actual estimated length corresponding to the longest pixel distance.

[0035] Furthermore, the step of calculating the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere includes:

[0036] When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, multiple second spatial position data are fitted to obtain the obstacle movement direction;

[0037] Extract the vertical plane corresponding to the direction of movement of the obstacle;

[0038] Based on the vertical plane, the current obstacle avoidance space sphere is divided into two equal hemispheres;

[0039] The hemisphere that the obstacle's direction of movement passes through is designated as the risk hemisphere, and the hemisphere that the obstacle's direction of movement does not pass through is designated as the non-risk hemisphere.

[0040] Calculate the obstacle avoidance route based on the risk hemisphere and the non-risk hemisphere.

[0041] Furthermore, the step of calculating the obstacle avoidance route based on the risk hemisphere and the non-risk hemisphere includes:

[0042] If the flight direction intersects with the risk hemisphere, then obtain the two first intersection points between the flight direction and the current obstacle avoidance space sphere;

[0043] Construct a first plane containing the two first intersection points and the direction of the obstacle's movement;

[0044] In the first plane, extract two first straight lines that are perpendicular to each other and tangent to the arc of the non-risk hemispherical body;

[0045] Construct the second straight line corresponding to the two first intersection points;

[0046] In the triangle formed by the second straight line and the two first straight lines, the two right-angled sides are used as the first obstacle avoidance route;

[0047] If the flight direction intersects with the non-risk hemisphere, then obtain two second intersection points between the flight direction and the current obstacle avoidance space sphere;

[0048] Extract the shortest path between the two second intersection points on the surface edge of the non-risk hemisphere;

[0049] The shortest path is used as the second obstacle avoidance route.

[0050] A second aspect of the present invention provides a visual obstacle avoidance device for a drone, comprising:

[0051] The acquisition unit is used to acquire multiple real-time depth images based on a preset sampling frequency, and determine whether there are dynamic obstacles based on the real-time depth images;

[0052] A construction unit is used to construct a current obstacle avoidance space sphere based on the depth value in the real-time depth image, the speed of the UAV, and the actual estimated length of the obstacle when a dynamic obstacle is determined to exist; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle;

[0053] The calculation unit is used to calculate the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere;

[0054] The control unit is used to control the UAV to perform obstacle avoidance flight based on the obstacle avoidance detour route.

[0055] A third aspect of the present invention provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps in the visual obstacle avoidance method for the UAV described in the first aspect.

[0056] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the visual obstacle avoidance method for the UAV described in the first aspect.

[0057] The beneficial effects of this invention compared to existing technologies are as follows: When dynamic obstacles exist, a current obstacle avoidance space sphere is constructed based on the depth value in the depth image, the drone's speed, and the estimated actual length of the obstacle. Utilizing depth image data, precise perception of the obstacle's position and shape can be achieved, thereby constructing a more accurate obstacle avoidance model. Traditional obstacle avoidance methods based on lidar, ultrasonic, and infrared sensors often perform poorly when dealing with dynamic obstacles. This invention, by constructing an obstacle avoidance space sphere centered on the dynamic obstacle, can effectively cope with the obstacle's movement and changes. An obstacle avoidance detour route is calculated based on the intersection of the drone's flight direction and the current obstacle avoidance space sphere. This method can dynamically adjust the drone's flight path to avoid collisions with moving obstacles. By calculating the obstacle avoidance detour route based on the surface edges of the current obstacle avoidance space sphere, obstacles can be avoided and the optimal flight path selected. Compared to traditional obstacle avoidance methods, this invention can more intelligently plan the drone's detour route, ensuring a smooth and safe flight process. Controlling drones to perform obstacle avoidance flights based on calculated detour routes not only improves the success rate of obstacle avoidance but also optimizes the flight path, reducing detour time and energy consumption. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A schematic flowchart of a visual obstacle avoidance method for a drone provided by the present invention is shown.

[0060] Figure 2 A schematic diagram of a visual obstacle avoidance device for a drone according to an embodiment of the present invention is shown;

[0061] Figure 3 A schematic diagram of a terminal device provided in an embodiment of the present invention is shown. Detailed Implementation

[0062] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0063] This invention provides a visual obstacle avoidance method and apparatus for unmanned aerial vehicles (UAVs) to address the technical problem that there is currently no efficient and reliable solution for handling dynamic obstacles in terms of real-time performance and obstacle avoidance path planning.

[0064] First, this invention provides a visual obstacle avoidance method for unmanned aerial vehicles (UAVs). Please see [link to relevant documentation]. Figure 1 , Figure 1 A schematic flowchart of a visual obstacle avoidance method for unmanned aerial vehicles (UAVs) provided by the present invention is shown. Figure 1 As shown, the visual obstacle avoidance method of this drone may include the following steps:

[0065] Step 101: Acquire multiple real-time depth images based on a preset sampling frequency, and determine whether there are dynamic obstacles based on the real-time depth images;

[0066] The drone uses a depth camera to capture depth images of the environment at a certain frequency (preset sampling frequency). The depth image is an image that contains distance information.

[0067] By analyzing these depth images, drones can detect dynamic obstacles in the environment. Dynamic obstacles are moving objects that may pose a threat to the drone's flight path.

[0068] Specifically, step 101 includes steps 1011 to 1015:

[0069] Step 1011: Acquire multiple real-time depth images based on a preset sampling frequency, and obtain first spatial location data of the multiple real-time depth images acquired by the UAV; wherein, the first spatial location data includes longitude, latitude, and altitude;

[0070] The drone acquires depth images of the environment at a preset frequency, while simultaneously recording its spatial position and attitude data during image acquisition. The spatial position data includes longitude, latitude, and altitude, representing the drone's specific location in three-dimensional space. The attitude data includes yaw angle (the angle of the drone's forward direction relative to north), pitch angle (the angle of the drone's forward and backward tilt), and roll angle (the angle of the drone's left and right tilt). These data collectively describe the drone's orientation and attitude in space.

[0071] Step 1012: Perform edge detection on the real-time depth image to obtain multiple image regions;

[0072] Edge detection is performed on the acquired depth images to identify the edges and contours of different objects in the image, thereby segmenting multiple image regions. Each image region may represent a potential obstacle.

[0073] Step 1013: Calculate the second spatial location data corresponding to the center of the image region based on the first spatial location data and the depth value corresponding to the image region;

[0074] By using the spatial location data of the drone and combining it with the depth values ​​corresponding to the image regions, the specific location of the center point of each image region in three-dimensional space can be calculated. This location data can help determine the position and movement of obstacles.

[0075] Specifically, step 1013 includes steps 10131 to 10134:

[0076] Step 10131: Obtain the camera's intrinsic parameter matrix, the UAV's rotation matrix, the UAV's translation vector, and the rotation matrix of the world coordinate system;

[0077] First, obtain the camera's intrinsic parameter matrix (used to convert image coordinates to camera coordinates), the drone's rotation matrix (describing the drone's rotation in its coordinate system), the drone's translation vector (describing the drone's translation in its coordinate system), and the world coordinate system rotation matrix (describing the drone's position and attitude in the world coordinate system).

[0078] Step 10132: Convert the coordinates of the center of the image region in the image coordinate system to the first coordinates in the camera coordinate system using the intrinsic parameter matrix;

[0079] Using the camera's intrinsic parameter matrix, the coordinates of the center of the image region are transformed from the image coordinate system to the camera coordinate system. This step converts the image pixel coordinates into three-dimensional coordinates in the camera coordinate system, which are then calculated using the camera's intrinsic parameter matrix (including focal length, principal point coordinates, etc.).

[0080] Camera intrinsic parameter matrix Transform the coordinates (u, v) of the image region center to the coordinates (X, v) in the camera coordinate system. c ,Y c Z c The calculation process is as follows:

[0081]

[0082] f x and f y These are the focal lengths along the x and y axes of the imaging plane, respectively (pixel size is usually taken into account).

[0083] c x and c y These are the principal point coordinates, representing the position of the intersection of the optical axes in the image coordinate system.

[0084] Step 10133: Based on the rotation matrix and translation vector of the UAV, convert the first coordinates into the second coordinates in the UAV coordinate system;

[0085] By using the UAV's rotation matrix and translation vector, the first coordinate in the camera coordinate system is transformed to the UAV coordinate system. Specifically, the coordinates calculated in the camera coordinate system are rotated and translated to obtain the corresponding UAV coordinates.

[0086] The rotation matrix of the camera relative to the drone is R. cam2uav The translation vector is t cam2uav The coordinates (X) in the UAV coordinate system u ,Y u Z u ), calculated in the following way:

[0087]

[0088] Step 10134: Based on the rotation matrix of the world coordinate system and the spatial position data, convert the second coordinates into second spatial position data in the world coordinate system.

[0089] Finally, using the rotation matrix and translation vector of the world coordinate system, along with the UAV's spatial position data, the coordinates in the UAV coordinate system are transformed to the world coordinate system. In this way, the coordinates of the image region center are transformed from the image coordinate system through the camera coordinate system and the UAV coordinate system, and finally to the world coordinate system.

[0090] The UAV's attitude data provides the rotation matrix R from the UAV coordinate system to the world coordinate system. uav2world and the location of the drone (x) u ,y u ,z u Obstacle position in world coordinate system (X) w ,Y w Z w Calculated as follows:

[0091]

[0092] The attitude data of a UAV includes three angles: yaw (ψ), pitch (θ), and roll (φ), which represent the UAV's rotation about the vertical, horizontal, and longitudinal axes, respectively. These angles can be used to construct a rotation matrix from the UAV coordinate system to the world coordinate system.

[0093] Rotation about the Z-axis (yaw):

[0094] Rotation (pitch) around the Y-axis:

[0095] Rotation (roll) about the X-axis:

[0096]

[0097] In the embodiments corresponding to steps 10131 to 10134, the center point of a certain region in the image can be transformed from the image coordinate system to the world coordinate system. This transformation is crucial for the visual perception and obstacle avoidance of the UAV, because only in the world coordinate system can the actual position and motion state of the obstacle be accurately assessed, thereby making the correct obstacle avoidance decision.

[0098] Step 1014: If the distance between the second spatial location data corresponding to multiple real-time depth images exceeds the first threshold, then the image region is determined to be a dynamic obstacle;

[0099] By comparing the changes in the center point position of the same image region in depth images acquired at different time points, if the distance values ​​of these position changes exceed a preset threshold (first threshold), it can be determined that the image region represents a dynamic obstacle. This means that the object is moving and may pose a threat to the drone's flight path.

[0100] Step 1015: If the distance value between the second spatial location data corresponding to the multiple real-time depth images does not exceed the first threshold, then the image region is determined to be a static obstacle.

[0101] If the distance value of the position change does not exceed the preset threshold, the image area is considered to represent a static obstacle, that is, the object is fixed and poses little threat to the drone's flight.

[0102] In the embodiments corresponding to steps 1011 to 1015, these steps enable the UAV to more accurately identify and distinguish between dynamic and static obstacles, thereby performing more effective obstacle avoidance operations. This method significantly improves the safety and autonomy of the UAV in complex dynamic environments.

[0103] Step 102: When a dynamic obstacle is determined to exist, construct a current obstacle avoidance space sphere based on the depth value in the real-time depth image, the speed of the UAV, and the actual estimated length of the obstacle; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle;

[0104] Once a dynamic obstacle (such as another aircraft, bird, or moving structure) is identified, the drone will further process this information. The drone will use depth values ​​from the depth image (i.e., the distance to the obstacle), the drone's flight speed, and the estimated length of the obstacle (the actual estimated size of the obstacle obtained through image analysis) to construct a spatial sphere. This spatial sphere is a three-dimensional region representing the space the obstacle may occupy over a future period. Centered on the dynamic obstacle, this sphere reflects the area the drone needs to avoid.

[0105] Specifically, step 102 includes steps 1021 to 1028:

[0106] Step 1021: When a dynamic obstacle is determined to exist, extract the longest pixel distance between all edge pixels in the image region; wherein the longest pixel distance is represented by the number of pixels;

[0107] When a dynamic obstacle is identified, the edge pixels of the obstacle are extracted from the image, and the maximum distance between these edge pixels is calculated. This distance, expressed in pixels, represents the maximum extent of the obstacle in the image.

[0108] Step 1022: Calculate the actual estimated length corresponding to the longest pixel distance based on the first depth value corresponding to the center of the image region, the longest pixel distance, and the camera parameters; wherein, the camera parameters include focal length and sensor size;

[0109] Using the depth value at the center of the image region (representing the distance of the obstacle from the camera), the longest pixel distance (representing the extent of the obstacle in the image), and camera parameters (such as focal length and sensor size), the actual physical length corresponding to the longest pixel distance in the image is calculated.

[0110] Specifically, step 1022 includes steps 10221 to 10223:

[0111] Step 10221: Calculate the camera angle of view based on the focal length and sensor size;

[0112] The field of view (FOV) refers to the range of angles that a camera can see. It is closely related to the camera's focal length and sensor size.

[0113] By using the focal length and sensor size, the camera's angle of view in the horizontal and vertical directions can be calculated. This angle of view determines the actual physical range that the camera can cover within a certain distance.

[0114] Perspective: d h It is the horizontal dimension of the sensor, d vis the vertical dimension of the sensor, and f is the focal length of the camera.

[0115] Step 10222: Calculate the pixel projection value based on the camera viewpoint, the depth value corresponding to the image region, and the sensor size; wherein the pixel projection value is used to represent the actual length corresponding to each pixel;

[0116] The pixel projection value represents the length of each pixel in the actual physical world.

[0117] The actual length corresponding to each pixel:

[0118]

[0119] Z represents the depth value. N h This indicates the size of the sensor.

[0120] Step 10223: Multiply the pixel projection value by the longest pixel distance and divide by the focal length to obtain the actual estimated length corresponding to the longest pixel distance.

[0121] In the embodiments corresponding to steps 10221 to 10223, the camera's field of view is calculated to determine the angular range that the camera can cover; combined with the depth value, the projected length of each pixel in the actual physical world is calculated; the pixel projection length is multiplied by the maximum pixel distance in the image, and corrected by the focal length to finally obtain the actual physical length. This length is crucial for the UAV to accurately determine the size and position of obstacles during obstacle avoidance.

[0122] Step 1023: Multiply the actual estimated length by the preset conversion coefficient to obtain the expansion coefficient;

[0123] The calculated estimated length is multiplied by a preset conversion factor to obtain an expansion factor. This factor is used to expand the actual size of the obstacle to ensure sufficient safety distance during obstacle avoidance.

[0124] Step 1024: Divide the minimum depth value among the multiple depth values ​​corresponding to the image region by the current speed of the drone to obtain the estimated impact time;

[0125] Using the minimum depth value among multiple depth values ​​in the image region (representing the closest distance between the obstacle and the camera) and the drone's current speed, calculate the time it would take for the drone to collide with the obstacle while maintaining its current speed.

[0126] Step 1025: Multiply the estimated impact time by the preset obstacle avoidance coefficient to obtain the first value;

[0127] The estimated impact time is multiplied by a preset obstacle avoidance coefficient to obtain a first value. This step takes into account some additional safety factors to ensure that the calculated obstacle avoidance radius is more conservative.

[0128] Step 1026: Obtain the radius of the obstacle avoidance space sphere corresponding to each of the multiple numerical ranges, and match the radius of the target obstacle avoidance space sphere corresponding to the numerical range in which the first value is located; wherein, the obstacle avoidance space sphere refers to a three-dimensional space sphere centered on the second spatial position data of the image region;

[0129] Based on a predefined numerical range and the corresponding obstacle avoidance sphere radius, find the obstacle avoidance sphere radius corresponding to the first numerical value. The obstacle avoidance sphere is a three-dimensional sphere centered on the center of the image region, used for obstacle avoidance calculations in three-dimensional space.

[0130] Step 1027: Multiply the radius of the target obstacle avoidance space sphere by the expansion coefficient to obtain the obstacle avoidance radius;

[0131] The radius of the target obstacle avoidance sphere is multiplied by the previously calculated expansion coefficient to obtain the final obstacle avoidance radius. This step ensures that the radius of the obstacle avoidance sphere takes into account the actual size of the obstacle and safe expansion.

[0132] Step 1028: Use the spatial sphere corresponding to the obstacle avoidance radius as the current obstacle avoidance spatial sphere.

[0133] Finally, using the calculated obstacle avoidance radius, an obstacle avoidance spatial sphere centered on the spatial location of the image region is constructed. This is the current obstacle avoidance spatial sphere, used for obstacle avoidance decisions by the UAV.

[0134] In the embodiments corresponding to steps 1021 to 1028, these steps constitute a complete process that ensures that the UAV can accurately identify and avoid dynamic obstacles and dynamically adjust the obstacle avoidance strategy using depth image data, camera parameters, and UAV motion information.

[0135] Step 103: When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, calculate the obstacle avoidance detour route based on the surface edge of the current obstacle avoidance space sphere;

[0136] If the drone's current flight path intersects with this obstacle avoidance sphere (and it's about to collide with an obstacle), a new flight path needs to be calculated to avoid a collision. The drone will calculate a new flight path that circles around the edge of the sphere's surface. This means the drone will bypass the obstacle to ensure safety.

[0137] Specifically, step 103 includes steps 1031 to 1035:

[0138] Step 1031: When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, fit multiple second spatial position data to obtain the obstacle movement direction;

[0139] When the drone's flight path intersects with the obstacle avoidance sphere, obstacle position data (i.e., second spatial position data) is collected at multiple time points.

[0140] Using these data points, the direction of the obstacle's movement is calculated through fitting algorithms (such as linear regression, curve fitting, etc.). This direction represents the obstacle's tendency to move in space.

[0141] Step 1032: Extract the vertical plane corresponding to the direction of movement of the obstacle;

[0142] By calculating the obstacle's direction of movement, a plane perpendicular to that direction can be determined. This plane is perpendicular to the obstacle's direction of movement and is used to divide the obstacle avoidance sphere.

[0143] Step 1033: Based on the vertical plane, divide the current obstacle avoidance space sphere into two equal hemispheres;

[0144] Using the extracted vertical plane, the current obstacle avoidance space sphere (assumed to be a single sphere) is divided into two equal hemispheres. These two hemispheres are located on opposite sides of the vertical plane.

[0145] Step 1034: Designate the hemispheres through which the obstacle moves as risk hemispheres and the hemispheres through which the obstacle moves as non-risk hemispheres.

[0146] In the two hemispheres, find the hemisphere that the obstacle's moving direction passes through and mark it as the "risk hemisphere". The other hemisphere that is not passed through by the obstacle's moving direction is marked as the "non-risk hemisphere".

[0147] Step 1035: Calculate the obstacle avoidance route based on the risk hemisphere and the non-risk hemisphere.

[0148] To avoid obstacles, the drone should choose to detour through a non-risk hemisphere, thus avoiding the obstacle's path. Based on this principle, a specific detour route is calculated. This route should avoid entering the risky hemisphere as much as possible to ensure obstacle avoidance safety.

[0149] The detour route can be calculated based on optimization objectives such as shortest path, minimum energy consumption, and shortest time.

[0150] In the embodiments corresponding to steps 1031 to 1035, through the above steps, a safe obstacle avoidance route can be calculated based on the obstacle's movement direction when the UAV's flight direction intersects with the obstacle avoidance space sphere. This process involves: collecting multiple spatial position data of the obstacle and fitting its movement direction; extracting a plane perpendicular to the movement direction and dividing the obstacle avoidance space sphere into two hemispheres; determining which hemisphere is the risk hemisphere and which is the non-risk hemisphere; and calculating the UAV's detour route to ensure that the UAV avoids the obstacle's movement path, thereby achieving effective obstacle avoidance. Through these steps, the UAV's obstacle avoidance path can be effectively planned, improving flight safety and reliability.

[0151] Specifically, step 1035 includes steps 10351 to 10358:

[0152] Step 10351: If the flight direction intersects with the risk hemisphere, then obtain the two first intersection points between the flight direction and the current obstacle avoidance space sphere;

[0153] When the drone's flight path passes through the risk hemisphere, find two intersection points (called the first intersection points) between the flight path and the surface of the obstacle avoidance space sphere. These intersection points are the locations where the flight path intersects the sphere's surface.

[0154] Step 10352: Construct the first plane containing the two first intersection points and the direction of obstacle movement;

[0155] Using the two first intersection points and the direction of movement of the obstacle, a plane (called the first plane) is determined.

[0156] Step 10353: In the first plane, extract two first straight lines that are perpendicular to each other and tangent to the non-risk hemispherical arc;

[0157] Within the first plane, find two straight lines that are perpendicular to the direction of the obstacle's movement and tangent to the arc of the non-risk hemisphere (referred to as the first straight lines).

[0158] These straight lines are used to provide a reference direction for detour routes, ensuring that obstacles are avoided when detouring.

[0159] Step 10354: Construct the second straight line corresponding to the two first intersection points;

[0160] A straight line (called the second straight line) is determined by the two first intersection points. This straight line represents the projection of the UAV's current flight path onto the first plane.

[0161] Step 10355: In the triangle formed by the second straight line and the two first straight lines, the two right-angled sides are used as the first obstacle avoidance route;

[0162] The second straight line and the two first straight lines form a triangle in the first plane. Select the two right-angled sides of this triangle (i.e., the path perpendicular to the original flight direction and avoiding the direction of obstacle movement) as the first obstacle avoidance route.

[0163] By following these two straight paths, drones can effectively bypass obstacles.

[0164] Step 10356: If the flight direction intersects with the non-risk hemisphere, then obtain two second intersection points between the flight direction and the current obstacle avoidance space sphere;

[0165] When the drone's flight path passes through the non-risk hemisphere, find the two intersection points (called the second intersection points) between the flight path and the surface of the obstacle avoidance space sphere.

[0166] Step 10357: Extract the shortest path between the two second intersection points on the surface edge of the non-risk hemisphere;

[0167] On the surface of the non-risk hemisphere, find the shortest path connecting the two second intersection points. This shortest path is the optimal detour route for the drone to avoid obstacles within the obstacle avoidance space.

[0168] Step 10358: Use the shortest path as the second obstacle avoidance detour route.

[0169] The shortest path described above is used as the second obstacle avoidance route. Using this route, the drone can navigate around the surface of the non-risk hemisphere, avoiding entering the risky hemisphere and ensuring flight safety.

[0170] In the embodiments corresponding to steps 10351 to 10358, a plane containing these intersection points and the obstacle avoidance space sphere is constructed by determining the intersection points of the flight path and the obstacle avoidance space sphere. Within this plane, a straight line perpendicular to the obstacle's movement direction and tangent to the non-risk hemisphere is determined, forming a detour path. An effective first obstacle avoidance detour route is constructed by selecting the right-angled sides. The shortest path on the surface of the non-risk hemisphere is extracted by determining the intersection points of the flight path and the obstacle avoidance space sphere. This shortest path is used as the second obstacle avoidance detour route to ensure obstacle avoidance. These steps ensure that the UAV maintains an efficient and safe flight path while avoiding obstacles.

[0171] Step 104: Based on the obstacle avoidance and detour route, control the UAV to perform obstacle avoidance flight.

[0172] Finally, the drone will adjust its flight direction and speed based on the calculated detour route to avoid the obstacle. This step involves the drone's flight control system, which will perform the necessary flight adjustments to ensure the drone safely bypasses the obstacle and continues its planned route.

[0173] In the embodiments corresponding to steps 101 to 104, when dynamic obstacles exist, a current obstacle avoidance space sphere is constructed based on the depth value in the depth image, the drone's speed, and the actual estimated length of the obstacle. Utilizing depth image data, precise perception of the obstacle's position and shape can be achieved, thereby constructing a more accurate obstacle avoidance model. Traditional obstacle avoidance methods based on lidar, ultrasonic, and infrared sensors often perform poorly when dealing with dynamic obstacles. This invention, by constructing an obstacle avoidance space sphere centered on the dynamic obstacle, can effectively cope with the obstacle's movement and changes. An obstacle avoidance detour route is calculated based on the intersection of the drone's flight direction and the current obstacle avoidance space sphere. This method can dynamically adjust the drone's flight path to avoid collisions with moving obstacles. By calculating the obstacle avoidance detour route based on the surface edges of the current obstacle avoidance space sphere, obstacles can be avoided and the optimal flight path selected. Compared to traditional obstacle avoidance methods, this invention can more intelligently plan the drone's detour route, ensuring a smooth and safe flight process. Controlling drones to perform obstacle avoidance flights based on calculated detour routes not only improves the success rate of obstacle avoidance but also optimizes the flight path, reducing detour time and energy consumption.

[0174] like Figure 2 This invention provides a visual obstacle avoidance device for unmanned aerial vehicles (UAVs). Please refer to [link / reference]. Figure 2 , Figure 2 A schematic diagram of a visual obstacle avoidance device for a drone provided by the present invention is shown, such as... Figure 2 The visual obstacle avoidance device for a drone shown includes:

[0175] The acquisition unit 21 is used to acquire multiple real-time depth images based on a preset sampling frequency, and determine whether there are dynamic obstacles based on the real-time depth images;

[0176] The construction unit 22 is used to construct a current obstacle avoidance space sphere based on the depth value in the real-time depth image, the speed of the UAV, and the actual estimated length of the obstacle when it is determined that a dynamic obstacle exists; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle;

[0177] The calculation unit 23 is used to calculate the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere.

[0178] Control unit 24 is used to control the UAV to perform obstacle avoidance flight based on the obstacle avoidance detour route.

[0179] This invention provides a visual obstacle avoidance device for unmanned aerial vehicles (UAVs). When a dynamic obstacle exists, it constructs a current obstacle avoidance space sphere based on depth values ​​from a depth image, the UAV's speed, and the estimated actual length of the obstacle. Utilizing depth image data, it enables precise perception of the obstacle's position and shape, thereby constructing a more accurate obstacle avoidance model. Traditional obstacle avoidance methods based on lidar, ultrasonic, and infrared sensors often perform poorly when dealing with dynamic obstacles. This invention, by constructing an obstacle avoidance space sphere centered on the dynamic obstacle, effectively addresses the obstacle's movement and changes. An obstacle avoidance detour is calculated based on the intersection of the UAV's flight direction and the current obstacle avoidance space sphere. This method dynamically adjusts the UAV's flight path to avoid collisions with moving obstacles. By calculating the obstacle avoidance detour based on the surface edges of the current obstacle avoidance space sphere, obstacles can be avoided and the optimal flight path selected. Compared to traditional obstacle avoidance methods, this invention can more intelligently plan the UAV's detour route, ensuring a smooth and safe flight process. Controlling drones to perform obstacle avoidance flights based on calculated detour routes not only improves the success rate of obstacle avoidance but also optimizes the flight path, reducing detour time and energy consumption.

[0180] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3 As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a visual obstacle avoidance program for a drone. When the processor 30 executes the computer program 32, it implements the steps described in the various embodiments of the visual obstacle avoidance method for a drone, for example... Figure 1 Steps 101 to 104 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0181] For example, the computer program 32 can be divided into one or more units, which are stored in the memory 31 and executed by the processor 30 to complete the present invention. The one or more units can be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 32 in the terminal device 3. For example, the specific functions of each unit of the computer program 32 can be divided as follows:

[0182] The acquisition unit is used to acquire multiple real-time depth images based on a preset sampling frequency, and determine whether there are dynamic obstacles based on the real-time depth images;

[0183] A construction unit is used to construct a current obstacle avoidance space sphere based on the depth value in the real-time depth image, the speed of the UAV, and the actual estimated length of the obstacle when a dynamic obstacle is determined to exist; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle;

[0184] The calculation unit is used to calculate the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere;

[0185] The control unit is used to control the UAV to perform obstacle avoidance flight based on the obstacle avoidance detour route.

[0186] The terminal device includes, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of a terminal device 3 and does not constitute a limitation on a terminal device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.

[0187] The processor 30 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0188] The memory 31 can be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 3. Furthermore, the memory 31 can include both internal and external storage units of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0189] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0190] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0191] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0192] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0193] This invention provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0194] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0195] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0196] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0197] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0198] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units.

[0199] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0200] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0201] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [the described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [the described condition or event] is detected," or "in response to detection of [the described condition or event]."

[0202] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0203] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0204] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A visual obstacle avoidance method for unmanned aerial vehicles (UAVs), characterized in that, The visual obstacle avoidance method of the UAV includes: Multiple real-time depth images are acquired based on a preset sampling frequency, and the presence of dynamic obstacles is determined based on the real-time depth images. When a dynamic obstacle is identified, a current obstacle avoidance space sphere is constructed based on the depth value in the real-time depth image, the drone speed, and the actual estimated length of the obstacle; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle; When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, the obstacle avoidance detour route is calculated based on the surface edge of the current obstacle avoidance space sphere; Based on the obstacle avoidance and detour route, the drone is controlled to perform obstacle avoidance flight.

2. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 1, characterized in that, The step of acquiring multiple real-time depth images based on a preset sampling frequency and determining whether dynamic obstacles exist based on the real-time depth images includes: Multiple real-time depth images are acquired based on a preset sampling frequency, and the first spatial location data of the multiple real-time depth images acquired by the UAV is obtained; wherein, the first spatial location data includes longitude, latitude and altitude; Edge detection is performed on the real-time depth image to obtain multiple image regions; Based on the first spatial location data and the depth value corresponding to the image region, calculate the second spatial location data corresponding to the center of the image region; If the distance between the second spatial location data corresponding to multiple real-time depth images exceeds a first threshold, then the image region is determined to be a dynamic obstacle. If the distance between the second spatial location data corresponding to multiple real-time depth images does not exceed the first threshold, then the image region is determined to be a static obstacle.

3. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 2, characterized in that, The step of calculating the second spatial location data corresponding to the center of the image region based on the depth value corresponding to the image region, the first spatial location data, and the depth value corresponding to the image region includes: Obtain the camera's intrinsic parameter matrix, the UAV's rotation matrix, the UAV's translation vector, and the rotation matrix of the world coordinate system; The coordinates of the center of the image region in the image coordinate system are converted to the first coordinates in the camera coordinate system using the intrinsic parameter matrix. Based on the rotation matrix and translation vector of the UAV, the first coordinates are converted into second coordinates in the UAV coordinate system; Based on the rotation matrix of the world coordinate system and the spatial position data, the second coordinates are converted into second spatial position data in the world coordinate system.

4. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 1, characterized in that, The step of constructing the current obstacle avoidance space sphere based on the depth value in the real-time depth image, the drone speed, and the actual estimated length of the obstacle when a dynamic obstacle is determined to exist includes: When a dynamic obstacle is identified, the longest pixel distance between all edge pixels in the image region is extracted; wherein, the longest pixel distance is represented by the number of pixels. Based on the first depth value corresponding to the center of the image region, the longest pixel distance, and camera parameters, the actual estimated length corresponding to the longest pixel distance is calculated; wherein, the camera parameters include focal length and sensor size; Multiply the actual estimated length by the preset conversion coefficient to obtain the expansion coefficient; The estimated impact time is obtained by dividing the minimum depth value among multiple depth values ​​corresponding to the image region by the current speed of the drone. Multiply the estimated impact time by the preset obstacle avoidance coefficient to obtain the first value; Obtain the radius of the obstacle avoidance space sphere corresponding to each of the multiple numerical ranges, and match the radius of the target obstacle avoidance space sphere corresponding to the numerical range in which the first value is located; wherein, the obstacle avoidance space sphere refers to a three-dimensional space sphere centered on the second spatial position data of the image region; Multiply the radius of the target obstacle avoidance space sphere by the expansion coefficient to obtain the obstacle avoidance radius; The spatial sphere corresponding to the obstacle avoidance radius is taken as the current obstacle avoidance spatial sphere.

5. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 4, characterized in that, The step of calculating the actual estimated length corresponding to the longest pixel distance based on the first depth value corresponding to the center of the image region, the longest pixel distance, and camera parameters includes: Calculate the camera angle of view based on the focal length and sensor size; Pixel projection values ​​are calculated based on the camera viewpoint, the depth value corresponding to the image region, and the sensor size; wherein, the pixel projection values ​​are used to represent the actual length corresponding to each pixel; Multiply the pixel projection value by the longest pixel distance and divide by the focal length to obtain the actual estimated length corresponding to the longest pixel distance.

6. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 1, characterized in that, The step of calculating the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere includes: When the flight direction of the UAV intersects with the current obstacle avoidance space sphere, multiple second spatial position data are fitted to obtain the obstacle movement direction; Extract the vertical plane corresponding to the direction of movement of the obstacle; Based on the vertical plane, the current obstacle avoidance space sphere is divided into two equal hemispheres; The hemisphere that the obstacle's direction of movement passes through is designated as the risk hemisphere, and the hemisphere that the obstacle's direction of movement does not pass through is designated as the non-risk hemisphere. Calculate the obstacle avoidance route based on the risk hemisphere and the non-risk hemisphere.

7. The visual obstacle avoidance method for unmanned aerial vehicles as described in claim 6, characterized in that, The step of calculating the obstacle avoidance route based on the risk hemisphere and the non-risk hemisphere includes: If the flight direction intersects with the risk hemisphere, then obtain the two first intersection points between the flight direction and the current obstacle avoidance space sphere; Construct a first plane containing the two first intersection points and the direction of the obstacle's movement; In the first plane, extract two first straight lines that are perpendicular to each other and tangent to the arc of the non-risk hemispherical body; Construct the second straight line corresponding to the two first intersection points; In the triangle formed by the second straight line and the two first straight lines, the two right-angled sides are used as the first obstacle avoidance route; If the flight direction intersects with the non-risk hemisphere, then obtain two second intersection points between the flight direction and the current obstacle avoidance space sphere; Extract the shortest path between the two second intersection points on the surface edge of the non-risk hemisphere; The shortest path is used as the second obstacle avoidance route.

8. A visual obstacle avoidance device for unmanned aerial vehicles (UAVs), characterized in that, The visual obstacle avoidance device of the drone includes: The acquisition unit is used to acquire multiple real-time depth images based on a preset sampling frequency, and determine whether there are dynamic obstacles based on the real-time depth images; A construction unit is used to construct a current obstacle avoidance space sphere based on the depth value in the real-time depth image, the speed of the UAV, and the actual estimated length of the obstacle when a dynamic obstacle is determined to exist; wherein, the current obstacle avoidance space sphere refers to a space sphere centered on the dynamic obstacle; The calculation unit is used to calculate the obstacle avoidance route based on the surface edge of the current obstacle avoidance space sphere when the flight direction of the UAV intersects with the current obstacle avoidance space sphere; The control unit is used to control the UAV to perform obstacle avoidance flight based on the obstacle avoidance detour route.

9. A terminal device, characterized in that, The terminal device includes: a memory, a processor, and a visual obstacle avoidance program for the UAV stored in the memory and executable on the processor, the visual obstacle avoidance program for the UAV being configured to implement the steps in the visual obstacle avoidance method for the UAV as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps in the visual obstacle avoidance method for the UAV as described in any one of claims 1 to 7.

Images