Obstacle avoidance method, device, equipment, medium and product

Abstract

The embodiment of the invention discloses an obstacle avoidance method and device, equipment, a medium and a product. The obstacle avoidance method comprises the steps that a first open area in a first ground top view image is determined; under the condition that the available space of the first open area supports landing of the unmanned aerial vehicle, a first area corresponding to a second open area in the real world is determined, the unmanned aerial vehicle is controlled to fly to the position above the first area, and the second open area is an open area in a second ground top view image; the absolute value of the shooting angle of the second ground top view image is smaller than the absolute value of the shooting angle of the first ground top view image; and determining a landing area of the unmanned aerial vehicle in the first area according to the height of the unmanned aerial vehicle from the ground. According to the technical scheme, the second open area is determined in the first open area, the landing area of the unmanned aerial vehicle is determined according to the height of the unmanned aerial vehicle from the ground in the first area corresponding to the second open area in the real world, it is ensured that the unmanned aerial vehicle can safely descend to the safe area, and the adaptability of the unmanned aerial vehicle is enhanced.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the technical field of unmanned aerial vehicles, and particularly to an obstacle avoidance method, device, equipment, medium and product. Background Art

[0002] With the continuous development of unmanned aerial vehicle technology, ensuring that the unmanned aerial vehicle can land safely when the battery is low or a flight failure occurs has become a key issue. Traditional landing methods usually rely on GPS positioning and sensor data, but in complex environments, these methods are prone to errors, especially when the GPS signal is unstable or there are obstacles. Therefore, how to effectively combine multiple sensor technologies, improve the environmental perception ability of the unmanned aerial vehicle, and introduce a more intelligent autonomous decision-making system has become a technical problem that needs to be solved urgently. Summary of the Invention

[0003] Embodiments of the present disclosure provide an obstacle avoidance method, device, equipment, medium and product, which ensure that the unmanned aerial vehicle can complete an emergency landing in a safe manner and descend to a safe area, enhance the adaptability of the unmanned aerial vehicle in complex and dynamic environments, and improve the safety, accuracy and efficiency of the landing process.

[0004] In a first aspect, an obstacle avoidance method is provided, including:

[0005] Determine a first open area in a first ground top-down image;

[0006] When the available space in the first open area supports the landing of the unmanned aerial vehicle, determine a first area corresponding to a second open area in the real world, and control the unmanned aerial vehicle to fly above the first area, where the second open area is an open area in a second ground top-down image, and the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image;

[0007] According to the height of the unmanned aerial vehicle from the ground, determine the landing area of the unmanned aerial vehicle in the first area.

[0008] In a second aspect, an obstacle avoidance device is provided, including:

[0009] A first open area determination module, configured to determine a first open area in a first ground top-down image;

[0010] A second open area determination module, configured to determine a first area corresponding to a second open area in the real world and control the drone to fly above the first area when the available space in the first open area supports the landing of the drone. The second open area is an open area in a second ground top-down image, and the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image;

[0011] A landing area determination module, configured to determine a landing area of the drone in the first area according to the height of the drone from the ground.

[0012] In a third aspect, an electronic device is provided, including:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor is enabled to execute the obstacle avoidance method as described in the first aspect above.

[0016] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the obstacle avoidance method as described in the first aspect above is implemented.

[0017] In a fifth aspect, a computer program product is provided, the computer program product includes a computer program, and when the computer program is executed by a processor, the obstacle avoidance method as described in the first aspect above is implemented.

[0018] Embodiments of the present disclosure disclose an obstacle avoidance method, apparatus, device, medium and product. The method includes: determining a first open area in a first top-down ground image; when the available space in the first open area supports the landing of a drone, determining a first area in the real world corresponding to a second open area, and controlling the drone to fly above the first area, where the second open area is an open area in a second top-down ground image, and the absolute value of the shooting angle of the second top-down ground image is less than the absolute value of the shooting angle of the first top-down ground image; determining a landing area of the drone in the first area according to the height of the drone from the ground. The technical solution provided in this embodiment first determines a first open area in the first top-down ground image, determines a second open area in the first open area, and in the first area in the real world corresponding to the second open area, determines the landing area of the drone according to the height of the drone from the ground, ensuring that the drone can complete an emergency landing in a safe manner and descend to a safe area, enhancing the adaptability of the drone in complex and dynamic environments, and improving the safety, accuracy and efficiency of the landing process.

[0019] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the embodiments of the present disclosure. Other features of the embodiments of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0021] Figure 1 is a flowchart of an obstacle avoidance method provided in Embodiment 1 of the present disclosure;

[0022] Figure 2 is a schematic structural diagram of an SPP-U2Net image segmentation network provided in Embodiment 1 of the present disclosure;

[0023] Figure 3 is a schematic structural diagram of an SPP layer provided in Embodiment 1 of the present disclosure;

[0024] Figure 4 is a schematic structural diagram of a target yolov8 detection algorithm provided in Embodiment 1 of the present disclosure;

[0025] Figure 5 is a schematic structural diagram of a DSC-C2f provided in Embodiment 1 of the present disclosure;

[0026] Figure 6 It is a schematic structural diagram of a TTE module provided in the first embodiment of the present disclosure;

[0027] Figure 7 It is a schematic diagram of an obstacle avoidance process provided in the first embodiment of the present disclosure;

[0028] Figure 8 It is a schematic structural diagram of an obstacle avoidance device provided in the second embodiment of the present disclosure;

[0029] Figure 9 It is a schematic structural diagram of an electronic device provided in the third embodiment of the present disclosure. Detailed implementation manners

[0030] In order to enable those skilled in the art to better understand the solutions of the embodiments of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the embodiments of the present disclosure.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the embodiments of the present disclosure are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily need to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 It is a flowchart of an obstacle avoidance method provided in the first embodiment of the present disclosure. This embodiment is applicable to the situation of obstacle avoidance during the landing of a drone. This method can be executed by an obstacle avoidance device, which can be implemented in the form of hardware and / or software, and the obstacle avoidance device can be configured in an electronic device, and the electronic device includes, but is not limited to, devices with data processing capabilities such as computers, computers, terminals, and servers. As Figure 1 shown, this method includes:

[0034] S110. Determine the first open area in the first ground overhead image.

[0035] In this embodiment, the first ground overhead image can be obtained by using a binocular camera. Exemplarily, the binocular camera group of the unmanned aerial vehicle can be adjusted to face vertically downward (-90 degrees) to ensure that the camera can accurately acquire the ground image, facilitating subsequent analysis and judgment.

[0036] Continuing with the above description, after determining the first ground overhead image, the first open area in the first ground overhead image can be determined. The first open area can be an open area in the first ground overhead image that meets the second preset condition. Specifically, an image segmentation model can be used to segment the first ground overhead image, thereby determining multiple segmentation regions of the first ground overhead image.

[0037] Exemplarily, the image segmentation model can be called to process the obtained first ground overhead image, automatically identifying and segmenting multiple segmentation regions on the ground; among them, the image segmentation model can be the SPP-U2Net image segmentation network.

[0038] Figure 2 FIG. 13 is a schematic structural diagram of an SPP-U2Net image segmentation network provided in this embodiment, which is mainly responsible for efficiently extracting features and segmenting the input image. The core feature of this network is the introduction of a Spatial Pyramid Pooling (SPP) module and an attention mechanism to enhance the ability to capture context information. The overall network structure includes two main parts: an encoder and a decoder. The encoder consists of multiple stages (En-1 to En-5). Each stage gradually extracts multi-scale features of the input image through convolution and downsampling operations. And in the Stage4 (En-4) and Stage5 (En-5) stages, the SPP module is adopted. By performing grouped max-pooling operations on the feature maps of different stages, the context information of feature extraction is enhanced, and the segmentation accuracy of the network is improved. The decoder consists of multiple symmetric stages (De_1 to De_5). The spatial resolution of the feature map is gradually restored through upsampling and convolution operations. The attention module is adopted, replacing the Skip Connection in the original U2Net. The feature maps provided by the encoder and the decoder are fused by weighted averaging, further optimizing the process of feature fusion.

[0039] The working process of the SPP-U2Net image segmentation network is as follows: The input image first undergoes multi-stage convolution and downsampling operations of the encoder (En-1 to En-5) to gradually extract multi-scale features. Subsequently, in Stage4 and Stage5, the feature maps of the En_2, En_3, En_4 and En_3, En_4, En_5 encoders are respectively input into the max pooling layers with convolution kernel sizes of 2, 4, and 8 for multi-scale feature extraction to enhance the context information capture ability. Then, the attention mechanism function g(x) is introduced, and the formula is as follows:

[0040]

[0041] Among them, x E can be the feature map of the encoder, and x D can be the feature map of the decoder, and W q can be the matrix weight of the query, and W k can be the weight matrix of the key, and d k can be the dimension of the key. By calculating the inner product of the query matrix and the key matrix, the network can adjust the weight of each feature according to the similarity between the encoder and the decoder. Finally, the network concatenates these weighted feature maps (Sup1-Sup5) to obtain Sup0 and uses the Sigmoid function to output the final concatenation result S fuse .

[0042] Figure 3 Figure [ID] shows the structural schematic diagram of an SPP layer provided in this embodiment. As Figure 3 shown, it is usually used in a convolutional neural network (CNN) to process input images of different sizes and extract multi-scale features. The following is the explanation of each part in the figure: Feature Map (feature map): It is the output of a certain layer in the convolutional neural network, usually a three-dimensional tensor (height, width, number of channels). The feature map contains high-level features of the image. The feature map is fed into max pooling layers with three different kernel sizes to extract features of different scales. Figure 3There are three different max pooling layers, each using a kernel of a different size: the first max pooling layer uses a 2x2 pooling kernel; the second max pooling layer uses a 4x4 pooling kernel; and the third max pooling layer uses an 8x8 pooling kernel. Each max pooling operation downsamples the input feature map to extract features at different scales. Smaller pooling kernels retain more details, while larger pooling kernels capture more extensive context information. After the SPP layer, there is usually a convolutional layer. This convolutional layer further extracts and integrates features from the different-scale features output by the SPP layer. The SPP layer of this technical solution can process input images of any size and extract multi-scale features through pooling operations at different scales. This makes the network more adaptable to changes in the size of the input image and enables it to capture feature information at different levels.

[0043] Specifically, after obtaining multiple segmented regions of the first ground top-down image, the number of pixels in each segmented region can be determined. And the segmented region corresponding to the number of pixels that meets the second preset condition is used as the first open area; where the second preset condition can be the segmented region with the largest number of pixels. Exemplarily, the segmented region with the largest number of pixels can be determined as the first open area.

[0044] S120. When the available space in the first open area supports the landing of the drone, determine the first area corresponding to the second open area in the real world, and control the drone to fly above the first area. The second open area is the open area in the second ground top-down image, and the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image.

[0045] Specifically, when the available space in the first open area supports the landing of the drone, the shooting angle of the binocular camera can be adjusted to re-take the second ground top-down image, where the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image. Exemplarily, the angle of the drone binocular camera group can be adjusted to -45°, and then four pictures are taken by horizontal rotation. The four pictures are the second ground top-down image.

[0046] It should be noted that after determining the second ground top-down image, the second open area in the second ground top-down image can be determined. The second open area can be the open area in the second ground top-down image that meets the second preset condition. Specifically, the second ground top-down image can be segmented using an image segmentation model to determine multiple segmented regions of the second ground top-down image, and the segmented region with the largest number of pixels is determined as the second open area.

[0047] Continuing with the above description, after the second open area is determined, the pixel coordinates of the second open area can be converted into the actual spatial coordinates in the real world, so that the first area corresponding to the second open area in the real world can be determined. Exemplarily, the first area can be determined based on the average disparity value of the binocular camera, the focal length of the binocular camera, the baseline distance, and the principal point coordinates of the binocular camera. After the first area is determined, the drone can be controlled to fly above the first area. Exemplarily, the drone can be controlled to fly above the center of the first area.

[0048] S130. Determine the landing area of the drone in the first area according to the height of the drone from the ground.

[0049] In this embodiment, after the drone flies above the first area, the landing area of the drone can be determined according to the height of the drone from the ground.

[0050] This embodiment provides an obstacle avoidance method, including: determining a first open area in a first ground top-down image; when the available space in the first open area supports the landing of the drone, determining a first area corresponding to a second open area in the real world, and controlling the drone to fly above the first area, where the second open area is an open area in a second ground top-down image, and the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image; determining the landing area of the drone in the first area according to the height of the drone from the ground. The technical solution provided in this embodiment first determines the first open area in the first ground top-down image, determines the second open area in the first open area, and in the first area corresponding to the second open area in the real world, determines the landing area of the drone according to the height of the drone from the ground, ensuring that the drone can complete an emergency landing in a safe manner and descend to a safe area, enhancing the adaptability of the drone in complex and dynamic environments, and improving the safety, accuracy, and efficiency of the landing process.

[0051] As an optional implementation manner of this embodiment, when the available space in the first open area does not support the landing of the drone, the method further includes:

[0052] 1) Based on a third ground top-down image, use an image segmentation algorithm to determine a third open area; the shooting angle of the third ground top-down image is the same as the shooting angle of the second ground top-down image.

[0053] Specifically, in the case where the available space in the first open area does not support the landing of the drone, the shooting angle of the binocular camera can be adjusted to re-shoot the third ground top-down image, and multiple third ground top-down images can be obtained. The shooting angle of the third ground top-down image can be the same as that of the second ground top-down image. Exemplarily, the drone camera angle is adjusted to -45° and rotated horizontally to shoot four images, and then the third ground top-down image can be obtained.

[0054] Continuing with the above description, after obtaining the third ground top-down image, the third ground top-down image can be macroscopically monitored using an image segmentation algorithm. Among them, macroscopic detection can refer to the overall and large-scale analysis of an image to quickly extract the main structures, regional distributions, or key features in the image, rather than focusing on the detailed information in the image. Macroscopic detection is a process of initially understanding and classifying an image from a global perspective.

[0055] Continuing with the above description, through the image segmentation algorithm, macroscopic detection is performed on these four images to automatically identify and segment multiple segmented areas on the ground. Subsequently, the pixel ratio of each segmented area is calculated to determine the largest open area, and the largest open area is used as the third open area.

[0056] 2) Determine the second area of the third open area in the real world based on the parallax information of the binocular camera and the first target parameter; the first target parameter includes the focal length of the binocular camera and the baseline distance of the binocular camera.

[0057] Specifically, after the third open area is determined, the second area corresponding to the third open area in the real world can be determined based on the parallax information of the binocular camera and the first target parameter. Among them, the parallax information of the binocular camera is used to describe the horizontal position difference of an object in the same scene on the imaging planes of the left and right cameras.

[0058] Exemplarily, the parallax information of the binocular camera can be the average parallax value. The average parallax value can use the Oriented FAST and Rotated BRIEF (ORB) feature matching algorithm to extract and match the feature points of the regions in the two images taken by the binocular camera at the same time, obtain the matching regions, and then calculate the average parallax value of the feature matching pairs in the matching regions. The formula for the average parallax value is as follows:

[0059]

[0060] Among them, can be the horizontal coordinate of the pixel point of the feature matching point in the left image in the horizontal direction, can be the horizontal coordinate of the pixel point of the feature matching point in the right image in the horizontal direction.

[0061] According to the above description, the depth information of the matching region can be calculated using the average parallax value. The depth information can refer to the distance from the object to the camera, that is, the position information of the object in the three-dimensional space. Exemplarily, if the depth information is represented as Z, then:

[0062]

[0063] Among them, f can be the focal length of the camera, B can be the baseline distance between the binocular cameras, the baseline distance can be the horizontal distance between the optical centers of the two cameras in the binocular camera system, and d can be the average parallax value. After obtaining the depth information, the coordinates of each pixel point in the second region of the real world for the third empty region can be determined. Exemplarily, the pixel coordinates of the third empty region can be represented as (u, v), and its actual space coordinates in the second region can be represented as (X, Y, Z), then:

[0064]

[0065] Among them: (u, v) can be the pixel coordinates in the image; (c x , c y ) can be the principal point coordinates of the camera, where c x can be the horizontal coordinate of the principal point, and c y can be the vertical coordinate of the principal point. It should be noted that in the binocular vision system, each camera has its own principal point coordinates. In ORB feature matching and depth calculation, the principal point coordinates of both the left camera and the right camera usually need to be considered simultaneously. Therefore, when the third empty region belongs to the left camera, the principal point coordinates are the principal point coordinates of the left camera, and when the third empty region belongs to the right camera, the principal point coordinates are the principal point coordinates of the right camera; (f x , f y ) are the components of the focal length of the camera in the x and y directions.

[0066] 3) Adjust the flight path of the drone so that the drone flies above the second region, and return to execute the step of determining the first empty region in the first ground overhead image until the first empty region is sufficient.

[0067] It can be known that after the second region is determined, the flight path of the drone can be adjusted so that the drone flies above the second region, and return to execute the step of determining the first empty region in the first ground overhead image until the first empty region is sufficient. Exemplarily, the actual space coordinates of the second region can be represented as (X, Y, Z), and the drone can be guided to fly towards the target point (X, Y, Z), where (X, Y, Z) can be the actual space coordinates of the center point of the second region.

[0068] As an alternative implementation of this embodiment, determining the landing area of the drone in the first area according to the height of the drone from the ground includes:

[0069] 1) When the height of the drone from the ground reaches a first preset height and there are moving objects in the first area, determine an initial safe landing area based on the predicted movement trajectory of the moving objects, and control the drone to fly above the initial safe landing area; the initial safe landing area belongs to the coverage of the first area.

[0070] It should be noted that after the drone flies above the first area, the drone can start to descend, and during the descent process, obtain the height of the drone from the ground. When the height of the drone from the ground reaches the first preset height and there are moving objects in the first area, the predicted movement trajectory of the moving objects can be determined, and the initial safe landing area can be determined based on the predicted movement trajectory of the moving objects.

[0071] Among them, the first preset height can be a preset height. Exemplarily, the first preset height can be 20 meters. The initial safe landing area can belong to the coverage of the first area.

[0072] 2) When the height of the drone from the ground reaches a second preset height, determine the landing area based on a preset detection algorithm, and control the drone to land in the landing area; the landing area belongs to the coverage of the initial safe landing area; the second preset height is less than the first preset height.

[0073] In this embodiment, after the drone flies above the initial safe landing area, the drone can continue to descend, and during the descent process, obtain the height of the drone from the ground. When the height of the drone from the ground reaches the second preset height, determine the landing area based on a preset detection algorithm. Among them, the preset detection algorithm can be the target yolov8 detection algorithm, and the second preset height can be a preset height. Exemplarily, the second preset height can be 10 meters.

[0074] It should be noted that the target yolov8 detection algorithm adopted in this embodiment is an improved algorithm of the yolov8 detection algorithm. When the height of the drone from the ground reaches the second preset height, start to identify defects on the ground, and detect in real time whether there are potential obstacles or defects on the ground. According to the identification result, select a defect-free landing area to ensure that the drone can land safely and smoothly.

[0075] Figure 4 The structural schematic diagram of a target yolov8 detection algorithm provided for this embodiment is as Figure 4As shown, the improvement of the target YOLOv8 detection algorithm compared to the YOLOv8 detection algorithm lies in: using DSConv instead of traditional convolution operations in the backbone network, enhancing the model's ability to extract weak local structural features such as narrow cracks in the data. In the dynamic snake-shaped convolution network, the convolution kernel is deformed by the offset, and the offset can be oriented along the x-axis or y-axis, and the deformation range is specified by these parameters. By imposing continuity constraints, each position is associated with the previous position, enabling the free selection of the deformation direction.

[0076] Meanwhile, the DSConv module is also used to replace the original convolution module in the C2f module. Figure 5 The following is a schematic diagram of the structure of DSC-C2f provided in this embodiment, as Figure 5 shown, each depth convolution (DSConv) module receives the feature map from the previous layer as input. Through the combination of depth convolution and pointwise convolution, the DSConv module reduces the amount of computation and the number of parameters while reducing the computational complexity and improving the detection speed. At the same time, it also effectively enhances the model's feature extraction ability, enabling the model to more accurately distinguish different categories or scenarios. Finally, the scale sequence feature fusion (SSFF) module and the triple feature encoding (TFE) module are used to improve the neck of the network, enhancing the positioning and recognition accuracy of targets at different scales.

[0077] During the road surface defect detection process, the SSFF module effectively fuses features from different scales, avoiding redundant calculations, so that the model can comprehensively capture the details of road defects and their context information. The SSFF module is designed based on the feature map of stage 3 because the stage 3 feature map can provide more comprehensive and accurate information during the detection process. First, the number of channels of the two highest-level feature maps is adjusted to 256, and their spatial dimensions are adjusted to match the spatial dimensions of stage 3. Then, the unfolding method is used to flatten the tensor shape and concatenate it along the depth dimension. Finally, three-dimensional convolution, batch normalization, and the SiLU activation function are used to complete the feature extraction of the SSFF module. The concatenated feature map not only has the same resolution but also contains feature information of different scales, forming a multi-scale feature sequence, thus enhancing the model's sensitivity to targets at different scales.

[0078] To further enhance the feature extraction ability, the TTE module is introduced in the target YOLOv8 detection algorithm. Figure 6 The following is a schematic diagram of the structure of a TTE module provided in this embodiment, as Figure 6As shown, the TTE module enhances the expression of multi-scale features by separating operations onto three feature maps of different sizes. First, convolutional operations are applied to the large-scale, medium-scale, and small-scale feature maps. Then, through downsampling or upsampling operations, the sizes of the feature maps of different scales are adjusted. Finally, the feature maps of all scales undergo another convolutional operation and are concatenated along the channel dimension. Among them, when operating at each scale, it can go through the processing of a composite module, and the operations that the composite module can perform include convolutional layers, normalization layers, and activation functions; among them, the convolutional layer is used for feature extraction, the normalization layer is used to normalize the feature data to accelerate training and stabilize the network, and the activation function has a certain inhibitory effect on inputs less than 0 but does not completely zero them, thus retaining more information. This effectively avoids the problem of detail loss or blurring in small-scale feature maps caused by upsampling and cascading operations during the feature propagation of the FPN (Feature Pyramid Network), and improves the retention and expression of detail information.

[0079] It should be noted that when the height of the drone from the ground reaches the second preset height, multiple temporally consecutive images of the initial safe landing area can also be obtained; each frame of the image is divided into multiple small regions according to the preset grid size, and each grid will be assigned different scores according to the detection results of the defect model during the scoring process. According to the final score, the grid with the highest score is selected as the landing area of the drone. If there are multiple grids with the same score, the grid closest to the current drone is selected as the landing area of the drone.

[0080] Exemplarily, if the grid contains a pit or a ground protrusion (i.e., the pixel box of this area covers the pit or protrusion), the score of this grid is 0 points, indicating that this area is seriously unsuitable for landing. If the grid contains a crack (i.e., the pixel box of this area covers the crack), the score of this grid is 50 points, indicating that this area is not completely safe and is suitable for avoiding landing. If the grid is other normal areas without special obstacles (i.e., there are no pits, ground protrusions, or cracks), the score of this grid is 100 points, indicating that this area is completely suitable for landing.

[0081] As an optional implementation manner of this embodiment, the obstacle avoidance method provided in this embodiment further includes:

[0082] 1) When the height of the drone from the ground reaches the first preset height, obtain multiple temporally consecutive target images of the first area.

[0083] Specifically, when the height of the drone from the ground reaches the first preset height, a binocular camera can be used to capture images of the first area at consecutive times, and thus multiple temporally consecutive images of the first area can be obtained.

[0084] 2) Determine the pixel difference between two consecutive target images.

[0085] Specifically, the pixel difference between two consecutive target images can be determined. Exemplarily, when the drone descends to a height of 20 meters from the ground, consecutive target images of the ground (i.e., the first area) can be captured, and the frame extraction operation can be initiated. For each frame of the image, the frame difference method can be used to calculate the pixel difference between two consecutive frames. Among them, the frame difference method can detect moving objects by using the pixel gray difference between two adjacent frames or multiple frames in a video or image sequence.

[0086] 3) When the pixel difference satisfies the first preset condition, determine that there is a dynamic object in the first area.

[0087] It can be known that when the pixel difference between two consecutive target images satisfies the first preset condition, it can be determined that a dynamic object exists.

[0088] As an optional implementation manner of this embodiment, determining the initial safe landing area based on the predicted motion trajectory of the dynamic object includes:

[0089] 1) For any one target image, divide the target image to determine multiple regions of interest.

[0090] Specifically, for any one target image, the frame extraction operation can be initiated, and each frame of the image can be divided into multiple small regions according to a preset grid size. Each small region can be called a region of interest.

[0091] 2) Determine the predicted trajectory of the moving object based on the contour and trajectory prediction model of the dynamic object; the trajectory prediction model is trained based on historical trajectory data; the contour of the dynamic object is determined based on a preset contour determination algorithm;

[0092] In this embodiment, the contour of the dynamic object can be determined based on a preset contour determination algorithm. Exemplarily, the preset contour determination algorithm can be the fast level set algorithm. The specific operation includes: for any one target image, the contour of the moving object in the target image can be regarded as an arbitrary irregular closed curve on the plane, and its two-dimensional implicit expression can be represented as:

[0093]

[0094] Among them, C(t) can represent an irregular closed curve that changes with time, c can be a constant, representing the value, and C is a set composed of all points. Among them, as time changes, the closed curve can be expressed as a two-dimensional level set function that changes with time C(t) can evolve according to where F represents the evolution speed of the curve and N represents the normal direction of evolution. The expression of F can be:

[0095]

[0096] That is, F can be understood as the rate of change of the level set function with respect to time t. The expression of N can be:

[0097]

[0098] where can be a vector composed of the partial derivatives of the function with respect to x and y.

[0099] Specifically, first, each target image is grayscale processed, and target detection is performed using the grayscale differences between the target images. The grayscales of two adjacent frames of target images are subtracted to obtain a difference image. A suitable threshold is set, and the difference image is binarized and connected to complete the initial segmentation of the dynamic object and the background. The segmentation result is the set of pixel coordinates of the dynamic object. The obtained segmentation result can be used as the initialization of the level set function Then, during the curve evolution process, the changing points on the edge curve are solved. The level set method can dynamically adjust the target contour through curve evolution and edge point update. The outer edge and inner edge of the curve are defined as follows:

[0100]

[0101] where u and v respectively represent a block of pixels in the target image, and N4(u) represents the four-neighborhood of u. Subsequently, the fast level set sign function can be defined as The difference image is divided into four parts: the external region, the external edge, the inner edge region, and the internal region.

[0102]

[0103] It should be noted that if the pixel coordinates of u are (x, y), the four-neighborhood of u can be expressed as: {(x - 1, y), (x + 1, y), (x, y - 1), (x, y + 1)}. outside(C) can represent the external region of the contour curve of the dynamic object, and inside(C) can represent the internal region of the contour curve of the dynamic object. Specifically, the level set function and the contour evolution direction where can be understood as evolving along the normal direction N, and the specific direction depends on the evolution speed F. When F > 0 When F < 0 When F = 0, During the evolution process, count the points falling on the outer edge L out and the inner edge L in Next, calculate the curve evolution speed F and the evolution direction corresponding to each point on the outer edge L out and the inner edge L in Finally, update the points on the outer edge and the inner edge in the following way: Finally, update the points on the outer edge and the inner edge in the following way:

[0104] a1) Traverse all the points on the outer edge, and place the point u with the evolution direction onto L in ; where, can represent each evolution direction at point u;

[0105] b1) Traverse all the points on the inner edge. If in the 4-neighborhood of point u satisfies then it means that point u is in the internal region at this time;

[0106] c1) Traverse all the points on the inner edge, and place the point u with the evolution direction onto L out ;

[0107] d1) Traverse the points on the outer edge. If in the 4-neighborhood of point u satisfies then it means that point u is in the external region at this time.

[0108] Through the above steps, the boundary of the dynamic object can be dynamically adjusted according to the evolution direction and the parallax information to ensure the accuracy and real-time performance of the dynamic object contour.

[0109] It should be noted that the statistical histogram method can also be used to perform similarity matching between the segmented region obtained from the current target image and the corresponding segmented region of the previous frame of the target image. The correlation coefficient ρ is obtained through the formula. Those with a large correlation coefficient are considered to be the same target, and those with a small correlation coefficient are considered to be different targets.

[0110]

[0111] where, N represents the gray level, can represent the average gray level of the corresponding region of the current target image, can represent the average gray level of the corresponding region of the previous frame of the target image. x i can represent the gray value under the i-th segmented region obtained from the current image, and y i can represent the gray value under the i-th segmented region of the previous frame of the target image.

[0112] According to the above formula, the statistical histogram method is used to perform similarity matching between the currently obtained segmented region and each target in the previous frame. By calculating the correlation coefficient ρ, it is determined whether it is the same dynamic object. The central position of the region corresponding to the dynamic object or the position of the feature points is recorded, and the positions in each frame are connected in sequence to form the motion trajectory of the dynamic object, so as to realize the tracking of the moving object.

[0113] Based on the contour of the dynamic object, the motion trajectory of the dynamic object can be determined, and the motion trajectory of the dynamic object is input into the trajectory prediction model to determine the predicted trajectory of the moving object; the trajectory prediction model is trained based on historical trajectory data. Exemplarily, the trajectory prediction model can be a Long Short-Term Memory (LSTM) network. The LSTM network has unique advantages in processing time series data and can effectively capture the long-term dependencies in the target motion process. Through training, the LSTM network can learn the laws of target motion and finally generate an accurate trajectory prediction model.

[0114] 3) Score multiple regions of interest based on the contour of the dynamic object and the predicted trajectory to determine the scoring result; the scoring result is determined based on a preset scoring algorithm.

[0115] Specifically, after obtaining the predicted trajectory, multiple regions of interest can be scored based on the contour of the dynamic object and the predicted trajectory to determine the scoring result. Among them, the scoring result can be determined by a scoring propagation algorithm based on breadth-first search. The scoring propagation algorithm based on breadth-first search can be

[0116] Exemplarily, the score can be determined according to whether the contour and predicted trajectory of the dynamic object exist in each region of interest. Specifically, the 0-point region (dangerous region): that is, if the region of interest is directly passed by the contour of the dynamic object or its predicted trajectory, the score of this region of interest is 0 points, indicating that this region of interest is not suitable for landing; the 10-point region (near dangerous region): if the region of interest is adjacent to the grid with a score of 0 points, the score of this region of interest is 10 points, indicating that this region of interest is less affected by the moving target or trajectory, but not completely safe; the 20-point region (farther safe region): if the region of interest is separated from the region of interest with a score of 0 points by one region of interest (that is, there is a distance of one region of interest between the two regions of interest), the score of this region of interest is 20 points, indicating that this region of interest is relatively safe and suitable for landing; as the distance increases, the score of the region of interest gradually increases, ensuring that the drone preferentially selects the region that is farthest from the contour and predicted trajectory of the dynamic object for landing.

[0117] 4) Determine the initial safe landing area based on the scoring result.

[0118] Specifically, after the scoring result is determined, the initial safe landing area can be determined based on the scoring result; exemplarily, the area of the real world corresponding to the region of interest with the highest scoring result can be determined as the initial safe landing area. It should be noted that if there are multiple regions of interest with the same score, the area of the real world corresponding to the region of interest closest to the current UAV position is selected as the initial safe landing area.

[0119] As an optional implementation manner of this embodiment, the obstacle avoidance method provided in this embodiment further includes:

[0120] 1) Determine the area of the first open area based on the second target parameters of the binocular camera and the resolution of the first top-down ground image; the second target parameters include the width of the binocular camera and the height of the binocular camera.

[0121] In this embodiment, the second target parameters may be the width of the binocular camera and the height of the binocular camera, and the area of the first open area may be determined based on the second target parameters of the binocular camera and the resolution of the first top-down ground image. Specifically, the horizontal field of view angle and the vertical field of view angle can be calculated using the width of the binocular camera, the height of the binocular camera, and the focal length of the binocular camera. Exemplarily, the horizontal field of view angle can be expressed as θ h , and the vertical field of view angle can be expressed as θ v , then there are:

[0122]

[0123] where, w s can represent the width of the binocular camera, h s can represent the height of the binocular camera, and f can represent the focal length of the binocular camera.

[0124] Continuing with the above description, after the horizontal field of view angle and the vertical field of view angle are determined, the area of the first open area can be determined based on the horizontal field of view angle, the vertical field of view angle, and the resolution of the first top-down ground image. Exemplarily, the resolution of the first top-down ground image can be expressed as W×H, and the area of the first open area can be expressed as S, then there are:

[0125]

[0126] where, h s can be expressed as representing the height of the binocular camera, and α can represent the number of pixels

[0127] 2) If the area meets the third preset condition, it is determined that the available space in the first open area supports the landing of the UAV; if the area does not meet the third preset condition, it is determined that the available space in the first open area does not support the landing of the UAV.

[0128] It can be known that after the area of the first open area is determined, it can be determined whether the area meets the third preset condition. Among them, the third preset condition can be that the area of the first open area is greater than a preset threshold value, and the preset threshold value can be a threshold value set in advance. If the area of the first open area meets the third preset condition, that is, the area of the first open area is greater than the preset threshold value, it is determined that the available space of the first open area can support the landing of the drone; if the area does not meet the third preset condition, that is, the area of the first open area is less than and / or equal to the preset threshold value, it is determined that the available space of the first open area does not support the landing of the drone.

[0129] Figure 7 The figure is a schematic diagram of an obstacle avoidance process provided in this embodiment. As Figure 7 shown, when the drone needs to initiate an emergency landing due to low battery or other reasons, the obstacle avoidance process includes the following steps:

[0130] (1) Adjust the binocular camera group of the drone to face vertically downward (-90 degrees) to ensure that the camera can accurately obtain ground images for subsequent analysis and judgment.

[0131] (2) Call the image segmentation model to process the acquired image and automatically identify and segment the open areas on the ground;

[0132] (3) According to the image segmentation result, evaluate whether the area of the open area (the first open area) is greater than the threshold value. If it is not greater, the open area (the first open area) does not support the landing of the drone, then activate the obstacle avoidance mechanism and execute step 4 to search for and select the most suitable safe area for landing; if it is greater, start the emergency landing procedure and execute step 5.

[0133] (4) The open area (the first open area) does not support the landing of the drone. If no suitable area is screened out in the previous steps, the system automatically adjusts the camera angle of the drone to -45° and horizontally rotates to take four images (the third ground top-down image). Through macroscopic detection by the image segmentation algorithm, identify the largest open area (the third open area), and use the binocular camera to convert the pixel coordinates into actual space coordinates to determine the second area. The drone automatically adjusts its flight path according to the calculation result, flies above the second area and re-executes step 1.

[0134] (5) If a suitable target area is detected in the previous steps, use the method in step 4 to convert the extracted pixel coordinates into actual space coordinates, so as to accurately determine the position of the first area. According to the calculation result, the drone will automatically adjust its flight path and fly precisely to the center position of the first area, and the drone starts to descend.

[0135] (6) After the UAV descends to 20 meters (the first preset height), it starts to extract image frames, identify and track moving objects, update the position information in real time, and use the trajectory prediction model to predict the future trajectory of the objects. Based on the future trajectory of the objects, each area of the grid-like image is scored. According to the scoring results, the area with the highest score is selected and converted into spatial coordinates, and the flight path is adjusted to fly precisely above the center of the area in the real world corresponding to the area with the highest score (the initial safe landing area).

[0136] (7) When the UAV descends to 10 meters above the ground, it starts to identify defects on the ground, calls the defect recognition model to detect in real time whether there are potential obstacles or defects on the ground. According to the recognition results, a flat road surface (landing area) is selected to ensure that the UAV can land safely and smoothly.

[0137] This technical solution proposes an emergency safety landing obstacle avoidance strategy for UAVs. Through vertical downward shooting and real-time image segmentation, the system can accurately identify open areas on the ground and evaluate their usability, avoiding the errors caused by traditional GPS dependence and ensuring a safe landing. During the landing process, if it is detected that there is insufficient suitable area, the UAV will automatically adjust the camera angle and expand the search range, and accurately identify the optimal landing point in combination with the image segmentation algorithm. At the same time, the interference of moving objects is also considered. Through dynamic obstacle avoidance and object tracking functions, potential threats are predicted in real time during the descent process and the flight path is adjusted to avoid the risk of collision. In addition, the system can also detect ground defects at a height of 10 meters to avoid accidents caused by obstacles or uneven ground. By comprehensively using binocular cameras, advanced image processing algorithms and intelligent decision-making systems, the dependence on external signals such as GPS is reduced, the adaptability of the UAV in complex and dynamic environments is enhanced, and the safety, accuracy and efficiency of the landing process are improved.

[0138] Embodiment 2

[0139] Figure 8 is a schematic structural diagram of an obstacle avoidance device provided by the second embodiment of the present disclosure; as Figure 8 shown, the device includes: a first open area determination module 210, a second open area determination module 220, and a landing area determination module 230.

[0140] Among them, the first open area determination module 210 is used to determine the first open area in the first top-down ground image;

[0141] A second open area determination module 220, configured to determine a first area corresponding to a second open area in the real world and control the drone to fly above the first area when the available space in the first open area supports the landing of the drone, where the second open area is an open area in a second ground top-down image, and the absolute value of the shooting angle of the second ground top-down image is less than the absolute value of the shooting angle of the first ground top-down image;

[0142] A landing area determination module 230, configured to determine a landing area of the drone in the first area according to the height of the drone from the ground.

[0143] Embodiment 2 of the present disclosure provides an obstacle avoidance device, which ensures that the drone can complete an emergency landing in a safe manner and descend to a safe area, enhances the adaptability of the drone in a complex and dynamic environment, and improves the safety, accuracy, and efficiency of the landing process.

[0144] Furthermore, the device further includes:

[0145] A third open area determination module, configured to determine a third open area based on a third ground top-down image by using an image segmentation algorithm; the shooting angle of the third ground top-down image is the same as the shooting angle of the second ground top-down image;

[0146] A second area determination module, configured to determine a second area in the real world of the third open area based on the parallax information of the binocular camera and a first target parameter; the first target parameter includes the focal length of the binocular camera and the baseline distance of the binocular camera;

[0147] A path adjustment module, configured to adjust the flight path of the drone so that the drone flies above the second area and return to execute the step of determining the first open area in the first ground top-down image until the first open area is sufficient.

[0148] Furthermore, the landing area determination module 230 further includes:

[0149] An initial safe landing area determination unit, configured to determine an initial safe landing area based on the predicted movement trajectory of the dynamic object and control the drone to fly above the initial safe landing area when the height of the drone from the ground reaches a first preset height and there is a dynamic object in the first area; the initial safe landing area belongs to the coverage range of the first area;

[0150] A determination unit, configured to determine the landing area based on a preset detection algorithm and control the UAV to land in the landing area when the height of the UAV from the ground reaches a second preset height; the landing area belongs to the coverage range of the initial safe landing area; the second preset height is less than the first preset height.

[0151] Furthermore, the device further includes:

[0152] A target image acquisition module, configured to acquire a plurality of temporally consecutive target images of the first area when the height of the UAV from the ground reaches a first preset height;

[0153] A pixel difference determination module, configured to determine the pixel difference between two consecutive target images;

[0154] A moving object determination module, configured to determine that there is a moving object in the first area when the pixel difference meets a first preset condition.

[0155] Furthermore, the initial safe landing area determination unit is further configured to:

[0156] For any one of the target images, divide the target image to determine a plurality of regions of interest;

[0157] Determine the predicted trajectory of the moving object based on the contour and trajectory prediction model of the moving object; the trajectory prediction model is trained based on historical trajectory data; the contour of the moving object is determined based on a preset contour determination algorithm;

[0158] Score the plurality of regions of interest based on the contour and the predicted trajectory of the moving object to determine a first scoring result;

[0159] Determine the initial safe landing area based on the first scoring result and a second scoring result; the second scoring result is determined based on a preset scoring algorithm.

[0160] Furthermore, the device further includes:

[0161] An area determination module, configured to determine the area of the first open area based on the second target parameter of the binocular camera and the resolution of the first top-down ground image; the second target parameter includes the width and height of the binocular camera;

[0162] An available space determination module, configured to determine that the available space in the first open area supports the landing of the UAV if the area meets a third preset condition; and determine that the available space in the first open area does not support the landing of the UAV if the area does not meet the third preset condition.

[0163] The obstacle avoidance device provided by the embodiments of the present disclosure can execute the obstacle avoidance method provided by any embodiment of the embodiments of the present disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0164] Embodiment III

[0165] Figure 9 The structural schematic diagram of an electronic device 10 that can be used to implement the embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of the present disclosure described and / or claimed herein.

[0166] As Figure 9 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0167] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0168] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microprocessor, etc. The processor 11 executes the various methods and processes described above, such as the obstacle avoidance method.

[0169] In some embodiments, the obstacle avoidance method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by the processor 11, one or more steps of the obstacle avoidance method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the obstacle avoidance method by any other suitable means (e.g., by means of firmware).

[0170] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0171] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on the remote machine or server.

[0172] In the context of the embodiments of the present disclosure, a computer-readable storage medium may be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium may be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0173] In order to provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0174] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of the communication network include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0175] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of large management difficulty and weak business scalability existing in traditional physical hosts and VPS services. It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the embodiments of the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions of the embodiments of the present disclosure can be achieved, and no limitation is made herein.

[0176] The above specific implementation manners do not constitute a limitation to the protection scope of the embodiments of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the embodiments of the present disclosure shall be included within the protection scope of the embodiments of the present disclosure.

[0177] The embodiments of the present disclosure also provide a computer program product, including a computer program and / or instructions, and the computer program, when executed by a processor, implements the obstacle avoidance method provided in any embodiment of the present application.

[0178] In the process of implementing the computer program product, computer program code for performing the operations of the embodiments of the present disclosure can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0179] Note that the above are only the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art will understand that the embodiments of the present disclosure are not limited to the specific embodiments here, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the embodiments of the present disclosure. Therefore, although the embodiments of the present disclosure have been described in more detail through the above embodiments, the embodiments of the present disclosure are not limited to the above embodiments only. Without departing from the concept of the embodiments of the present disclosure, more other equivalent embodiments can be included, and the scope of the embodiments of the present disclosure is determined by the scope of the appended claims.

Claims

1. An obstacle avoidance method, characterized in that, Including: Determine a first open area in the first ground overhead image; When the available space in the first open area supports the landing of the drone, determine the first area corresponding to the second open area in the real world, and control the drone to fly above the first area. The second open area is the open area in the second ground overhead image, and the absolute value of the shooting angle of the second ground overhead image is less than the absolute value of the shooting angle of the first ground overhead image; Determine the landing area of the drone in the first area according to the height of the drone from the ground.

2. The method according to claim 1, wherein When the available space in the first open area does not support the landing of the drone, the method further includes: Based on the third ground overhead image, use an image segmentation algorithm to determine the third open area; the shooting angle of the third ground overhead image is the same as the shooting angle of the second ground overhead image; Determine the second area of the third open area in the real world based on the parallax information of the binocular camera and the first target parameter; the first target parameter includes the focal length of the binocular camera and the baseline distance of the binocular camera; Adjust the flight path of the drone so that the drone flies above the second area, and return to execute the step of determining the first open area in the first ground overhead image until the first open area is sufficient.

3. The method according to claim 1, wherein The determining the landing area of the drone in the first area according to the height of the drone from the ground includes: When the height of the drone from the ground reaches the first preset height and there are moving objects in the first area, determine the initial safe landing area based on the predicted motion trajectory of the moving objects, and control the drone to fly above the initial safe landing area; the initial safe landing area belongs to the coverage range of the first area; When the height of the drone from the ground reaches the second preset height, determine the landing area based on a preset detection algorithm, and control the drone to land in the landing area; the landing area belongs to the coverage range of the initial safe landing area; the second preset height is less than the first preset height.

4. The method according to claim 3, characterized in that, The method further includes: When the height of the drone from the ground reaches the first preset height, obtain multiple consecutive target images of the first area; Determine the pixel difference between two consecutive target images; When the pixel difference meets the first preset condition, determine that there are moving objects in the first area.

5. The method according to claim 4, wherein The determining the initial safe landing area based on the predicted motion trajectory of the moving objects includes: For any one of the target images, divide the target image to determine multiple regions of interest; Determine the predicted trajectory of the moving object based on the contour of the moving object and the trajectory prediction model; the trajectory prediction model is trained based on historical trajectory data; the contour of the moving object is determined based on a preset contour determination algorithm; Score the multiple regions of interest based on the contour of the moving object and the predicted trajectory to determine the scoring result; the scoring result is determined based on a preset scoring algorithm; Determine an initial safe landing area based on the scoring result.

6. The method according to claim 1, wherein The method further includes: Determine the area of the first open area based on the second target parameter of the binocular camera and the resolution of the first top-down ground image; the second target parameter includes the width and height of the binocular camera; If the area meets the third preset condition, determine that the available space in the first open area supports the landing of the drone; if the area does not meet the third preset condition, determine that the available space in the first open area does not support the landing of the drone.

7. An obstacle avoidance device, characterized in that, Includes: A first open area determination module for determining a first open area in the first top-down ground image; A second open area determination module for determining a first area corresponding to the second open area in the real world and controlling the drone to fly above the first area when the available space in the first open area supports the landing of the drone, where the second open area is an open area in the second top-down ground image, and the absolute value of the shooting angle of the second top-down ground image is less than the absolute value of the shooting angle of the first top-down ground image; A landing area determination module for determining the landing area of the drone in the first area according to the height of the drone from the ground.

8. An electronic device, characterized in that, Includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the obstacle avoidance method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the obstacle avoidance method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program, and when the computer program is executed by the processor, it implements the obstacle avoidance method according to any one of claims 1-6.

Images