Obstacle avoidance method and device for unmanned aerial vehicle, and medium

Abstract

The invention discloses an obstacle avoidance method and device for an unmanned aerial vehicle and a medium, and the method comprises the steps: controlling the unmanned aerial vehicle to carry out the information collection of a target obstacle under the condition that the unmanned aerial vehicle is determined to be located in a preset range of the target obstacle according to first image information, first position information and initial voiceprint information, and obtaining third image information, generating target edge information according to the third image information; performing adaptive adjustment on the target edge information based on the flight speed of the unmanned aerial vehicle to obtain a target edge region width; and generating a second route according to the target marginal area width and the target marginal information, and controlling the unmanned aerial vehicle to perform obstacle avoidance flight according to the second route. According to the embodiment of the invention, the third image information with the details of the target obstacle can be collected again on the basis of accurately determining that the unmanned aerial vehicle is located in the preset range of the target obstacle, the second route is generated according to the third image information and the flight speed of the unmanned aerial vehicle, and the unmanned aerial vehicle is controlled to fly in an obstacle avoidance manner according to the second route. The autonomous obstacle avoidance effect can be improved.

Description

Technical Field

[0001] This application relates to, but is not limited to, the field of unmanned aerial vehicle (UAV) technology, and particularly to an obstacle avoidance method, device, and medium for UAVs. Background Technology

[0002] During autonomous navigation, drones frequently encounter various obstacles, and the effectiveness of their obstacle avoidance directly determines flight safety. Therefore, obstacle avoidance methods are a key technology for drones. Existing obstacle avoidance methods involve pre-constructing a 3D environment map using GIS (Geographic Information System) data. Obstacles to be avoided during autonomous flight are identified from this map, allowing for obstacle avoidance during path planning. The drone then follows its planned path guided by its GPS (GPS) system to avoid obstacles until autonomous flight is complete. However, the positioning accuracy of the drone's GPS system is subject to uncontrollable disturbances such as external signal interference, system errors, and errors in the hardware sensors involved, resulting in positioning errors. Furthermore, complex 3D environments, such as urban canyons and mountainous terrain, undergo subtle changes over time, including positional shifts. Constructing high-precision 3D environment maps requires significant resources for on-site surveying and data annotation, leading to lags in map database updates. The drone relies on an inaccurate 3D environment map database with an imprecise positioning system to avoid known obstacles, resulting in a reduced autonomous obstacle avoidance performance and failing to meet the obstacle avoidance requirements of the drone. Summary of the Invention

[0003] This application provides an obstacle avoidance method, device, and medium for unmanned aerial vehicles (UAVs), which can improve the autonomous obstacle avoidance performance of UAVs against known target obstacles, thereby meeting the obstacle avoidance requirements of UAVs.

[0004] In a first aspect, embodiments of this application provide an obstacle avoidance method for a drone, applied to the controller of the drone, the method comprising:

[0005] The system acquires target environment information by collecting data from the surrounding environment of the drone at a first moment during its flight along the first route. The target environment information includes first image information, first position information, and initial voiceprint information corresponding to the first moment. The first image information is an image of the surrounding environment of the drone at the first moment. The first position information is the location of the drone at the first moment. The initial voiceprint information is the sound wave information of the surrounding environment of the drone at the first moment. The first moment is the moment when the drone reaches the preset initial range corresponding to the target obstacle during its flight along the first route.

[0006] When it is determined that the UAV is located within a preset range of the target obstacle based on the first image information, the first location information, and the initial voiceprint information, the UAV is controlled to collect information about the target obstacle to obtain third image information, and target edge information is generated based on the third image information.

[0007] The target edge information is adaptively adjusted based on the flight speed of the UAV to obtain the width of the target edge region;

[0008] A second route is generated based on the width of the target edge region and the target edge information, and the UAV is controlled to perform obstacle avoidance flight based on the second route.

[0009] Secondly, an electronic device provided according to an embodiment of this application includes:

[0010] At least one processor;

[0011] At least one memory for storing at least one program;

[0012] When at least one of the programs is executed by at least one of the processors, the obstacle avoidance method for the drone according to any of the first aspects is implemented.

[0013] Thirdly, according to the embodiments of the application, a computer-readable storage medium is provided, storing computer-executable instructions, which are used to execute the obstacle avoidance method of the UAV as described in any of the first aspects.

[0014] In summary, the obstacle avoidance method for unmanned aerial vehicles (UAVs) according to the above embodiments of this application is applied to the controller of the UAV, including: acquiring the surrounding environment of the UAV at a first moment during the flight of the UAV along a first route to obtain target environment information, wherein the target environment information includes first image information, first position information, and initial acoustic signature information corresponding to the first moment, the first image information being an image of the surrounding environment of the UAV at the first moment, the first position information being the location of the UAV at the first moment, and the initial acoustic signature information being the sound wave information of the surrounding environment of the UAV at the first moment, the first moment being the moment when the UAV reaches the preset initial range corresponding to the target obstacle during the flight of the first route; adaptively adjusting the target edge information based on the flight speed of the UAV to obtain the target edge area width; generating a second route based on the target edge area width and the target edge information, and controlling the UAV to perform obstacle avoidance flight according to the second route. This embodiment first determines that the UAV is located within a preset range of the target obstacle based on the first image information, first position information, and initial voiceprint information. It can correct the UAV's first position information using the first image information, first position information, and initial voiceprint information to accurately determine the UAV's location within the preset range of the target obstacle. Then, it controls the UAV to collect information about the target obstacle to obtain third image information, and generates target edge information based on the third image information. Based on the accurately corrected first position information, it collects information about the target obstacle to obtain third image information with detailed information about the target obstacle, eliminating the need for remodeling to update the map database. This makes the generation of target edge information based on the third image information more efficient. Accuracy is achieved while also considering the computational response speed of the UAV. Then, the target edge information is adaptively adjusted based on the UAV's flight speed to obtain the target edge region width. On the basis of accurate target edge information, the adaptive edge region width data obtained by adjusting the UAV's flight speed and target edge information is more accurate. Finally, a second route is generated based on the target edge region width and target edge information to accurately avoid target obstacles. Compared with obstacle avoidance based on an inaccurate 3D environment map database on an inaccurate positioning system, this method can improve the UAV's autonomous obstacle avoidance effect on known target obstacles even when the details of known target obstacles are inaccurate, thus meeting the obstacle avoidance requirements of the UAV. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the steps of an obstacle avoidance method for an unmanned aerial vehicle (UAV) according to an embodiment of this application.

[0016] Figure 2 This is an architectural diagram of a preset target recognition model provided in one embodiment of this application;

[0017] Figure 3This is a hardware schematic diagram of an electronic device provided in one embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] It is understandable that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, or the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0020] During autonomous navigation, drones frequently encounter various obstacles, and the effectiveness of their obstacle avoidance directly determines flight safety. Therefore, obstacle avoidance methods are a key technology for drones. Existing obstacle avoidance methods involve pre-constructing a 3D environment map using GIS (Geographic Information System) data. Obstacles to be avoided during autonomous flight are identified from this map, allowing for obstacle avoidance during path planning. The drone then follows its planned path guided by its GPS (GPS) system to avoid obstacles until autonomous flight is complete. However, the positioning accuracy of the drone's GPS system is subject to uncontrollable disturbances such as external signal interference, system errors, and errors in the hardware sensors involved, resulting in positioning errors. Furthermore, complex 3D environments, such as urban canyons and mountainous terrain, undergo subtle changes over time, including positional shifts. Constructing high-precision 3D environment maps requires significant resources for on-site surveying and data annotation, leading to lags in map database updates. The drone relies on an inaccurate 3D environment map database with an imprecise positioning system to avoid known obstacles, resulting in a reduced autonomous obstacle avoidance performance and failing to meet the obstacle avoidance requirements of the drone.

[0021] Based on this, this application provides an obstacle avoidance method, device and medium for unmanned aerial vehicles (UAVs) to improve the autonomous obstacle avoidance effect of UAVs on known target obstacles, so as to meet the obstacle avoidance requirements of UAVs.

[0022] This application provides an obstacle avoidance method for a drone, applied to the drone's controller, with reference to... Figure 1 As shown, obstacle avoidance methods for drones can be achieved through steps including but not limited to the following:

[0023] Step S100: Obtain the target environment information collected by the UAV. The target environment information includes the first image information, the first position information and the initial acoustic signature information corresponding to the first moment. The first image information is an image of the surrounding environment of the UAV at the first moment. The first position information is the location of the UAV at the first moment. The initial acoustic signature information is the sound wave information of the surrounding environment of the UAV at the first moment. The first moment is the moment when the UAV reaches the preset initial range corresponding to the target obstacle during its flight along the first route.

[0024] For example, the drone is equipped with a positioning device, a camera device, and a sound acquisition device. The drone is guided by the positioning device to fly along a first route, reaching a preset initial range (the current time is the first moment) corresponding to a target obstacle on the first route. That is, the drone approaches the vicinity of the target obstacle in the map database, obtains the drone's first location information from the positioning device, and simultaneously captures images of the drone's surrounding environment using the camera device to obtain first image information. The sound acquisition device also collects sound wave information from the surrounding environment to obtain initial voiceprint information. This application does not limit the specific device types of the positioning device, camera, and sound acquisition device. Specifically, the positioning system can be GPS positioning, WIFI positioning, or Bluetooth positioning; the camera device can be a 2D camera device, an infrared camera device, or a fisheye camera device; and the sound acquisition device can be a microphone array or a voiceprint acquisition device.

[0025] For example, the user of the drone inputs a flight mission into the drone, which includes at least information such as the initial position and the destination position. When the drone receives the information in the flight mission, it first generates a first shortest straight line from the initial position to the destination position from the map database based on the initial position and the destination position. Then, it expands the range of the first shortest straight line according to a preset range to obtain the flight area that the flight mission needs to pass through. Next, it obtains all the flight obstacles existing in the flight area and the first obstacle position information corresponding to each flight obstacle from the map database based on the flight area. Then, based on the first shortest straight line, it filters out the target obstacles that need to be bypassed during the execution of the flight mission and the second obstacle position information corresponding to the target obstacles from all the flight obstacles. Finally, according to the direction from the starting position to the destination position, the second shortest straight line with the shortest distance between each pair of the starting position, the adjacent second obstacle position information, and the destination position is used as the first route.

[0026] Step S110: If it is determined that the UAV is located within the preset range of the target obstacle based on the first image information, the first position information and the initial voiceprint information, control the UAV to collect information on the target obstacle to obtain the third image information, and generate target edge information based on the third image information.

[0027] For example, at the visual level, target recognition is first performed on the target obstacle in the first image information collected by the drone, accurately locating the bounding box position information of the target obstacle in the image. Then, the first position information of the drone is corrected based on the bounding box position information. Simultaneously, at the auditory level, voiceprint recognition and localization analysis are first performed on the initial voiceprint information collected by the microphone array on the drone to obtain recognition and localization results. Then, the recognition results are compared with a preset voiceprint feature library of known obstacles around the target obstacle in the map database to select the known obstacle with the highest matching degree as the second marker. Then, the first position information of the drone is corrected based on the localization result corresponding to the second marker. Finally, the real position of the target obstacle in the map database is obtained, and then the shortest distance between the corrected first position information and the real position is calculated. The shortest distance is compared with a preset range. When the shortest distance is less than the preset range, it indicates that the drone is located within the preset range of the target obstacle; conversely, when the shortest distance is greater than or equal to the preset range, it indicates that the drone is not located within the preset range of the target obstacle. This application embodiment corrects the first position information of the UAV using the block diagram position information of the target obstacle at the visual level, and supplements the positioning constraint by matching the voiceprint of the second marker and voiceprint localization at the auditory level to correct the first position information. By performing dual correction at both the visual and auditory levels, the accuracy of the corrected first position information of the UAV can be improved, thereby improving the accuracy of determining whether the UAV is within the preset range of the target obstacle based on the corrected first position information.

[0028] For example, a processed image is obtained by performing grayscale conversion and Gaussian blur preprocessing operations on the third image information. Then, for the gradient between each pixel and its neighboring pixels in the processed image, the pixels corresponding to the abrupt gradient are taken as edge points, and the target edge information is determined based on all the corresponding edge points in the processed image.

[0029] Step S120: Adaptively adjust the target edge information based on the UAV's flight speed to obtain the target edge region width.

[0030] For example, since the flight speed and inertia of a drone are closely related, an increase in flight speed leads to an increase in inertia. If the drone attempts to avoid obstacles based on the target edge information at that time, it is prone to colliding with the target obstacle due to inertia. Therefore, when the drone's flight speed increases, a target edge area width is set as a safe zone for the target edge information, and this width is increased to prevent collisions due to inertia. Conversely, when the drone's flight speed decreases, a target edge area width is set as a safe zone for the target edge information, and this width is decreased to make the drone's obstacle avoidance more precise.

[0031] Step S130: Generate a second route based on the target edge area width and target edge information, and control the UAV to perform obstacle avoidance flight based on the second route.

[0032] For example, firstly, each edge of the target edge information is expanded outward according to the width of the target edge region to obtain the safe area corresponding to the target edge information. Then, Dijkstra's algorithm is used to search the map database for the shortest path that starts from the current location of the drone and completely avoids the safe area as the second route.

[0033] For example, the obstacle avoidance method for a drone can be implemented through the following steps: First, multiple first routes are sequentially obtained from the drone's flight path. The drone is controlled to fly from the starting position according to the first first route. When the drone reaches the second obstacle position on the first route, the drone is controlled to fly according to the second route to avoid the target obstacle at the second obstacle position. The multiple first routes and multiple second routes are sequentially spliced ​​together to obtain the flight path for the drone to avoid all target obstacles. The drone is then controlled to follow the flight path to avoid the target obstacles that need to be avoided from the starting position to the ending position.

[0034] Therefore, this embodiment first determines that the UAV is located within a preset range of the target obstacle based on the first image information, the first position information, and the initial voiceprint information. It can correct the first position information of the UAV using the first image information, the first position information, and the initial voiceprint information to accurately determine the UAV's location within the preset range of the target obstacle. Then, it controls the UAV to collect information about the target obstacle to obtain third image information, and generates target edge information based on the third image information. Based on the accurately corrected first position information, it collects information about the target obstacle to obtain third image information with detailed information about the target obstacle, eliminating the need for remodeling to update the map database. This allows for the generation of target edge information based on the third image information. This approach achieves greater accuracy while also considering the drone's computational response speed. Then, based on the drone's flight speed, the target edge information is adaptively adjusted to obtain the target edge width. This adaptive edge width data, obtained by adjusting the drone's flight speed and target edge information based on accurate target edge information, is even more accurate. Finally, a second route is generated based on the target edge width and target edge information to precisely avoid target obstacles. Compared to obstacle avoidance based on an inaccurate 3D environment map database using an inaccurate positioning system, this approach improves the drone's autonomous obstacle avoidance performance even when the details of known target obstacles are inaccurate, thus meeting the drone's obstacle avoidance requirements.

[0035] It is understandable that methods for avoiding unmarked obstacles in map databases can utilize multimodal sensor data fusion, such as LiDAR, visual sensors, and millimeter-wave radar, to achieve real-time detection and avoidance of these obstacles. This application does not limit the obstacle avoidance method for unknown obstacles; specific obstacle avoidance methods can be selected based on actual needs.

[0036] In some embodiments, the method for determining that a drone is located within a preset range of a target obstacle based on first image information, first location information, and initial voiceprint information includes: performing target recognition on the first image information according to a preset target recognition model to obtain a first marker and its corresponding block diagram location information in the first image information; calculating a deviation value based on the block diagram location information, the first location information, and the first true location information of the first marker in a map database to obtain first deviation information; performing voiceprint recognition on the initial voiceprint information to obtain the marker type of a second marker, and performing positioning analysis based on the initial voiceprint information to obtain distance and direction information between the second marker and the drone; determining the second true location information of the second marker in the map database based on the marker type, distance information, and direction information; calculating a deviation value based on the first location information, distance information, direction information, and the second true location information to obtain second deviation information; correcting the first location information based on the first deviation information and the second deviation information to obtain the target location information of the drone; and determining whether the drone is located within the preset range of the target obstacle based on the target location information.

[0037] Understandably, the first step is to analyze the first deviation information to infer the relative positional relationship between the UAV and the target obstacle, and the second deviation information to infer the relative positional relationship between the UAV and the second marker. Then, by combining the relative positional relationship between the UAV and the second marker, the relative positional relationship between the UAV and the target obstacle, the first deviation information, and the second deviation information, the first positional information of the UAV is corrected to obtain the target positional information.

[0038] For example, complex three-dimensional environments such as urban canyons and mountainous terrain undergo subtle changes over time, including changes in position and volume. This means that the target obstacles encountered by the UAV during its flight mission will also change in position and volume over time. Therefore, this embodiment first uses a preset target recognition model to identify the target in the first image information, obtaining the position information of the first marker and the block diagram. This allows for visual perception of the position and volume changes of the target obstacles, thereby improving the accuracy of the position information of the first marker and the block diagram.

[0039] For example, the preset target recognition model can be a Faster R-CNN model, a YOLO model, or an SSD (SingleShot MultiBoxDetector) model, and this application embodiment does not limit this.

[0040] Understandably, the process begins by calculating the deviation between the positional changes of the first marker and the target obstacle based on the location information of the block diagram, the first location information, and the first true location information of the first marker in the map database. This yields the first deviation information, which, based on accurate identification of the first marker and the location information of the block diagram, makes the calculation of the positional deviation between the first marker and the target obstacle more accurate. Then, voiceprint recognition is performed on the initial voiceprint information to obtain the marker type of the second marker. Based on the initial voiceprint information, positioning analysis is performed to obtain the distance and direction information between the second marker and the drone. Aesthetically, the uniqueness of the spectrum of the initial voiceprint information can be utilized to improve the accuracy of the second marker obtained through voiceprint recognition. Simultaneously, the propagation characteristics of the initial voiceprint information can be used to accurately obtain the distance and direction information between the second marker and the drone. Finally, based on the marker type, distance information, and direction information of the second marker obtained through voiceprint recognition, the location of the second marker in the map database is determined. The system first obtains true location information. Based on first location information, distance information, direction information, and second true location information, it calculates the distance and direction deviation between the second marker and the drone, obtaining second deviation information. This, combined with accurate distance and direction information and the marker type of the second marker, makes the calculated position deviation between the second marker and the drone more accurate. Next, the first location information is corrected using the first and second deviation information to obtain the drone's second location information. This correction is achieved by using both visual and auditory information to accurately determine the drone's target location. Finally, based on the target location information, it determines whether the drone is within a preset range of the target obstacle. This improved accuracy in determining the drone's location within the preset range of the target obstacle, providing a reliable basis for subsequent obstacle avoidance by the drone.

[0041] In some embodiments, voiceprint recognition is performed on the initial voiceprint information to obtain the identification type of the second identifier, and location analysis is performed based on the initial voiceprint information to obtain the distance and direction information between the second identifier and the UAV. This includes: firstly, feature extraction is performed on the initial voiceprint information to obtain voiceprint features; the voiceprint features are compared with preset voiceprint features corresponding to known obstacles around the target obstacle in the UAV's map database to obtain the comparison result; the identification type of the second identifier is determined based on the comparison result; then, the distance is estimated based on the initial voiceprint information according to the time delay to obtain the distance information; and the direction of arrival is estimated based on the initial voiceprint information and the sound acquisition device corresponding to the initial voiceprint information to obtain the direction information.

[0042] Understandably, the initial voiceprint information is a sound wave spectrum composed of multiple feature dimensions such as wavelength, frequency, and intensity. This gives the sound wave spectrum stability, measurability, and uniqueness. Voiceprint features are extracted from the sound wave spectrum, ensuring the uniqueness of the extracted voiceprint features based on the uniqueness of the sound wave spectrum. Then, the voiceprint features are compared one-to-one with preset voiceprint features corresponding to each target obstacle in the UAV's map database. Each preset voiceprint feature corresponds to an identifier type, resulting in comparison results of multiple preset voiceprint features that match the original voiceprint features. The uniqueness of the voiceprint features improves the accuracy of the comparison results. Finally, the preset voiceprint feature with the highest consistency among the comparison results is selected. The identification type is used as the identification type of the second identifier to improve the accuracy of the identification type of the second identifier obtained by voiceprint recognition; the time difference between the initial voiceprint information collected by the array of sound acquisition devices in the UAV is obtained, and the time difference is multiplied by the preset propagation speed of the sound wave to obtain the distance information between the second identifier and the UAV. The distance information between the second identifier and the UAV can be accurately calculated by utilizing the time delay of the sound wave transmission characteristics; by applying a preset weighted delay to the sound waves received by multiple microphone arrays and superimposing them, the array sound acquisition device has high gain in a specific direction and low gain in other directions. By scanning all directions to find the direction with the highest output power as the direction information, the direction information between the second identifier and the UAV can be accurately calculated by utilizing the intensity attenuation of the sound wave transmission characteristics.

[0043] In some embodiments, the second true location information can be determined by the following steps: first, intermediate markers are filtered from the map database using distance and direction information; then, a second marker is obtained from the intermediate markers using the marker type; and the second true location of the second marker in the map database is obtained.

[0044] For example, since the location information of the block diagram is two-dimensional pixel coordinates, it is necessary to further transform the location information of the block diagram from the pixel coordinate system to the world coordinate system to obtain the first predicted position of the target obstacle in the world coordinate system; on this basis, the first real position information corresponding to the first marker is obtained from the map database (the first real position information is the pre-mapped or preset real coordinates corresponding to the target obstacle), the first predicted position is compared with the first real position information, and the positional deviation between the first predicted position and the first real position information of the target obstacle is calculated to obtain the first deviation information.

[0045] For example, since distance and direction information belong to the acoustic coordinate system, the distance and direction information are transformed from the acoustic coordinate system (an acoustic coordinate system with the microphone array as the origin) to the world coordinate system to obtain the second predicted position of the second marker in the world coordinate system. Further, the second predicted position is compared with the second true position information, and the positional deviation between the acoustic positioning position and the second true position information is calculated to obtain the second deviation information.

[0046] In some embodiments, correcting the first position information based on the first deviation information and the second deviation information to obtain the second position information of the UAV can be achieved through the following steps: First, determine the prediction reliability based on the first predicted position and the second predicted position; then, determine the first deviation weight corresponding to the first deviation information and the second deviation weight corresponding to the second deviation information based on the prediction reliability; next, fuse the first deviation information and the second deviation information based on the first deviation weight and the second deviation weight to obtain the target deviation information; finally, correct the first position information based on the target deviation information to obtain the second position information. This embodiment of the application can filter out the first deviation information and the second deviation information with excessively low prediction reliability during the fusion process of the first deviation information and the second deviation information based on the first deviation weight and the second deviation weight, and correct the first position information after weighted fusion based on the first deviation weight and the second deviation weight to obtain the second position information. This can suppress abnormal situations through weighted fusion while utilizing prediction reliability to reduce errors. Thus, the accuracy of obtaining the second position information based on the target deviation information by correcting the first position information can be improved.

[0047] It is understandable that determining the prediction reliability based on the first predicted position and the second predicted position can be achieved by calculating the first prediction error between the first predicted position and the first true position based on Euclidean distance, calculating the second prediction error between the second predicted position and the second true position based on Euclidean distance, and then determining the prediction reliability by weighting the first prediction error and the second prediction error. Alternatively, the prediction reliability can be obtained by looking up the prediction reliability mapping table corresponding to the UAV based on the first predicted position and the second predicted position. The prediction reliability mapping table represents the mapping relationship between the prediction reliability corresponding to the first predicted position and the second predicted position. This embodiment does not specifically limit the method for determining the prediction reliability.

[0048] In some embodiments, refer to Figure 2 As shown, the preset target recognition model includes a motion parameter estimation layer, a first kernel function determination layer, a second kernel function determination layer, a first fuzzing layer, a second fuzzing layer, an image fusion layer, and a target recognition layer. The target environment information also includes the second image information corresponding to the second time moment. The second time moment is acquired after the first time moment. Target recognition is performed on the first image information according to the preset target recognition model to obtain the first marker and the corresponding block diagram position information of the first marker in the first image information. The obstacle avoidance method for the UAV includes: determining the corresponding motion speed and direction of the UAV by combining the motion parameter estimation layer with the corresponding flight parameter information of the UAV, and determining the target based on the motion speed and direction by the first kernel function determination layer. The first image information corresponds to a first kernel function; a first blurring layer performs a convolution operation on the first image information based on the first kernel function to obtain a first processing result; a second kernel function determination layer calculates the target weights corresponding to the target-same features between the second image information and the first image information, and determines the second kernel function based on the target weights and the target-same features; a second blurring layer performs a convolution operation on the second image information based on the second kernel function to obtain a second processing result; an image fusion layer fuses the first processing result and the second processing result to obtain a target fused image; a target recognition layer performs target recognition on the target fused image to obtain a first marker and the corresponding block diagram location information.

[0049] Since the first image information and the second image are images captured by the drone during its movement, when there is relative motion between the drone and the target being photographed, the light reflected from the target continuously moves on the imaging sensor during the shutter's exposure time, forming a trailing shadow, which blurs the background and the target in the image. Therefore, this embodiment first determines the drone's speed and direction of movement by combining the drone's flight parameter information with the motion parameter estimation layer, and then determines the first kernel function corresponding to the first image information based on the speed and direction of movement by the first kernel function, so that the first kernel function can dynamically fuse the drone's motion characteristics (speed and direction of movement); then, the first blur processing layer performs a convolution operation on the first image information based on the first kernel function to obtain the first processing result. This allows the drone's motion characteristics from the first kernel function to be used to perform a convolution operation on the first image information to obtain the first processing result, suppressing the blurring of the background and the same features of the target caused by motion in the first image information, thereby improving the expressive power of the same features of the background and the target in the first processing result; simultaneously, the second kernel function determines the target weights corresponding to the same features of the target between the second image information and the first image information according to the second kernel function determination layer, and determines the second kernel function based on the target weights and the same features of the target, and then performs a second blur processing layer. The first processing layer performs a convolution operation on the second image information based on the second kernel function to obtain the second processing result. This enhances the sensitivity of the second kernel function to the target-same features in the second image, thereby enhancing the expressive power of the second processing result for the target-same features. Then, the first processing result and the second processing result are fused together by the image fusion layer to obtain the target fused image. Based on the enhancement of the first and second processing results, the expressive power of the background and target-same features in the target fused image obtained by the image fusion of the first and second processing results is improved. Finally, the target recognition layer performs target recognition on the target fused image to obtain the first marker and the corresponding bounding box position information. Based on the improved expressive power of the background and target-same features in the target fused image, the anti-ambiguity, trajectory continuity, and directional robustness of the target fused image for target recognition are improved, thereby improving the accuracy of the first marker and the corresponding bounding box position information obtained by target recognition.

[0050] In some embodiments, the first kernel function can be determined through the following steps: First, obtain the first position information and the time information corresponding to the first position information within a preset time range corresponding to the first moment; then, use the motion parameter estimation layer to determine the motion speed and direction of the drone at the target moment based on the first position information, time information, and the flight parameter information corresponding to the drone; finally, use the first kernel function determination layer to determine the first kernel function corresponding to the first image information based on the motion speed and motion direction. The expression of the first kernel function is as follows:

[0051]

[0052] Wherein, K1(x, y) represents the first kernel function corresponding to the horizontal coordinate x and vertical coordinate y in the first image information, t1 represents the initial time within a preset time range, t2 represents the corresponding end time within the preset time range, v(t) represents the motion speed at time t, α represents the motion direction, δ represents the Dirac function, x0 and y0 represent the center position of the first kernel function, and cos represents the cosine function. The first kernel function in this embodiment enhances the response sensitivity to rapid motion through velocity weighting, and simultaneously utilizes the Dirac function to achieve selective filtering of motion direction. It can dynamically fuse the motion characteristics (motion speed and motion direction) of the UAV with spatiotemporal information, effectively suppressing the ambiguity interference of background and target having the same features. Furthermore, through an adaptive center position mechanism, it can accurately track the target trajectory. Simultaneously, through time-domain integration, it achieves multi-frame feature fusion, significantly improving the anti-ambiguity, trajectory continuity, and direction robustness of target recognition in complex motion scenarios.

[0053] In other embodiments, the drone's speed and direction of motion can be acquired in real time by hardware embedded within the drone, improving the drone's response speed in acquiring these parameters. Specifically, the hardware can be an attitude sensor or a speed sensor.

[0054] In some embodiments, the target recognition layer can employ a YOLO network architecture, which may include a backbone, a neck, and a head. The backbone includes Conv layers, C2f layers, and SPPF layers, with Conv and C2f layers alternately connected. The Conv layers (integrating convolutional layers, BN layers, and SiLU activation layers) perform spatial downsampling and channel expansion on the fused target image. The C2f layers then fuse the extracted multi-scale features with semantics at the same scale. Finally, the SPPF layer is concatenated at the end of the backbone to perform contextual fusion of the multi-scale features. The neck introduces a PAN path based on the Feature Pyramid Network (FPN) to form a bidirectional feature flow architecture, fusing multi-scale features bidirectionally from top to bottom (high semantics to low layers) and from bottom to top (high resolution to high layers). The head includes target classification and bounding box regression branches. The target classification branch classifies the fused multi-scale features to obtain the first identifier, while the bounding box regression branch performs bounding box regression prediction on the fused multi-scale features to obtain the bounding box location information. This application embodiment utilizes the high-precision real-time recognition capability of the YOLO network to enable the target recognition layer of the YOLO network architecture to perform high-precision real-time recognition of the target fusion image, thereby obtaining highly accurate first marker and block diagram location information.

[0055] In some embodiments, the first processing result and the second processing result are fused by an image fusion layer to obtain a target fused image. This can be achieved by weighted fusion of the first processing result and the second processing result, or by wavelet transform based on the first processing result and the second processing result. The embodiments of this application do not specifically limit the method.

[0056] In some embodiments, the second kernel function determination layer includes a feature tracking unit, a weight determination unit, and a second kernel function determination unit. The second kernel function determination layer calculates the target weights corresponding to the target-same features between the second image information and the first image information, and determines the second kernel function based on the target weights and the target-same features. The obstacle avoidance method for the UAV includes: performing feature recognition on the second image information and the first image information respectively through the feature tracking unit to obtain the first target feature corresponding to the second image information and the second target feature corresponding to the first image information, and calculating the similarity between the first target feature and the second target feature, and then determining the target-same features based on the similarity; performing weighted fusion based on the second image information, the first image information, and the similarity through the weight determination unit to obtain the target weights of the target-same features; and determining the second kernel function based on the target weights and the target-same features through the second kernel function determination unit.

[0057] Therefore, in this embodiment, the feature tracking unit first performs feature recognition on the second image information and the first image information respectively to obtain the first target feature corresponding to the second image information and the second target feature corresponding to the first image information, and calculates the similarity between the first target feature and the second target feature. Then, the target identical features are determined based on the similarity, thereby improving the accuracy of determining the target identical features based on similarity. Then, the weight determination unit performs weighted fusion based on the second image information, the first image information, and the similarity to obtain the target weight of the target identical features. The second kernel function determination unit determines the second kernel function based on the target weight and the target identical features, which can dynamically determine the target weight of the target identical features. This allows the second kernel function to focus on the target identical features based on the target weight, thereby improving the sensitivity of the target identical features of local features in complex scenes.

[0058] In some embodiments, the feature tracking unit calculates the Hamming distance between the BRIEF feature descriptors extracted from each feature point of the first image information and the second image information, calculates the similarity based on the calculated Hamming distance and a preset Hamming distance threshold, and selects the corresponding feature point pair with the highest similarity as the target identical feature for matching the first image information and the second image information.

[0059] In some embodiments, the expression for the second kernel function is as follows:

[0060]

[0061] Where K2(x,y) represents the second kernel function corresponding to the first image information with horizontal coordinate x and vertical coordinate y, M represents the number of identical features of the target, and w m Let x represent the target weights corresponding to the same features of the m-th target, δ represent the Dirac function, and x represent the target weights. i The y-coordinate represents the horizontal coordinate of the i-th pixel in the first image information. i This represents the vertical coordinate corresponding to the i-th pixel of the first image information. The second kernel function in this embodiment accurately locates the spatial distribution of target features using the Dirac function to suppress noise interference in the image. Simultaneously, it assigns importance to different features through target weights, thereby increasing the sensitivity to the same target features in local features, thus improving the robustness and discriminative power of target recognition.

[0062] In some embodiments, a weight determination unit performs weighted fusion based on second image information, first image information, and similarity to obtain target weights for targets with the same features. The obstacle avoidance method for the UAV includes: acquiring first region information corresponding to the same target features in the first image information and second region information corresponding to the same target features in the second image information; determining first true coordinates corresponding to the first region information based on first position information corresponding to the first image information, and determining second true coordinates corresponding to the second region information based on second position information corresponding to the second image information; determining target offsets corresponding to the same target features based on the first and second true coordinates, and determining first weights based on target offsets; determining first changes in the same target features at horizontal positions and second changes in vertical positions based on the first and second true coordinates, and determining second weights based on the sum of the first and second changes; determining third weights based on similarity, and determining target weights based on the first, second, and third weights.

[0063] For example, the first region information corresponding to the same features as the target in the first image information can be obtained through the Faster R-CNN model. The Faster R-CNN model includes a ResNet module, a region proposal module, a RoIPooling module, and a classification and regression module. Specifically, firstly, the ResNet module extracts features from the first image information to obtain a feature map; then, the region proposal module slides an anchor box on the feature map, continuously adjusting the coordinates of the predicted anchor box until the predicted anchor box corresponds to the same features as the target, generating candidate regions; then, the RoIPooling module maps the candidate regions to the feature map and extracts features of a fixed size; finally, the fully connected layer of the classification and regression module classifies the candidate regions and regresses the precise bounding box coordinates, and then superimposes and segments the bounding box coordinates and the first image information to obtain the first region information corresponding to the same features as the target. It is understood that the method for obtaining the second region information is the same as that for obtaining the first region information, and will not be elaborated here.

[0064] For example, firstly, the Euler angles output by the attitude sensor mounted on the UAV are obtained; then, a sensor coordinate system relationship chain is constructed based on the camera's preset intrinsic parameter matrix and preset camera extrinsic parameter matrix; next, the pixel coordinates of the first region information in pixel coordinates are inversely transformed from the pixel coordinate system to the camera coordinate system through perspective projection using the sensor coordinate system relationship chain, and the geometric position deviation in the pixel coordinate system is distorted to obtain a normalized direction vector in the camera coordinate system; then, the direction vector is mapped from the camera coordinate system to the UAV body coordinate system through rigid body transformation using the camera extrinsic parameters, and after rotation according to Euler angles, it is weighted and superimposed with the first position information to obtain the first true coordinates corresponding to the first region information rotated from the UAV body coordinate system to the world coordinate system. It is understood that the method for determining the second true coordinates is the same as the method for determining the first true coordinates, and will not be elaborated here.

[0065] For example, the target offset can be mapped to the first weight using a preset decay function, or the target offset can be input into a preset linear regression model to obtain the first weight. It is understood that the determination methods for the second weight, third weight, and target weight are the same as those for the first weight, and will not be elaborated upon here.

[0066] For example, the coordinate difference between the first true coordinate and the second true coordinate is calculated to obtain the coordinate difference, and the coordinate difference is used to determine the target offset corresponding to the same feature of the target.

[0067] For example, the horizontal change of the coordinate components of the first true coordinate and the second true coordinate at the horizontal position and the vertical change of the coordinate components at the vertical position are calculated. The horizontal change is used as the first change of the same feature of the target at the horizontal position and the vertical change is used as the second change of the same feature of the target at the vertical position.

[0068] Therefore, this embodiment first obtains the first region information corresponding to the same target feature in the first image information and the second region information corresponding to the same target feature in the second image information. Based on the first position information corresponding to the first image information, it determines the first true coordinates corresponding to the first region information, and based on the second position information corresponding to the second image information, it determines the second true coordinates corresponding to the second region information. This allows for accurate calculation of the first true coordinates corresponding to the first region information and the second true coordinates corresponding to the second region information through coordinate system transformation. Then, it determines the target offset corresponding to the same target feature based on the first and second true coordinates, and determines the first weight based on the target offset. By using the technical means of determining the target offset corresponding to the same target feature based on the first and second true coordinates and determining the first weight based on the target offset, the first and second true coordinates are accurate, enabling the calculation of the first true coordinates and the second true coordinates. The accuracy of determining the target offset corresponding to the same feature of the target is improved. Then, the offset of the same feature of the target is measured by the target offset, so that the first weight determined by the target offset can perceive the offset of the same feature of the target. Then, the first change of the same feature of the target in the horizontal position and the second change in the vertical position are determined by the first and second true coordinates. The second weight is determined by the sum of the first and second changes. With the accuracy of the first and second true coordinates, the second weight determined by the sum of the first and second changes can measure the deformation of the same feature of the target in the horizontal and vertical positions. Finally, the third weight is determined by similarity. The target weight is determined by the first weight, the second weight and the third weight. It can measure the position change by using the target offset while considering the similarity of the same feature of the target, and take into account the changes in the horizontal and vertical positions, thereby effectively improving the discrimination of target recognition.

[0069] It is understandable that image information obtained through optical imaging is subject to uncontrollable errors such as optical system errors (radial or tangential distortion), external environmental factors (atmospheric refraction), and mechanical system errors (vibration from UAV flight), leading to shifts and geometric deformations of identical target features in the image information. Therefore, this embodiment of the application ensures the reliability of identical target features by calculating the similarity between the first and second image information. Based on this, the shift of identical target features is quantified by calculating the target offset corresponding to the identical target features in the first and second image information, and the geometric deformation of identical target features is quantified by calculating the sum of the changes in the horizontal and vertical positions of identical target features in the first and second image information.

[0070] It is understandable that by performing convolution operations on the second image information using the second kernel function, offset correction and geometric deformation correction are applied to the target-identical features in the second image information, thereby enhancing the expressive power of the target-identical features in the second image.

[0071] In some embodiments, a deviation value is calculated based on the block diagram location information, the first location information, and the first true location information of the first marker in the map database to obtain the first deviation information. The obstacle avoidance method for the UAV includes: constructing a sensor coordinate system relationship chain based on the preset camera intrinsic parameter matrix and the preset camera extrinsic parameter matrix corresponding to the UAV, and performing inverse transformation and distortion correction on the block diagram location information according to the sensor coordinate system relationship chain to obtain the direction vector in the camera coordinate system; mapping the direction vector to the UAV body coordinate system through rigid body transformation based on the preset camera extrinsic parameters corresponding to the UAV to obtain intermediate coordinate information in the UAV body coordinate system; rotating the intermediate coordinate information based on the Euler angles corresponding to the UAV and superimposing it with the first location information to obtain the first predicted position of the first marker in the world coordinate system; and calculating the deviation value based on the first predicted position and the first true location information to obtain the first deviation information.

[0072] This application first constructs a sensor coordinate system relationship chain based on the preset camera intrinsic parameter matrix and preset camera extrinsic parameter matrix corresponding to the UAV. Then, based on the sensor coordinate system relationship chain, it performs an inverse transformation and distortion correction on the block diagram position information to obtain the direction vector in the camera coordinate system. This allows the block diagram position information to be transformed from the pixel coordinate system to the camera coordinate system while maintaining the accuracy of the direction vector. Next, based on the preset camera extrinsic parameters corresponding to the UAV, the direction vector is mapped to the UAV body coordinate system through rigid body transformation to obtain the intermediate coordinate information in the UAV body coordinate system. This ensures the accuracy of the direction vector and allows the rigid body transformation mapping of the direction vector based on the preset camera extrinsic parameters corresponding to the UAV to achieve this. The accuracy of intermediate coordinate information obtained by projecting the data into the UAV's body coordinate system is improved. Then, the intermediate coordinate information is rotated based on the UAV's corresponding Euler angles and superimposed with the first position information to obtain the first predicted position of the first marker in the world coordinate system. Based on the accurate intermediate coordinate information, the accuracy of obtaining the first predicted position of the first marker in the world coordinate system by rotating the intermediate coordinate information based on the UAV's corresponding Euler angles and superimposing with the first position information is improved. Finally, the deviation value is calculated based on the first predicted position and the first true position information to obtain the first deviation information that measures the UAV's position offset, thereby improving the accuracy of the first deviation information.

[0073] Specifically, the first predicted position can be achieved through the following steps: First, obtain the Euler angles output by the attitude sensor mounted on the UAV; then, construct a sensor coordinate system relationship chain based on the camera's preset intrinsic parameter matrix and preset camera extrinsic parameter matrix; then, perform perspective projection on the block diagram position information through the sensor coordinate system relationship chain to inversely transform it from the pixel coordinate system to the camera coordinate system, and perform distortion correction on the geometric position deviation in the pixel coordinate system to obtain a normalized direction vector in the camera coordinate system; next, map the direction vector from the camera coordinate system to the UAV body coordinate system through rigid body transformation using the camera extrinsic parameters, rotate it according to the Euler angles, and then weight and superimpose it with the first position information to obtain the first predicted position corresponding to the first marker in the world coordinate system after rotating it from the UAV body coordinate system.

[0074] Specifically, the positional deviation value between the first predicted position and the first true position information is calculated based on the Euclidean distance or Manhattan distance, thereby obtaining the first deviation information.

[0075] In some embodiments, a deviation value is calculated based on first position information, distance information, direction information, and second true position information to obtain second deviation information. The obstacle avoidance method for the UAV includes: determining the relevant coordinate information of the second marker in the UAV body coordinate system corresponding to the UAV based on distance information and direction information; determining the second predicted position corresponding to the second marker based on the relevant coordinate information and first position information; and calculating the deviation based on the second predicted position and the second true position information to obtain the second deviation information.

[0076] For example, due to the penetrating nature of sound waves, the initial voiceprint information obtained by the UAV from collecting sound wave information implicitly contains highly accurate distance and direction information between the second marker and the UAV. Therefore, based on the highly accurate distance and direction information, this embodiment of the application further improves the accuracy of obtaining relevant coordinate information in the UAV body coordinate system by transforming from the acoustic coordinate system to the UAV body coordinate system based on the distance and direction information; it also determines the second predicted position corresponding to the second marker based on the relevant coordinate information and the first position information, further improving the accuracy of determining the second predicted position corresponding to the second marker based on the accurate relevant coordinate information; thereby improving the accuracy of calculating the distance and direction deviation between the second marker and the UAV based on the second predicted position and the second actual position information.

[0077] For example, firstly, the Euler angles output by the attitude sensor mounted on the UAV are obtained; then, based on the preset intrinsic and extrinsic parameter matrices of the microphone array, a rigid body transformation relationship between the microphone coordinate system and the UAV body coordinate system is constructed; then, the distance and direction information are rotated and translated in the coordinate system through the rigid body transformation relationship to obtain the direction vector in the microphone coordinate system; then, the direction vector is mapped from the microphone coordinate system to the UAV body coordinate system through the microphone array extrinsic parameter through rigid body transformation, and after rotation according to the Euler angles, it is weighted and superimposed with the first position information to obtain the second predicted position corresponding to the second marker in the world coordinate system rotated from the UAV body coordinate system.

[0078] For example, the positional deviation value between the second predicted position and the second true position information is calculated based on the Euclidean distance or Manhattan distance, thereby obtaining the second deviation information.

[0079] In some embodiments, the target edge information is adaptively adjusted based on the flight speed of the UAV to obtain the target edge region width. The obstacle avoidance method of the UAV includes: determining the initial edge region width and obtaining the corresponding maximum speed of the UAV; determining the speed ratio based on the flight speed and the maximum speed; determining the adaptive adjustment coefficient based on the preset adjustment parameter and the speed ratio; and adaptively adjusting the initial edge region width according to the adaptive adjustment coefficient to obtain the target edge region width.

[0080] Due to inertia and response delay during flight, if the flight path is generated directly based on target edge information without considering these factors, collisions between the drone and target obstacles are likely to occur. Therefore, this embodiment first determines the speed ratio based on the flight speed and the maximum speed, then determines an adaptive adjustment coefficient based on preset adjustment parameters and the speed ratio, and finally adaptively adjusts the initial edge region width according to the adaptive adjustment coefficient to obtain the target edge region width. This reduces the probability of collisions between the drone and target obstacles caused by inertia and response delay.

[0081] For example, the initial edge region width can be determined by presetting the edge region width, or it can be determined by querying a mapping table between the target edge information and the initial edge region width.

[0082] In some embodiments, the expression for the target edge region width is as follows:

[0083]

[0084] Among them, W t ′ The width of the adaptive edge region of the UAV's second position information at time t is represented by W, where W represents the preset edge region width, K represents the preset adjustment parameter, and vt v represents the real-time velocity of the drone at the second position information at time t. max This represents the maximum speed of the drone. In this embodiment, the ratio of the drone's real-time speed to its maximum speed is adjusted using preset adjustment parameters to obtain an adaptive adjustment coefficient for the edge region width. This adaptive adjustment coefficient then adaptively adjusts the edge region width, enabling the drone to determine the target edge region width while considering inertia and response delay, thereby reducing the probability of collision between the drone and the target obstacle and improving obstacle avoidance performance.

[0085] Understandably, since the preset adjustment parameters and the corresponding maximum speed of the drone are constant, the adaptive adjustment coefficient generated based on the preset adjustment parameters and flight speed is only affected by the drone's real-time speed. When the drone's real-time speed increases, the adaptive adjustment coefficient for the edge region width increases accordingly, thus increasing the target edge region width and preventing obstacle avoidance errors due to inertia and response delay. When the drone's real-time speed decreases, the coefficient for the edge region width decreases accordingly, thus decreasing the target edge region width, allowing the drone to dynamically adapt to the target edge region width based on its real-time speed.

[0086] For example, when K is 5 and v max For 100km / h, v t Given a speed of 10 km / h and a W value of 2 m, we can obtain

[0087] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the obstacle avoidance method for the drone described above. This electronic device can be any smart terminal, including a tablet computer, an in-vehicle computer, or similar device.

[0088] Please see Figure 3 , Figure 3 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0089] The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0090] The memory 302 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 302 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 302 and is called by the processor 301 to execute the obstacle avoidance method of the UAV according to the embodiments of this application.

[0091] Input / output interface 303 is used to implement information input and output;

[0092] The communication interface 304 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0093] Bus 305 transmits information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304);

[0094] The processor 301, memory 302, input / output interface 303, and communication interface 304 are connected to each other within the device via bus 305.

[0095] In some embodiments, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the obstacle avoidance method of the aforementioned drone.

[0096] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0097] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0098] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0099] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0100] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0101] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0102] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0103] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0104] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0106] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An obstacle avoidance method for unmanned aerial vehicles (UAVs), characterized in that, The method, applied to the controller of the drone, includes: The target environment information collected by the UAV is obtained, wherein the target environment information includes first image information, first location information and initial voiceprint information. The first image information is an image of the surrounding environment of the UAV at a first moment. The first location information is the location of the UAV at the first moment. The initial voiceprint information is the sound wave information of the surrounding environment of the UAV at the first moment. The first moment is the moment when the UAV reaches the preset initial range corresponding to the target obstacle during its flight along the first route. When it is determined that the UAV is located within a preset range of the target obstacle based on the first image information, the first location information, and the initial voiceprint information, the UAV is controlled to collect information about the target obstacle to obtain third image information, and target edge information is generated based on the third image information. The target edge information is adaptively adjusted based on the flight speed of the UAV to obtain the width of the target edge region; A second route is generated based on the width of the target edge region and the target edge information, and the UAV is controlled to perform obstacle avoidance flight based on the second route.

2. The obstacle avoidance method for a UAV according to claim 1, characterized in that, The method for determining that the UAV is located within a preset range of the target obstacle based on the first image information, the first location information, and the initial voiceprint information includes: The first image information is subjected to target recognition according to a preset target recognition model to obtain the first marker and the corresponding block diagram position information of the first marker in the first image information; The deviation value is calculated based on the block diagram location information, the first location information, and the first real location information of the first marker in the map database to obtain the first deviation information; Voiceprint recognition is performed on the initial voiceprint information to obtain the identification type of the second identifier, and positioning analysis is performed based on the initial voiceprint information to obtain the distance and direction information between the second identifier and the UAV; The second real location information of the second marker in the map database is determined based on the marker type, the distance information, and the direction information; The deviation value is calculated based on the first location information, the distance information, the direction information, and the second true location information to obtain the second deviation information; The first position information is corrected based on the first deviation information and the second deviation information to obtain the target position information of the UAV; Based on the target location information, determine whether the drone is located within a preset range of the target obstacle.

3. The obstacle avoidance method for a UAV according to claim 2, characterized in that, The preset target recognition model includes a motion parameter estimation layer, a first kernel function determination layer, a second kernel function determination layer, a first fuzzing layer, a second fuzzing layer, an image fusion layer, and a target recognition layer. The target environment information also includes second image information corresponding to a second time moment, the second time moment being acquired after the first time moment. The target recognition is performed on the first image information according to the preset target recognition model to obtain a first marker and the corresponding block diagram position information of the first marker in the first image information. The obstacle avoidance method of the UAV includes: The motion parameter estimation layer combines the flight parameter information of the UAV to determine the motion speed and direction of the UAV, and the first kernel function determination layer determines the first kernel function corresponding to the first image information based on the motion speed and the motion direction. The first image information is convolved by the first blurring layer based on the first kernel function to obtain the first processing result; The second kernel function is used to determine the target weights corresponding to the target-same features between the second image information and the first image information, and the second kernel function is determined based on the target weights and the target-same features. The second image information is convolved by the second blurring layer based on the second kernel function to obtain the second processing result; The first processing result and the second processing result are fused together by the image fusion layer to obtain the target fused image; The target recognition layer performs target recognition on the target fusion image to obtain the first marker and the corresponding block diagram location information.

4. The obstacle avoidance method for a UAV according to claim 3, characterized in that, The second kernel function determination layer includes a feature tracking unit, a weight determination unit, and a second kernel function determination unit. The second kernel function determination layer calculates the target weights corresponding to the same target features between the second image information and the first image information, and determines the second kernel function based on the target weights and the same target features. The obstacle avoidance method for the UAV includes: The feature tracking unit performs feature recognition on the second image information and the first image information respectively to obtain the first target feature corresponding to the second image information and the second target feature corresponding to the first image information, and calculates the similarity between the first target feature and the second target feature, and then determines the target's common features based on the similarity. The weight determination unit performs weighted fusion based on the second image information, the first image information, and the similarity to obtain the target weights with the same features as the target. The second kernel function is determined by the second kernel function determination unit based on the target weight and the target common features.

5. The obstacle avoidance method for a UAV according to claim 4, characterized in that, The obstacle avoidance method of the UAV includes the following steps: The weight determination unit performs weighted fusion based on the second image information, the first image information, and the similarity to obtain the target weights with the same features. Obtain first region information corresponding to the same features of the target in the first image information and second region information corresponding to the same features of the target in the second image information; Based on the first location information corresponding to the first image information, the first true coordinates corresponding to the first region information are determined, and based on the second location information corresponding to the second image information, the second true coordinates corresponding to the second region information are determined. The target offset corresponding to the same feature of the target is determined based on the first true coordinate and the second true coordinate, and the first weight is determined based on the target offset; Based on the first true coordinates and the second true coordinates, determine the first change amount corresponding to the same feature of the target in the horizontal position and the second change amount corresponding to the vertical position, and determine the second weight based on the sum of the first change amount and the second change amount; A third weight is determined based on the similarity, and the target weight is determined based on the first weight, the second weight, and the third weight.

6. The obstacle avoidance method for a UAV according to claim 2, characterized in that, The first deviation information is obtained by calculating the deviation value based on the block diagram location information, the first location information, and the first real location information of the first marker in the map database. The obstacle avoidance method of the UAV includes: Based on the preset camera intrinsic parameter matrix and preset camera extrinsic parameter matrix corresponding to the UAV, a sensor coordinate system relationship chain is constructed, and the block diagram position information is inversely transformed and distortion corrected according to the sensor coordinate system relationship chain to obtain the direction vector in the camera coordinate system. Based on the preset camera extrinsic parameters corresponding to the UAV, the direction vector is mapped to the UAV body coordinate system through rigid body transformation to obtain the intermediate coordinate information in the UAV body coordinate system. Based on the Euler angles corresponding to the UAV, the intermediate coordinate information is rotated and then superimposed with the first position information to obtain the first predicted position of the first marker in the world coordinate system. The deviation value is calculated based on the first predicted position and the first actual position information to obtain the first deviation information.

7. The obstacle avoidance method for a UAV according to claim 2, characterized in that, The obstacle avoidance method for the UAV includes calculating a deviation value based on the first location information, the distance information, the direction information, and the second true location information to obtain second deviation information. Based on the distance information and the direction information, determine the relevant coordinate information of the second marker in the drone body coordinate system corresponding to the drone. The second predicted position corresponding to the second marker is determined based on the relevant coordinate information and the first position information; The deviation is calculated based on the second predicted position and the second actual position information to obtain the second deviation information.

8. The obstacle avoidance method for a UAV according to claim 1, characterized in that, The obstacle avoidance method of the UAV includes adaptively adjusting the target edge information based on the UAV's flight speed to obtain the target edge region width. Determine the initial edge region width and obtain the maximum speed corresponding to the drone; The speed ratio is determined based on the flight speed and the maximum speed; The adaptive adjustment coefficient is determined based on the preset adjustment parameters and the speed ratio; The target edge region width is obtained by adaptively adjusting the initial edge region width according to the adaptive adjustment coefficient.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When at least one of the programs is executed by at least one of the processors, the obstacle avoidance method for the UAV as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used to execute the obstacle avoidance method of any one of the UAVs described in claims 1 to 8.

Images