Obstacle avoidance processing method and system based on unmanned aerial vehicle inspection

Abstract

The invention relates to the technical field of unmanned aerial vehicle obstacle avoidance inspection, and discloses an obstacle avoidance processing method and system based on unmanned aerial vehicle inspection, and the method comprises the steps: obtaining a visual image through a monocular camera carried by an unmanned aerial vehicle, carrying out the preprocessing of two adjacent frames of images, and extracting a matched feature point pair; and performing vertical and horizontal distribution statistics by calculating the scale ratio of the feature point pairs and combining a preset standard obstacle avoidance scale ratio so as to determine the distribution range of the obstacles. Based on the distribution range, whether the height of the obstacle is in a preset flight height interval can be judged. When the height of the obstacle is detected to be in the interval, an obstacle avoidance path is planned according to a preset safety window area and an obstacle distribution range, and obstacle avoidance flight control of the unmanned aerial vehicle is realized. The reliability of monocular vision in unmanned aerial vehicle obstacle avoidance application is effectively improved, and the technical problem that accurate obstacle avoidance cannot be achieved when the unmanned aerial vehicle flies due to the fact that a traditional monocular vision system is difficult to directly obtain accurate image information is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicle (UAV) obstacle avoidance inspection, and in particular to an obstacle avoidance processing method and system based on UAV inspection. Background Art

[0002] With the rapid advancement of science and technology, unmanned aerial vehicle (UAV) technology has achieved remarkable development over the past decade and is widely used in various fields, such as inspection, surveying and mapping, logistics, agriculture, environmental monitoring, and film and television production. With their advantages of flexibility, low cost, and easy deployment, drones have gradually become an indispensable tool in various industries. For example, in power inspections, drones can efficiently monitor transmission lines and substation equipment, promptly identifying potential safety hazards. In geographic surveying and mapping, drones equipped with high-resolution cameras can quickly acquire terrain data over large areas, greatly improving surveying efficiency and accuracy. In the logistics field, the use of drones facilitates fast and low-cost cargo delivery, especially in difficult transportation or emergency situations, demonstrating their unique advantages.

[0003] However, as drone applications continue to expand, their flight environments are becoming increasingly complex. When flying in urban environments with densely populated areas, drones are prone to encountering a variety of static and dynamic obstacles, such as buildings, power lines, trees, other aircraft, and even pedestrians. These obstacles not only threaten the safety of the drone itself but also pose potential dangers to surrounding personnel and property. Therefore, real-time, accurate, and efficient obstacle avoidance technology is crucial to ensuring safe drone flight and successful mission completion.

[0004] Currently, commercially available drone obstacle avoidance technologies primarily rely on a combination of binocular vision, LiDAR, ultrasonic sensors, and other sensors. Each of these technologies has its own advantages and disadvantages. Binocular vision systems mimic the workings of the human eye, acquiring two images with slight parallax to calculate depth information and thereby detect and locate obstacles. However, binocular vision systems have high lighting requirements and are prone to misjudgment on surfaces with monotonous or repetitive textures. Furthermore, binocular vision systems are computationally complex and require high processor performance, increasing the overall cost and energy consumption of drones.

[0005] Monocular vision is a low-cost, simple visual perception technology that uses a single camera to acquire image information and, combined with advanced image processing algorithms, can detect and locate obstacles. Compared to binocular vision, monocular vision systems offer significantly lower hardware costs and a more compact system, making them suitable for lightweight drone platforms.

[0006] However, existing monocular vision systems have inherent limitations in acquiring depth information. Relying solely on a single image, traditional monocular vision systems struggle to directly obtain accurate image information, making them incapable of precise obstacle avoidance. Summary of the Invention

[0007] The present invention provides an obstacle avoidance processing method and system based on drone inspection, which solves the technical problem that traditional monocular vision systems are difficult to directly obtain accurate image information, making it impossible for drones to accurately avoid obstacles during flight.

[0008] The first aspect of the present invention provides an obstacle avoidance method based on drone inspection, comprising:

[0009] Obtain visual images through the monocular camera carried by the drone;

[0010] Preprocessing the two adjacent frames of the visual image, and determining matching feature point pairs based on the preprocessing results;

[0011] Calculating the scale ratio of the matching feature point pairs, and performing vertical and horizontal distribution statistics on the matching feature point pairs in combination with a preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacle;

[0012] Based on the distribution range, determining whether the height of the obstacle is within a preset flight altitude range;

[0013] When the height of the obstacle is within the preset flight altitude range, an obstacle avoidance path is planned based on the preset safety window area and the distribution range, and obstacle avoidance flight control is performed on the UAV.

[0014] Optionally, preprocessing the two adjacent frames of the visual image and determining matching feature point pairs based on the preprocessing results includes:

[0015] Preprocessing two adjacent frames of the visual image to determine feature points corresponding to the visual images;

[0016] Performing brute force matching on the feature points corresponding to the visual images of two adjacent frames to obtain matching point pairs;

[0017] Determining a first rejection threshold based on calculating an average value of the Euclidean distances of all the matching point pairs;

[0018] When the Euclidean distance value of the matching point pair is greater than the first elimination threshold, the matching point pair is eliminated;

[0019] When the Euclidean distance value of the matching point pair is less than or equal to the first rejection threshold, the matching point pair is retained to obtain a first screening point pair group;

[0020] The first screening point pair group is screened to obtain matching feature point pairs.

[0021] Optionally, the filtering the first filtering point pair group to obtain matching feature point pairs includes:

[0022] Calculating the group scale ratio of the matching point pairs within the first screening point pair group;

[0023] When the group scale ratio meets a preset elimination condition, the matching point pair is eliminated from the first screening point pair group;

[0024] When the group scale ratio does not meet the preset elimination condition, the matching point pair is used as a matching feature point pair.

[0025] Optionally, the preset rejection condition is specifically that the group size ratio is less than a second rejection threshold, or the group size ratio is greater than a third rejection threshold.

[0026] Optionally, performing vertical and horizontal distribution statistics on the matching feature point pairs in combination with a preset standard obstacle avoidance scale to determine the distribution range of the obstacles includes:

[0027] When the scale ratio is greater than or equal to the preset standard obstacle avoidance scale ratio, the matching feature point pair is divided into a preset dangerous feature point pair group;

[0028] When the number of matching feature point pairs in the preset dangerous feature point pair group is greater than a preset value, the vertical and horizontal distribution of the feature points corresponding to the obstacles are counted;

[0029] The distribution range of the obstacle is determined according to the vertical and horizontal distribution, wherein the distribution range includes the left and right boundaries and the upper and lower boundaries of the obstacle in the visual image.

[0030] Optionally, the statistical analysis of the vertical and horizontal distribution of feature points corresponding to obstacles includes:

[0031] Dividing the visual image into a plurality of longitudinal regions and transverse regions, wherein each longitudinal region and each transverse region is provided with a corresponding distribution array;

[0032] According to the longitudinal coordinates of the feature points in the preset dangerous feature point pair group in the visual image, performing element frequency accumulation on the longitudinal distribution array corresponding to the longitudinal area containing the longitudinal coordinates;

[0033] Traversing the preset dangerous feature point pairs to obtain a longitudinal distribution;

[0034] According to the horizontal coordinate of the feature point in the visual image, performing element frequency accumulation on the horizontal distribution array corresponding to the horizontal area containing the horizontal coordinate;

[0035] The preset dangerous feature point pairs are traversed to obtain the horizontal distribution.

[0036] Optionally, determining the distribution range of the obstacles according to the longitudinal and transverse distribution includes:

[0037] Traversing the longitudinal distribution array, taking the first element in the longitudinal distribution array whose value is greater than a preset threshold as the first element;

[0038] The element whose last element value in the longitudinal distribution array is greater than the preset threshold is used as the second element;

[0039] Using the first column of the longitudinal area corresponding to the first element as the left limit of the obstacle in the visual image;

[0040] Using the last column of the longitudinal area corresponding to the second element as the right limit of the obstacle in the visual image;

[0041] Traversing the horizontal distribution array, taking the first element in the horizontal distribution array whose value is greater than the preset threshold as the third element;

[0042] The element whose last element value in the horizontal distribution array is greater than the preset threshold is used as the fourth element;

[0043] using the first row of the lateral area corresponding to the third element as the upper limit of the obstacle in the visual image;

[0044] The fourth element corresponds to the last row of the horizontal area as the lower limit of the obstacle in the visual image.

[0045] Optionally, planning an obstacle avoidance path based on a preset safety window area and the distribution range includes:

[0046] When the distribution range intersects with the preset safety window area, the left distance value and the right distance value of the drone to the left and right boundaries are determined according to the distribution range;

[0047] Comparing the left distance value with the right distance value, and selecting the side with the smaller distance value as the flight direction according to the comparison result;

[0048] When the obstacle is located on one side of the visual image, the other side of the visual image is used as the flight direction.

[0049] Optionally, it also includes:

[0050] Based on the triangle similarity theorem and the camera imaging principle, the actual obstacle avoidance physical width is determined as the safe obstacle avoidance distance of the UAV.

[0051] A second aspect of the present invention provides an obstacle avoidance processing system based on drone inspection, comprising:

[0052] A data acquisition module is used to acquire visual images through a monocular camera carried by the drone;

[0053] A screening and matching module is used to preprocess the two adjacent frames of the visual image and determine matching feature point pairs based on the preprocessing results;

[0054] an obstacle distribution range processing module, configured to calculate the scale ratio of the matching feature point pairs, and perform vertical and horizontal distribution statistics on the matching feature point pairs in combination with a preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacles;

[0055] A judgment module, configured to judge whether the height of the obstacle is within a preset flight altitude range based on the distribution range;

[0056] The output obstacle avoidance module is used to plan an obstacle avoidance path based on a preset safety window area and the distribution range when the height of the obstacle is within the preset flight altitude range, and to perform obstacle avoidance flight control on the UAV.

[0057] It can be seen from the above technical solutions that the present invention has the following advantages:

[0058] The present invention obtains visual images through a monocular camera carried by a drone, pre-processes two adjacent frames of images and extracts matching feature point pairs. By calculating the scale ratio of the feature point pairs and combining the preset standard obstacle avoidance scale ratio to perform vertical and horizontal distribution statistics, the distribution range of the obstacle is determined. Based on this distribution range, it is possible to determine whether the obstacle height is within the preset flight altitude range. When the obstacle height is detected to be within this range, an obstacle avoidance path is planned based on the preset safety window area and the obstacle distribution range to achieve obstacle avoidance flight control of the drone. This effectively improves the reliability of monocular vision in drone obstacle avoidance applications and solves the technical problem that traditional monocular vision systems are difficult to directly obtain accurate image information, making it impossible for drones to achieve accurate obstacle avoidance during flight. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1A flowchart of the steps of an obstacle avoidance method based on drone inspection provided in Example 1 of the present invention;

[0061] Figure 2 A flowchart of the steps of an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0062] Figure 3 A schematic diagram illustrating the principle of an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0063] Figure 4 A schematic diagram of camera imaging of an obstacle avoidance processing method based on drone inspection provided in the second embodiment of the present invention;

[0064] Figure 5 A schematic diagram of the area range of an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0065] Figure 6 A schematic diagram showing the lift direction of the rotor of a UAV when it is moving forward, according to a second embodiment of the present invention, based on a method for avoiding obstacles during inspection of a UAV.

[0066] Figure 7 A schematic diagram of imaging of a drone moving forward according to an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0067] Figure 8 A schematic diagram of a camera imaging model of a drone moving forward, according to a second embodiment of the present invention, in accordance with an obstacle avoidance method based on drone inspection;

[0068] Figure 9 A schematic diagram of an improved camera installation scheme for an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0069] Figure 10 A schematic diagram comparing four types of obstacle distribution situations in an obstacle avoidance method based on drone inspection provided in Example 2 of the present invention;

[0070] Figure 11 A schematic diagram of a camera front-facing imaging model for an obstacle avoidance method based on drone inspection provided in the second embodiment of the present invention;

[0071] Figure 12 This is a structural block diagram of an obstacle avoidance processing system based on drone inspection provided in Example 3 of the present invention. DETAILED DESCRIPTION

[0072] The embodiments of the present invention provide an obstacle avoidance processing method and system based on drone inspection, which is used to solve the technical problem that traditional monocular vision systems are difficult to directly obtain accurate image information, making it impossible for drones to accurately avoid obstacles during flight.

[0073] In order to make the purpose, features, and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0074] See also Figure 1 , Figure 1 This is a flowchart of the steps of an obstacle avoidance method based on drone inspection provided in Example 1 of the present invention.

[0075] The present invention provides an obstacle avoidance processing method based on drone inspection, comprising:

[0076] Step 101: Acquire a visual image through a monocular camera carried by a drone.

[0077] A monocular camera is a camera device equipped with a single optical sensor (single lens) that is used to capture two-dimensional RGB images from a single perspective.

[0078] Visual image refers to the two-dimensional RGB image captured by a monocular camera, that is, a digital image containing three-channel color information of red (R), green (G), and blue (B).

[0079] In an embodiment of the present invention, a two-dimensional RGB image is continuously acquired by a monocular camera carried by a drone (ie, two adjacent frames of two-dimensional RGB images are continuously captured using the monocular camera).

[0080] Step 102: pre-process two adjacent frames of visual images, and determine matching feature point pairs based on the pre-processing results.

[0081] Preprocessing refers to extracting feature points from two adjacent frames of visual images.

[0082] In this embodiment of the present invention, after continuously capturing visual images through a drone's monocular camera, two adjacent frames of visual imagery are first preprocessed to determine the image's feature points. A matching algorithm is then used to find and identify corresponding pairs of feature points between the two frames. These successfully matched feature point pairs serve as the basis for subsequent obstacle detection and obstacle avoidance path planning.

[0083] Step 103: Calculate the scale ratio of the matching feature point pairs, and perform vertical and horizontal distribution statistics on the matching feature point pairs in combination with the preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacles.

[0084] The scale ratio refers to the change in pixel size between a pair of matching feature points in two adjacent visual image frames. After extracting matching feature point pairs from the two frames through preprocessing and matching algorithms, the size ratio of the pixel coverage area of ​​the successfully matched feature point pair in the two frames is calculated.

[0085] The preset standard obstacle avoidance scale ratio refers to the critical threshold for safe obstacle avoidance that is set or calibrated experimentally, and is used to determine whether the dynamic changes of feature points pose a danger.

[0086] Vertical and horizontal distribution statistics refer to the spatial density statistics of matching feature point pairs in the image coordinate system, which respectively count the distribution density of feature points in the vertical direction (i.e., vertical direction) and the horizontal direction (i.e., horizontal direction) in the visual image.

[0087] The distribution range of obstacles refers to the boundary coordinates of obstacles in the visual image determined by vertical and horizontal distribution statistics, including upper and lower boundaries and left and right boundaries.

[0088] In this embodiment of the present invention, the scale change ratio of all successfully matched feature point pairs in two adjacent visual image frames is calculated (i.e., the scale ratio of the feature point size in the current frame relative to the previous frame). These scale ratios are then compared with a preset safety obstacle avoidance threshold to screen out dangerous feature point pairs that exceed the threshold. The spatial distribution of these dangerous feature points is then analyzed within the image coordinate system, with their dense areas counted horizontally (X-axis) and vertically (Y-axis). Finally, based on the statistical results, the specific area occupied by the obstacle in the visual image is determined, including the left and right boundaries and the upper and lower boundaries. This step achieves the critical transformation required to extract three-dimensional obstacle information from a monocular two-dimensional image sequence.

[0089] Step 104: Based on the distribution range, determine whether the height of the obstacle is within a preset flight altitude range.

[0090] The preset flight altitude range refers to the range of vertical flight space that is pre-set based on the UAV mission requirements and safety requirements.

[0091] In an embodiment of the present invention, based on the specific distribution range of the current obstacle, it is determined whether the height of the obstacle is within a preset flight altitude range, that is, whether the obstacle is close to the flight altitude of the drone.

[0092] Step 105: When the height of the obstacle is within the preset flight altitude range, an obstacle avoidance path is planned based on the preset safety window area and distribution range, and obstacle avoidance flight control is performed on the UAV.

[0093] The preset safety window area refers to the safe flight channel range pre-delineated in the drone's visual image. With the center of the visual image as the reference, a collision-free area of ​​a certain width is set symmetrically on both sides.

[0094] The obstacle avoidance path refers to the local heading adjustment plan calculated in real time by the drone to bypass obstacles, including distance priority strategy and unilateral obstacle avoidance strategy.

[0095] In this embodiment of the present invention, when an obstacle is within a preset flight altitude range, the drone makes an obstacle avoidance decision based on the spatial relationship between the horizontal safety window and the obstacle's distribution range. Heading control commands are generated based on this decision, adjusting the drone's flight path in real time. This achieves rapid horizontal obstacle avoidance response based on monocular vision, without relying on depth sensors or 3D reconstruction.

[0096] The present invention obtains visual images through a monocular camera carried by a drone, pre-processes two adjacent frames of images and extracts matching feature point pairs. By calculating the scale ratio of the feature point pairs and combining the preset standard obstacle avoidance scale ratio to perform vertical and horizontal distribution statistics, the distribution range of the obstacle is determined. Based on this distribution range, it is possible to determine whether the obstacle height is within the preset flight altitude range. When the obstacle height is detected to be within this range, an obstacle avoidance path is planned based on the preset safety window area and the obstacle distribution range to achieve obstacle avoidance flight control of the drone. This effectively improves the reliability of monocular vision in drone obstacle avoidance applications and solves the technical problem that traditional monocular vision systems are difficult to directly obtain accurate image information, making it impossible for drones to achieve accurate obstacle avoidance during flight.

[0097] See also Figure 2 , Figure 2 A flowchart of the steps of an obstacle avoidance method based on drone inspection provided in Example 2 of the present invention.

[0098] The present invention provides an obstacle avoidance processing method based on drone inspection, comprising:

[0099] Step 201: Acquire a visual image through a monocular camera carried by a drone.

[0100] In the embodiment of the present invention, the specific implementation process of step 201 is similar to that of step 101 and will not be repeated here.

[0101] Step 202: pre-process two adjacent frames of visual images, and determine matching feature point pairs based on the pre-processing results.

[0102] Furthermore, step 202 may include the following sub-steps:

[0103] S11. Preprocess two adjacent frames of visual images to determine feature points corresponding to the visual images.

[0104] In an embodiment of the present invention, feature points are extracted from two adjacent RGB visual image frames using an improved ORB algorithm (Oriented FAST and Rotated BRIEF, ORB).

[0105] It should be noted that, in specific applications, the improved ORB algorithm is used for feature point extraction and matching. Compared to traditional algorithms, this embodiment of the present invention maintains high matching accuracy while significantly improving computational speed. This enables the drone to process continuously acquired image data in real time during flight, rapidly respond to environmental changes, and achieve instant obstacle avoidance. This efficient computing performance ensures stable operation even at high speeds or in complex environments, avoiding collision risks caused by computational delays.

[0106] Furthermore, S11 may include the following sub-steps:

[0107] A1. Perform image conversion on two frames of RGB visual images to obtain corresponding grayscale images;

[0108] A2. Based on each pixel in the grayscale image, calculate the brightness values ​​of multiple initial pixels within a circle with a preset radius for each pixel;

[0109] In the embodiment of the present invention, for each pixel point P, the brightness values ​​of multiple initial pixel points P1 within a circle with a preset radius of each pixel point P are calculated. For ease of understanding, it is assumed that there are 16 initial pixel points P1 within the circle with the preset radius.

[0110] A3. Eliminate the initial pixels corresponding to the maximum brightness value and the minimum brightness value, and calculate the average brightness value of the remaining initial pixels;

[0111] In the embodiment of the present invention, the maximum brightness value Imax and the minimum brightness value Imin are screened out from the brightness values ​​of the multiple initial pixel points P1, and the average brightness value Ia of the remaining initial pixel points P1 is calculated, that is, the average brightness value is calculated using the remaining 14 initial pixel points P1;

[0112] A4. Selecting a first preset number of initial pixel points within a circle of a preset radius as equidistant pixel points;

[0113] The first preset number refers to selecting a certain number of equidistant initial pixel points (for example, the 1st, 5th, 9th, and 13th) within a circle of a preset radius.

[0114] In the embodiment of the present invention, a certain number of equidistant initial pixel points are selected from a circle of a preset radius according to a preset selection rule as the equidistant pixel points P2.

[0115] A5. Calculate the absolute value of the difference between the brightness value associated with the equidistant pixel points and the brightness value associated with the pixel points to determine a first difference value corresponding to the equidistant pixel points;

[0116] In the embodiment of the present invention, the above process is converted into a formula, specifically:

[0117]

[0118] Where, Indicates the The brightness value associated with each of the equally spaced pixels, Indicates the brightness value associated with the pixel.

[0119] A6. Determine the target screening threshold using the brightness value, average brightness value, maximum brightness value, minimum brightness value, and preset dynamic coefficient associated with equidistant pixels;

[0120] In the embodiment of the present invention, the above process is converted into a formula, specifically:

[0121]

[0122] Where, Indicates the target screening threshold associated with equidistant pixels, Indicates the preset dynamic coefficient.

[0123] A7. When the first difference corresponding to the preset first screening number of equally spaced pixel points is greater than the target screening threshold, the pixel point is selected as a candidate pixel point;

[0124] The preset first screening number refers to a pre-set screening number condition, that is, a pixel point is considered as a candidate pixel point only when the preset first screening number is met.

[0125] In the embodiment of the present invention, the above process is converted into a formula, specifically:

[0126]

[0127] It should be noted that pixel points are only considered as candidate pixel points when they meet the preset first screening number. For example, if the number of equidistant pixel points is 4, then the 75% number needs to be met, that is, the first difference corresponding to at least 3 equidistant pixel points must be greater than the target screening threshold before the associated pixel points are considered as candidate pixel points.

[0128] A8. Select a second preset number of initial pixels within a circle with a preset radius of the candidate pixel as target pixels;

[0129] The second preset number refers to the total number of initial pixels within a circle of a preset radius minus the first preset number.

[0130] In the embodiment of the present invention, an initial pixel point that is not defined as an equidistant pixel point within a circle of a preset radius is used as the target pixel point P3.

[0131] A9. Calculate the absolute value of the difference between the brightness value associated with the target pixel and the brightness value associated with the candidate pixel to determine a second difference value corresponding to the target pixel;

[0132] In the embodiment of the present invention, the above process is converted into a formula, specifically:

[0133]

[0134] Where, Indicates the The brightness value associated with each target pixel.

[0135] A10. When the second difference corresponding to the target screening threshold of the preset second screening number is greater than the target screening threshold, the candidate pixel point is used as a feature point.

[0136] It should be noted that the target screening threshold is a dynamic screening threshold, which can be calculated using the formula in A6 above, specifically:

[0137]

[0138] Where, Target filtering threshold representing target pixel association

[0139] The preset second screening number refers to a pre-set screening number condition, that is, a pixel point is used as a feature point only when the preset second screening number is met.

[0140] In the embodiment of the present invention, the above process is converted into a formula, specifically:

[0141]

[0142] It should be noted that pixel points are only considered as candidate pixel points when the preset second screening number is met. For example, if the number of target pixel points is 12 (16-4=12), then the associated pixel points are considered as feature points only when 75% of the number is met, that is, the first difference corresponding to at least 9 target pixel points must be greater than the target screening threshold.

[0143] S12. Perform brute force matching on the feature points corresponding to two adjacent frames of visual images to obtain matching point pairs.

[0144] In the embodiment of the present invention, Hamming distance is used to perform brute force matching on feature points to screen out matching point pairs with high similarity.

[0145] In the specific implementation, the feature points are first represented by binary strings. The descriptor of each feature point is a binary string of fixed length. The binary descriptor of any feature point in the current frame visual image is compared with the binary descriptor of any feature point in the adjacent frame visual image. The comparison starts from the first bit. The number of different bits is the Hamming distance between the binary descriptors of the two feature points. For example, if the binary descriptor of the current frame visual image is "0101" and the binary descriptor in the adjacent frame visual image is "0011", the second and third bits are different, and the Hamming distance is 2. Then all feature point combinations are traversed and the Hamming distance between each pair is calculated. By setting a Hamming distance threshold, feature point pairs with a Hamming distance less than the set Hamming distance threshold are judged to have high similarity, and the feature point pair is used as a matching point pair.

[0146] S13. Determine a first rejection threshold based on calculating the average Euclidean distance of all matching point pairs.

[0147] In this embodiment of the present invention, during the point matching process, the Euclidean distance between different matching point pairs reflects the degree of similarity or difference between the matching points in the feature space. Calculating the average Euclidean distance of all matching point pairs yields a value representative of the overall distance between the matching point pairs. Therefore, the Euclidean distance values ​​for each matching point pair are first calculated, then averaged. This average Euclidean distance is then multiplied by a preset coefficient, and the resulting value serves as the first rejection threshold.

[0148] S14: When the Euclidean distance value of the matching point pair is greater than a first elimination threshold, the matching point pair is eliminated.

[0149] In the embodiment of the present invention, when the Euclidean distance value of the matching point pair is greater than the first elimination threshold, the matching point pair is eliminated.

[0150] S15. When the Euclidean distance value of the matching point pair is less than or equal to the first elimination threshold, the matching point pair is retained to obtain a first screening point pair group.

[0151] In the embodiment of the present invention, when the Euclidean distance value of the matching point pair is less than or equal to the first elimination threshold, the matching point pair is retained to obtain a first screening point pair group, and the first elimination is completed.

[0152] S16. Filter the first filter point pair group to obtain matching feature point pairs.

[0153] It should be noted that the group of feature point pairs removed for the first time is then screened again to remove mismatched points.

[0154] Furthermore, the preset rejection condition is specifically that the group size ratio is less than the second rejection threshold, or the group size ratio is greater than the third rejection threshold. S16 may include the following sub-steps:

[0155] S161: Calculate the group scale ratio of the matching point pairs in the first screening point pair group.

[0156] In the embodiment of the present invention, the group scale ratio of the matching point pairs in the first screening point pair group is calculated, and the matching point pairs are represented as 、 ,in, and Indicates the visual image sequence number where the matching point is located, Represents the matching point sequence number of the matching point in each visual image. First, determine the scale group to which each matching point pair in the first screening point pair group belongs (the scale group is based on the scale space layering during feature extraction, such as the pyramid level or scale parameter of the matching algorithm). Assume Belong to visual images No. group, whose group scale parameter is , assuming Belong to visual images No. group, whose group scale parameter is , then the group scale ratio is .

[0157] S162: When the group scale ratio meets a preset elimination condition, the matching point pair is eliminated from the first screening point pair group.

[0158] S163: When the group scale ratio does not meet the preset elimination condition, the matching point pair is used as a matching feature point pair.

[0159] In the embodiment of the present invention, the second rejection threshold is preferably 1, and the third rejection threshold is preferably threshold T. , then the group scale parameter of the second matching point is determined to be larger than that of the first matching point, and it is regarded as a wrong matching point and is removed. , the group scale parameter of the second matching point is determined to be too large than that of the first matching point, and it is also regarded as a wrong matching point and eliminated, and the rest are retained, and finally the matching feature point pairs with high similarity are output.

[0160] It should be noted that in the above process, the visual image is first preprocessed to determine the feature points, and then the Hamming distance is used to perform brute force matching on the feature points to screen out the matching point pairs with high similarity. Then, the first elimination and screening operation is performed to eliminate and screen the matching point pairs by calculating the average value of the Euclidean distance. As the drone approaches the obstacle, the distance between the object and the drone becomes closer and closer, so the size of the object in the visual image becomes larger and smaller, and the scale becomes smaller and smaller. Based on this feature, the first filtered point pair group after the first elimination and screening operation is again eliminated for mismatching. According to the actual situation, the group scale is larger than that in The obstacle avoidance has started when , and will not start obstacle avoidance when it is greater than T. It is also regarded as an error and eliminated, and the rest are retained.

[0161] Step 203 : Calculate the scale ratio of the matching feature point pairs, and perform vertical and horizontal distribution statistics on the matching feature point pairs in combination with the preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacles.

[0162] It should be noted that the preset standard obstacle avoidance scale ratio avoidmultiple is the standard obstacle avoidance scale ratio corresponding to obstacles that are farther away from the drone than the safe distance. It can be a preset value obtained based on historical experience, or it can be calculated based on the shooting distance between two adjacent frames of visual images and the flight speed safety distance.

[0163] It should be noted that the scale ratio of the matching feature point pair is similar to the above group scale ratio. For example, the detection scale of a feature point in the current frame visual image is , the detection scale of a feature point in the adjacent frame visual image is , then the scale ratio , which reflects the size change of feature points caused by the movement of the monocular camera or the change of object distance.

[0164] Furthermore, step 203 may include the following sub-steps:

[0165] It should be noted that two feature point pairs are set: safepoints (i.e., safe feature point pair group) and dangerpoints (i.e., dangerous feature point pair group). Then, based on the shooting distance between two adjacent frames of visual images and the flight speed safety distance, the standard obstacle avoidance scale ratio (avoidmultiple) corresponding to obstacles that are farther away from the drone than the safe distance is calculated. Specifically,

[0166]

[0167] Where, Indicates the shooting distance. Indicates the flight speed safety distance.

[0168] S21. When the scale ratio is greater than or equal to the preset standard obstacle avoidance scale ratio, the matching feature point pairs are divided into a preset dangerous feature point pair group.

[0169] In the embodiment of the present invention, before counting the number of feature points whose scale ratio is greater than or equal to the set standard obstacle avoidance scale ratio in the longitudinal region and the transverse region, it is necessary to group the matching feature point pairs. It should be noted that since the matching feature point pairs are obtained through the screening of matching point pairs, the matching feature point pairs here are consistent with the matching point pairs, both of which are 、 , when the scale ratio of the matching feature point pair is greater than or equal to the preset standard obstacle avoidance scale ratio, Classify as dangerpoints, otherwise classify as safepoints.

[0170] S22. When the number of matching feature point pairs in the preset dangerous feature point pair group is greater than a preset value, the vertical and horizontal distribution of the feature points corresponding to the obstacles are counted.

[0171] It should be noted that when the number of matching feature point pairs in the preset dangerous feature point pair group is greater than the preset number value, that is, the number of feature points in dangerpoints is greater than the preset number value When , the vertical and horizontal distribution of the feature points corresponding to the obstacles are counted.

[0172] Furthermore, S22 may include the following sub-steps:

[0173] S221. Divide the visual image into a plurality of longitudinal regions and transverse regions, wherein each longitudinal region and transverse region is provided with a corresponding distribution array.

[0174] In this embodiment of the present invention, the visual image P containing the obstacle in the current frame is t It is divided into several vertical areas and horizontal areas. Each vertical area corresponds to a vertical distribution array verticle, and each horizontal area corresponds to a horizontal distribution array transverse.

[0175] It should be noted that for the division of vertical regions, first take the 1st column to the widthth column as the first vertical region, and shift right by step pixels to form the 2nd, 3rd, ..., (cols-width) / step+1st vertical regions, corresponding to a vertical distribution array verticle.

[0176] For the horizontal region division, first take the 1st row to the stepth row as the first horizontal region, and then move down step pixels in sequence as the 2nd, 3rd, ..., (rows-width) / step+1th horizontal regions, corresponding to a horizontal distribution array transverse.

[0177] S222. According to the longitudinal coordinates of the feature points in the preset dangerous feature point pair group in the visual image, perform element frequency accumulation on the longitudinal distribution array corresponding to the longitudinal area containing the longitudinal coordinates.

[0178] S223: Traverse the preset dangerous feature point pairs to obtain the longitudinal distribution.

[0179] In an embodiment of the present invention, according to the longitudinal coordinates of the feature points in the preset dangerous feature point pair group, the corresponding element of the verticle corresponding to the longitudinal area containing the longitudinal coordinates is added by 1, and the feature points in dangerpoints are traversed to obtain the longitudinal distribution of the feature points whose scale ratio is greater than or equal to the set standard obstacle avoidance scale ratio.

[0180] S224. According to the horizontal coordinates of the feature points in the visual image, perform element frequency accumulation on the horizontal distribution array corresponding to the horizontal area containing the horizontal coordinates.

[0181] S225. Traverse the preset dangerous feature point pairs to obtain the horizontal distribution.

[0182] In an embodiment of the present invention, according to the horizontal coordinates of the feature points in the visual image, the corresponding element of transverse corresponding to the horizontal area containing the horizontal coordinates is added by 1, and the feature points in dangerpoints are traversed to obtain the horizontal distribution of feature points whose scale ratio is greater than or equal to the set standard obstacle avoidance scale ratio.

[0183] S23. Determine the distribution range of the obstacle based on the vertical and horizontal distribution, where the distribution range includes the left and right boundaries and the upper and lower boundaries of the obstacle in the visual image.

[0184] In the embodiment of the present invention, the distribution range of the obstacle is determined by determining the left and right boundaries and the upper and lower boundaries of the obstacle in the visual image Pt according to the vertical and horizontal distribution of the feature points.

[0185] Furthermore, S23 may include the following sub-steps:

[0186] S231 , traverse the vertical distribution array, and take the first element in the vertical distribution array whose value is greater than a preset threshold as the first element.

[0187] S232: The element whose last element value in the vertical distribution array is greater than a preset threshold is used as the second element.

[0188] S233. Use the first column of the longitudinal area corresponding to the first element as the left limit of the obstacle in the visual image.

[0189] S234. Use the last column of the longitudinal area corresponding to the second element as the right limit of the obstacle in the visual image.

[0190] In this embodiment of the present invention, traverse the verticles to find the first and last values ​​greater than a preset threshold. Elements (including the first element and the second element ), Corresponding to the first column of the vertical area, The last column of the corresponding vertical area is the left limit of the dangerpoints distribution range and right limit , where the preset threshold Preferably 2.

[0191] S235 , traverse the horizontal distribution array, and take the first element in the horizontal distribution array whose value is greater than a preset threshold as the third element.

[0192] S236: The element whose last element value in the horizontal distribution array is greater than a preset threshold is used as the fourth element.

[0193] S237. Use the first row of the horizontal area corresponding to the third element as the upper limit of the obstacle in the visual image.

[0194] S238. Use the last row of the horizontal area corresponding to the fourth element as the lower limit of the obstacle in the visual image.

[0195] In the embodiment of the present invention, the transverse is traversed to find the first and last values ​​greater than the preset threshold. Elements (including the third element and the fourth element ), Corresponding to the first row of the horizontal area, The last row of the corresponding horizontal area is the upper limit of the dangerpoints distribution range and lower bound .

[0196] Step 204: Based on the distribution range, determine whether the height of the obstacle is within a preset flight altitude range.

[0197] In the embodiment of the present invention, a height difference threshold is first set. , determine the actual height range corresponding to the upper and lower limits of the obstacle in the visual image , obtain the current flight altitude of the drone through the altitude measurement equipment carried by the drone ,like and , that is, the height range of the obstacle Completely within the preset flight altitude range , then it is determined that the obstacle and the UAV’s flight altitude are similar, otherwise it is determined that the two altitudes are not similar.

[0198] Step 205: When the height of the obstacle is within the preset flight altitude range, an obstacle avoidance path is planned based on the preset safety window area and distribution range, and obstacle avoidance flight control is performed on the UAV.

[0199] Furthermore, step 205 may include the following sub-steps:

[0200] S31. When the distribution range intersects with the preset safety window area, the left distance value and the right distance value from the UAV to the left and right boundaries are determined according to the distribution range.

[0201] In the embodiment of the present invention, by comparing the distribution range of the obstacle and the coordinate information of the preset safety window area (such as the left and right limit coordinates of the horizontal area, and the upper and lower limit coordinates of the vertical area), it is determined whether the two have an intersection. If there is no intersection, it means that the obstacle will not affect the flight of the drone under the current circumstances. If there is an intersection, the coordinate position of the left and right limits of the obstacle in the visual image (for example, the horizontal coordinate of the left limit is , the horizontal coordinate of the right limit is ) Calculate the central lateral coordinate of the obstacle , specifically:

[0202]

[0203] Get the current lateral coordinates of the drone in the visual image , respectively calculate the left distance value from the drone to the left limit of the obstacle The right distance from the drone to the right limit of the obstacle .

[0204] S32. Compare the left distance value with the right distance value, and select the side with the smaller distance value as the flight direction according to the comparison result.

[0205] In the embodiment of the present invention, the comparison and The size of the object is determined by the distance, and the closer side is selected as the flight direction to avoid obstacles.

[0206] S33. When the obstacle is located on one side of the visual image, the other side of the visual image is used as the flight direction.

[0207] In an embodiment of the present invention, when an obstacle is located on one side of the visual image, the left and right boundary coordinates of the obstacle are compared with the image boundary coordinates (for example, the left boundary coordinate of the image is 0, and the right boundary coordinate is the image width value) to determine whether the obstacle is located on the left or right side of the visual image. If the obstacle is located on the left side of the image, the drone will give priority to flying to the right to avoid the obstacle. If the obstacle is located on the right side of the image, the drone will give priority to flying to the left to avoid the obstacle.

[0208] See also Figure 3 The present invention adopts monocular vision obstacle avoidance technology. Monocular vision obstacle avoidance is to identify obstacles from continuous two-dimensional RGB visual images through a specific algorithm and avoid obstacles through path planning. The specific obstacle avoidance process is as follows:

[0209] The monocular camera onboard the drone acquires two adjacent frames of two-dimensional RGB visual images (i.e., a monocular camera is used to continuously capture two two-dimensional RGB visual images). Based on the improved ORB algorithm, feature points are extracted from the two consecutive RGB images, and matching point pairs are obtained by Hamming distance matching. Then, a secondary screening is performed to select matching feature point pairs with high similarity. The scale ratio of the matching feature point pairs is calculated. Feature points with a scale ratio greater than the set standard obstacle avoidance scale ratio are grouped, and their longitudinal and lateral distribution in the image is counted to determine the specific distribution range of the current obstacle. Based on the specific distribution range of the current obstacle, it is determined whether the obstacle is close to the flight altitude of the drone. If so, a safety window area is set, and an obstacle avoidance path is planned based on the left and right positions of the obstacle in the image, choosing to fly left or right to avoid the obstacle.

[0210] Analysis of the technical solution for the obstacle avoidance method for drone inspections reveals that: First, image acquisition and preprocessing are performed. The drone is equipped with a monocular camera, which continuously captures a sequence of two 2D RGB images. These two 2D RGB images are then preprocessed, including denoising and grayscaling, to improve the accuracy of subsequent feature extraction.

[0211] Then, based on the improved ORB algorithm, feature points and their descriptors are extracted from the two consecutive images. A brute-force matching of the feature points is performed using the Hamming distance to identify highly similar matching pairs. The average Euclidean distance of all matching points is calculated. A threshold is set, and matching points with a Euclidean distance greater than the threshold are eliminated to obtain highly similar matching feature point pairs. Simultaneously, obstacle detection and distribution range determination are performed: matching feature points are grouped vertically and horizontally, and the distribution of feature points with a scale ratio greater than the threshold is statistically analyzed within the image. The left and right, upper and lower bounds of the obstacle in the image are determined to determine its specific distribution range. Obstacle avoidance path planning is implemented: the obstacle is determined to be at a similar altitude to the drone. If so, it is further determined whether obstacle avoidance is necessary. A safety window is set. If the obstacle distribution range intersects with the safety window, the obstacle avoidance direction (left or right) is determined based on the left and right position of the obstacle. The required flight distance for obstacle avoidance is calculated to ensure that the drone maintains a safe distance from the obstacle after avoiding it.

[0212] Step 206: Based on the triangle similarity theorem and the camera imaging principle, the actual obstacle avoidance physical width is determined as the safe obstacle avoidance distance of the UAV.

[0213] In an embodiment of the present invention, based on the triangle similarity theorem and the camera imaging principle, the pixel width occupied by the obstacle in the visual image is used, combined with the camera focal length and field of view angle, to calculate the actual obstacle avoidance physical width, and the actual obstacle avoidance physical width is used as the safe obstacle avoidance distance of the drone to ensure the safe distance after obstacle avoidance.

[0214] Furthermore, step 206 may include the following sub-steps:

[0215] S41. Acquire an image area directly in front of the drone, where the image area includes the obstacle area AB, the range EF of the obstacle in the image, the range CD of the object corresponding to the image area, and the image GH of the image area, which can be expressed as the number of image columns cols, the width L2 of the object corresponding to the image area, the distance S1 between the camera and the obstacle, the distance S2 between the camera and the object corresponding to EF, the horizontal field of view HFOV of the camera, and the focal length f of the camera;

[0216] S42. Based on the triangle similarity theorem and the camera imaging principle, the image area is used to solve and obtain the width of the obstacle;

[0217]

[0218] Combine the following formula:

[0219]

[0220] The width L1 of the obstacle is obtained by joint solution:

[0221]

[0222] S43. After the drone avoids the obstacle, the drone is controlled to maintain a physical width corresponding to a preset half-width pixel between the drone and the obstacle.

[0223] If you dodge to the left, you need to fly The physical width corresponding to the pixel;

[0224] If you dodge to the right, you need to fly The physical width in pixels.

[0225] Where, Indicates the width of the image in pixels. Indicates the preset half width, Indicates the pixel index of the middle position in the horizontal direction of the image. Specifically, it refers to a reference position in the horizontal direction of the image, which is used to determine the position relationship of the obstacle relative to the center of the image.

[0226] Specific examples:

[0227] The obstacle avoidance technology adopted in the embodiment of the present invention is to first match two continuous visual images, then calculate the size change of each feature point, and based on the relationship between distance and size, and based on an improved ORB algorithm, according to the scale ratio of the matching feature points and the triangle similarity theorem, generate a depth map, and then use this as the basis for obstacle avoidance path planning.

[0228] (1) According to the relationship between the distance between the camera and the object and the image scale ratio: when taking images at different distances from the object, the size of the object in the image is different. It is not difficult to find that the closer the camera is to the object, the larger the image size of the object in the camera and the smaller the scale. The imaging effect is as follows: Figure 4 As shown:

[0229] exist Figure 4 middle, is the actual size of the object, is the focal length of the camera. The camera captures image 1 at the optical center 1, where the distance between the optical center 1 and the object is , the image size of the object in the camera is ; Then the camera moves toward the obstacle to the optical center 2 to obtain image 2, where the distance between the optical center 2 and the object is , the image size of the object in the camera is 2.

[0230] According to the triangle similarity theorem and the camera imaging principle, we can obtain formula (1):

[0231] (1)

[0232] Combining the above formulas, we can get:

[0233] (2)

[0234] make , that is, the size (degree) ratio of the object in the latter image to the object in the previous image, and The distance is ,Right now , substitute into formula (3):

[0235] (3)

[0236] According to formula (3), we can get:

[0237] (4)

[0238] The study found that the obstacle avoidance strategy of drones is generally when the distance between the drone and the object is less than a certain limit, that is, the safe distance When , dodge. , we can get:

[0239] (5)

[0240] Assuming that the distance between the two images is 1m and the safety distance is 5m, according to formula (5) That is, when the size ratio of an object in two images is greater than 1.2, the distance between the second image and the object is less than 5 meters. Therefore, there is no need to calculate the actual distance between the camera and the object; the image size ratio and the distance between the images can be used to determine whether the distance is below the safe distance.

[0241] (2) Calculation of scale ratio of matching feature points

[0242] Then, based on the improved ORB algorithm, the feature points of the two visual images are extracted and the descriptors are calculated. Brute force matching is performed based on Hamming similarity. The Hamming similarity can be easily calculated through logical operations such as XOR, and the average Euclidean distance of all matching points is calculated. , combined with the preset coefficient , set the first rejection threshold ,If the Euclidean distance of the matching point is greater than the first elimination threshold, it will be eliminated, otherwise it will be retained.

[0243] Then, we can remove the object again based on the group where the feature points are located. As the drone approaches the obstacle, the distance between the object and the drone becomes closer and closer, so the size of the object in the image becomes larger and smaller. Based on this feature, we can remove the mismatch again: let a pair of matching feature points 、 ,in, and Indicates the visual image sequence number where the matching point is located, Indicates the matching point sequence number of the matching point in each visual image. Let The scale of the group The scale ratio of the group is ,like , then the group scale parameter of the second matching point is determined to be larger than that of the first matching point, and it is regarded as a wrong matching point and is removed. , then the group scale parameter of the second matching point is determined to be too large than that of the first matching point, and it is also regarded as a wrong matching point and is removed, and the rest are retained, and finally the matching feature point pairs with high similarity are output. According to the actual situation, the group scale ratio is The obstacle avoidance has started when , and will not start obstacle avoidance when it is greater than T. It is also regarded as an error and eliminated, and the rest are retained.

[0244] (3) Distribution range of obstacles

[0245] According to the previous conclusion, the feature points with a scale ratio greater than or equal to the threshold correspond to the distance between the object and the camera, that is, the distance between the drone and the camera is less than the safe distance. The distribution range of these feature points is the distribution range of obstacles that need to be avoided.

[0246] The specific calculation method for obstacle distribution range is as follows.

[0247] 1) Each matching feature point is grouped according to the scale ratio: two feature point queues, safepoints and dangerpoints, are set, and then the scale ratio avoidmultiple corresponding to obstacles whose distance from the drone is greater than the safe distance is calculated based on the shooting distance of the two visual images and the safe distance of the flight speed. 、 The scale is like , then Add avoidmultiple, otherwise add safepoints. Finally, if the number of matching feature points in avoidmultiple , Is a preset number value, then the feature points in avoidmultiple are counted in image P ti (represents the distribution range of the i-th visual image), otherwise it continues to fly in the original direction without obstacle avoidance.

[0248] 2) Statistical analysis of the vertical and horizontal distribution of feature points corresponding to obstacles: Image resolution P ti The first column to the width column in Figure 5 ACGE region in the image P ti The first vertical area, and then move the area to the right step, step column to the width + step column area, that is Figure 5 The BDHF area in the middle is used as the second vertical area, and then the area is moved right by step pixels as the third vertical area, and the area is moved until the edge of the image. For the convenience of calculation, step is the common divisor of cols and rows, and width is an integer multiple of step. Finally, the image P ti Contains (cols-width) / step+1 vertical regions. Set up an array verticle with a length of (cols-width) / step+1. Each element in the array corresponds to a vertical region, and the initial value of each element is 0.

[0249] Traverse the feature points in dangerpoints and, based on the coordinates of the feature point, increment the corresponding element of the verticle array corresponding to the vertical region containing the coordinates by 1. For example, if the coordinates of a feature point in dangerpoints are (100, 100), and the 6th and 7th vertical regions contain the point, then increment the 6th and 7th elements of the verticle array by 1. After the traversal is complete, the vertical distribution of feature points with a scale ratio greater than the obstacle avoidance scale ratio can be obtained. Next, use rows 1 to step as the first horizontal region, and shift them down by step pixels to form the 2nd, 3rd, ..., (rows-width) / step+1 horizontal regions, corresponding to an array transverse. Traverse dangerpoints in the same way to obtain the horizontal distribution of feature points.

[0250] 3) Obtain the left, right, and upper and lower limits of the obstacle based on the vertical and horizontal distribution: Finally, obtain the left, right, and upper and lower limits of the obstacle based on verticle and transverse. The specific method is: traverse the verticle to find the first and last elements whose values ​​are greater than ε 、 , Corresponding to the first column of the vertical area, The last column of the corresponding vertical area is the left and right limits of the dangerpoints distribution range 、 ; Then traverse the transverse and find the first and last values ​​greater than Elements 、 , Corresponding to the first row of the horizontal area, The last row of the corresponding area is the upper and lower limits of the dangerpoints distribution range 、 , preset threshold Preferably 2.

[0251] Obstacle Avoidance Path Planning: The distance between the object and the drone is determined based on the scale ratio of the matching feature points in the two images. The location of these feature points then determines the distribution of obstacles in the image. Next, we use this distribution to determine whether and how to avoid obstacles.

[0252] (1) The position of objects at the same height as the drone in the image

[0253] Generally speaking, when avoiding obstacles, a drone only needs to consider obstacles that are about the same height as the drone. Even if other obstacles with a large height difference are detected, they do not need to be avoided. When the drone body is parallel to the ground, the image directly in front is naturally the neighborhood image near the middle row of the image. However, due to the characteristics of multi-rotor drones, the fuselage is not parallel to the ground during the forward movement, but has a pitch angle with the horizontal direction of the ground. In this way, the force generated by the rotor of the multi-rotor drone has an angle with the vertical direction, which can be decomposed into an upward force and a forward force to maintain the height of the drone and propel the drone forward. Figure 6 Therefore, if the camera is installed in front of the drone, the image captured is not the image in front of the drone’s direction of travel, but the image diagonally below. The imaging model is as follows: Figure 7 shown.

[0254] Among them, OO1 is the direction of the optical center, CD is the actual range of the imaged object, AB is the image of the object in the camera, and CE is the projection of the image. With O as the origin, the direction parallel to the ground (drone forward) as the x-axis, and the direction perpendicular to the ground as the y-axis, a coordinate system is established, such as Figure 8 As shown:

[0255] Among them, VFOV is the vertical field of view of the camera, which is a fixed parameter inside the camera. is the pitch angle of the drone during flight (the pitch angle varies at different speeds), AB is the image in the camera, and its size can be expressed as image data, that is, the number of rows of the image, CD is the area of ​​the imaged object, CE is the projection of the imaged area, the distance between the camera and the imaged object is s, f is the focal length of the camera, point F is the point directly in front of the camera in the horizontal direction, I is the corresponding position of this point in the image projection, and J is the corresponding position in the camera image. According to the triangle similarity theorem and the camera imaging principle, we can get:

[0256] (6)

[0257] It can be deduced that:

[0258] (7)

[0259] in, , ,but ,in, Substitute the following formula:

[0260] (8)

[0261] For a vertical field of view angle VFOV, the current pitch angle For a camera, its horizontal front corresponds to the image OK. ,but This means that objects in front of the drone horizontally cannot appear in the camera image, which makes visual obstacle avoidance a big safety hazard. The faster the drone flies, The larger the value, the higher the VFOV value determines the upper speed limit of the drone's visual obstacle avoidance. On the other hand, only the upper half of the camera image is useful while the lower half is almost useless, which is a waste of image resources. Therefore, considering these situations, the following improvements can be made: Figure 9 As shown, when the camera is installed, its optical center path is at a certain angle to the horizontal direction. , The specific value should be set based on the relationship between speed and pitch angle, the flight speed limit, VFOV, and other conditions. This allows the lower half of the camera image to be used at slower flight speeds, and also increases the speed limit for monocular obstacle avoidance. Alternatively, a gimbal can be installed directly, so that the image captured is always directly in front of the drone horizontally.

[0262] (2) Obstacle avoidance path decision

[0263] Currently, commonly used path planning algorithms are divided into two categories: global static environment path decision making and local static environment path decision making. The former includes pose space methods and grid methods, while the latter includes fuzzy control methods and artificial potential field methods. These methods each have their own advantages and disadvantages, but the common disadvantage is that the calculation and modeling are relatively complex. Therefore, the present invention adopts a relatively simple reactive obstacle avoidance strategy for obstacle avoidance, which is as follows:

[0264] 1) Obstacle avoidance range is the original image To If the obstacle distribution range does not intersect with the obstacle avoidance range, it means that the height difference between the obstacle and the drone is too large and there is no need to avoid it. If there is an intersection, it means that the height is roughly the same and further judgment is needed on whether obstacle avoidance is needed (if the pitch angle value is less than 5°, then ; If the pitch angle is greater than 5°, calculate The value of ).

[0265] 2) Get the first Column to Column as a security window, where is the size of the safe window, The middle column of the image shows that if the distribution range of the obstacle intersects with the safety window range, obstacle avoidance is required. Otherwise, it means that the obstacle is far away from the drone's forward direction and no obstacle avoidance is required.

[0266] 3) The distribution of obstacles that need to be avoided can be divided into four categories, such as Figure 10 As shown in Figures (a) and (b), the left and right boundaries of the obstacle are , or , , then calculate the center of the safety window and the left and right boundaries of the obstacle 、 distance, avoid to the closer side; if it is the situation in Figure (c), that is, , or , , then avoid to the right. If it is the case of Figure (d), that is, , dodge to the left.

[0267] (3) Obstacle avoidance distance

[0268] After determining the flight direction, the flight distance must be calculated. The imaging model in front of the drone is as follows: Figure 11As shown, AB is the obstacle area, EF is the range of the obstacle in the imaging, CD is the range of the object corresponding to the imaging area, GH is the imaging of the area, which can be expressed as the number of columns of the image cols, L1 is the width of the obstacle, L2 is the width of the object corresponding to the imaging area, S1 is the distance between the camera and the obstacle, S2 is the distance between the camera and the object corresponding to EF, HFOV is the horizontal field of view of the camera, and f is the focal length of the camera.

[0269] According to the triangle similarity theorem and the camera imaging principle, we can get:

[0270] (9)

[0271] Combine the following formula:

[0272] (10)

[0273] We can get:

[0274] (11)

[0275] For example, if we use an image with a resolution of 640*480, then cols=640, and in formula (11) The distance S1 is the number of pixels (width) occupied by the obstacle area in the image. The value of the distance S1 can be determined based on the size ratio of the corresponding feature points of the object in the two images and the position spacing when the images were taken. S1 uses the safety distance by default. The value can be converted based on the physical width that allows the drone to pass safely. At low speed, the safety distance is 10 and the physical width is set to 4m. Substituting it into the value, we can get After avoiding, there should be a distance of The physical width of the pixel. If you dodge to the left, you need to fly Pixel corresponds to distance. If you dodge to the right, you need to fly. The physical width in pixels.

[0276] The embodiments of the present invention calculate the scale ratio of matching feature point pairs and, in combination with the triangle similarity theorem, generate a depth map, effectively estimating the distance to obstacles. Monocular vision can accurately identify and locate the position and distance of obstacles without relying on depth information provided by binocular vision or lidar. This image-feature-based depth estimation method reduces system complexity and improves adaptability to various lighting conditions. By grouping feature points with scale ratios greater than a set standard and statistically analyzing their longitudinal and lateral distribution in the image, the specific distribution range of the current obstacle can be accurately determined. Based on this distribution information, the present invention can intelligently determine whether the obstacle is close to the drone's flight altitude. If obstacle avoidance is required, a safety window area is further defined, and an obstacle avoidance path to the left or right is intelligently selected based on the obstacle's left and right position in the image. This refined path planning strategy ensures that the drone can flexibly and safely avoid various obstacles in complex environments.

[0277] See also Figure 12 , Figure 12 This is a structural block diagram of an obstacle avoidance processing system based on drone inspection provided in Example 3 of the present invention.

[0278] The present invention provides an obstacle avoidance processing system based on drone inspection, comprising:

[0279] The data acquisition module 301 is used to acquire visual images through a monocular camera carried by the drone;

[0280] The screening and matching module 302 is used to pre-process two adjacent frames of visual images and determine matching feature point pairs based on the pre-processing results;

[0281] The obstacle distribution range processing module 303 is used to calculate the scale ratio of the matching feature point pairs, and perform vertical and horizontal distribution statistics on the matching feature point pairs in combination with the preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacles;

[0282] A determination module 304 is configured to determine whether the height of the obstacle is within a preset flight altitude range based on the distribution range;

[0283] The output obstacle avoidance module 305 is used to plan an obstacle avoidance path based on the preset safety window area and distribution range when the height of the obstacle is within the preset flight altitude range, and perform obstacle avoidance flight control on the UAV.

[0284] Furthermore, the screening and matching module 302 includes:

[0285] A preprocessing submodule is used to preprocess two adjacent frames of visual images and determine the feature points corresponding to the visual images;

[0286] The matching point pair submodule is used to perform brute force matching on the feature points corresponding to two adjacent frames of visual images to obtain matching point pairs;

[0287] A first rejection threshold submodule, configured to determine a first rejection threshold based on calculating an average value of the Euclidean distances of all matching point pairs;

[0288] A rejection submodule is used to reject matching point pairs when the Euclidean distance value of the matching point pair is greater than a first rejection threshold;

[0289] A retaining submodule, configured to retain the matching point pairs when the Euclidean distance value of the matching point pairs is less than or equal to a first rejection threshold, thereby obtaining a first screening point pair group;

[0290] The screening submodule is used to screen the first screening point pair group to obtain matching feature point pairs.

[0291] Furthermore, the screening submodule includes:

[0292] a group scale ratio unit, configured to calculate the group scale ratio of the matching point pairs within the first screening point pair group;

[0293] A first processing unit, configured to remove matching point pairs from the first screening point pair group when the group scale ratio satisfies a preset removal condition;

[0294] The second processing unit is configured to use the matching point pair as a matching feature point pair when the group scale ratio does not meet a preset elimination condition.

[0295] Furthermore, the preset elimination condition is specifically that the group size ratio is less than the second elimination threshold, or the group size ratio is greater than the third elimination threshold.

[0296] Furthermore, the obstacle distribution range processing module 303 includes:

[0297] A division submodule is used to divide the matching feature point pairs into a preset dangerous feature point pair group when the scale ratio is greater than or equal to the preset standard obstacle avoidance scale ratio;

[0298] The statistics submodule is used to count the vertical and horizontal distribution of the feature points corresponding to the obstacles when the number of matching feature point pairs in the preset dangerous feature point pair group is greater than the preset number value;

[0299] The distribution range submodule is used to determine the distribution range of the obstacle based on the vertical and horizontal distribution, wherein the distribution range includes the left and right boundaries and the upper and lower boundaries of the obstacle in the visual image.

[0300] Furthermore, the statistics submodule includes:

[0301] A region unit, configured to divide the visual image into a plurality of longitudinal regions and transverse regions, wherein each longitudinal region and each transverse region is provided with a corresponding distribution array;

[0302] A first element frequency accumulation unit is configured to perform element frequency accumulation on a longitudinal distribution array corresponding to a longitudinal region containing the longitudinal coordinates of the feature points in the preset dangerous feature point pair group in the visual image;

[0303] A longitudinal distribution unit is used to traverse a preset dangerous feature point pair group to obtain a longitudinal distribution;

[0304] A second element frequency accumulation unit is used to perform element frequency accumulation on a horizontal distribution array corresponding to a horizontal area containing the horizontal coordinate according to the horizontal coordinate of the feature point in the visual image;

[0305] The horizontal distribution unit is used to traverse the preset dangerous feature point pairs to obtain the horizontal distribution.

[0306] Furthermore, the distribution range submodule includes:

[0307] The first element unit is used to traverse the vertical distribution array and take the first element in the vertical distribution array whose value is greater than a preset threshold as the first element;

[0308] The second element unit is used to take the element whose last element value in the vertical distribution array is greater than a preset threshold as the second element;

[0309] A left limit unit, configured to use the first column of the longitudinal area corresponding to the first element as the left limit of the obstacle in the visual image;

[0310] a right limit unit, used to use the last column of the longitudinal area corresponding to the second element as the right limit of the obstacle in the visual image;

[0311] The third element unit is used to traverse the horizontal distribution array and take the first element in the horizontal distribution array whose value is greater than a preset threshold as the third element;

[0312] A fourth element unit is used to set the element whose last element value in the horizontal distribution array is greater than a preset threshold as the fourth element;

[0313] an upper limit unit, configured to use the first row of the horizontal area corresponding to the third element as the upper limit of the obstacle in the visual image;

[0314] The lower limit unit is used to use the last row of the horizontal area corresponding to the fourth element as the lower limit of the obstacle in the visual image.

[0315] Furthermore, the output obstacle avoidance module 305 includes:

[0316] The distance value submodule is used to determine the left and right distance values ​​of the drone to the left and right boundaries according to the distribution range when the distribution range intersects with the preset safety window area;

[0317] The first flight direction determination submodule is used to compare the left distance value with the right distance value, and select the side with the smaller distance value as the flight direction according to the comparison result;

[0318] The second flight direction determination submodule is configured to use the other side of the visual image as the flight direction when the obstacle is located on one side of the visual image.

[0319] Furthermore, it also includes:

[0320] The flight safety obstacle avoidance distance module is used to determine the actual obstacle avoidance physical width as the drone's flight safety obstacle avoidance distance based on the triangle similarity theorem and camera imaging principle.

[0321] In summary, the obstacle avoidance method for drone inspections presented here, through optimization of the monocular vision system and an improved ORB algorithm, provides an efficient, low-cost, precise, and intelligent obstacle avoidance solution. This method not only significantly reduces the hardware cost and computational complexity of the drone obstacle avoidance system, but also improves the real-time performance and accuracy of obstacle avoidance, ensuring safe flight and mission execution in complex environments. Its excellent environmental adaptability and system scalability give it broad application prospects and market value in a variety of drone application scenarios, including inspection, surveying and mapping, and logistics.

[0322] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0323] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interface, device or unit, which can be electrical, mechanical or other forms.

[0324] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0325] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0326] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0327] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An obstacle avoidance method based on drone inspection, characterized in that: include: Obtain visual images through the monocular camera carried by the drone; Preprocessing the two adjacent frames of the visual image, and determining matching feature point pairs based on the preprocessing results; Calculating the scale ratio of the matching feature point pairs, and performing vertical and horizontal distribution statistics on the matching feature point pairs in combination with a preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacle; Based on the distribution range, determining whether the height of the obstacle is within a preset flight altitude range; When the height of the obstacle is within the preset flight altitude range, an obstacle avoidance path is planned based on the preset safety window area and the distribution range, and obstacle avoidance flight control is performed on the UAV.

2. The obstacle avoidance method based on drone inspection according to claim 1 is characterized in that: The preprocessing of the two adjacent frames of visual images and determining matching feature point pairs based on the preprocessing results includes: Preprocessing two adjacent frames of the visual image to determine feature points corresponding to the visual images; Performing brute force matching on the feature points corresponding to the visual images of two adjacent frames to obtain matching point pairs; Determining a first rejection threshold based on calculating an average value of the Euclidean distances of all the matching point pairs; When the Euclidean distance value of the matching point pair is greater than the first elimination threshold, the matching point pair is eliminated; When the Euclidean distance value of the matching point pair is less than or equal to the first rejection threshold, the matching point pair is retained to obtain a first screening point pair group; The first screening point pair group is screened to obtain matching feature point pairs.

3. The obstacle avoidance method based on drone inspection according to claim 2 is characterized in that: The step of screening the first screening point pair group to obtain matching feature point pairs includes: Calculating the group scale ratio of the matching point pairs within the first screening point pair group; When the group scale ratio meets a preset elimination condition, the matching point pair is eliminated from the first screening point pair group; When the group scale ratio does not meet the preset elimination condition, the matching point pair is used as a matching feature point pair.

4. The obstacle avoidance method based on drone inspection according to claim 3 is characterized in that: The preset elimination condition is specifically that the group size ratio is less than a second elimination threshold, or the group size ratio is greater than a third elimination threshold.

5. The obstacle avoidance method based on drone inspection according to claim 1 is characterized in that: The performing of vertical and horizontal distribution statistics on the matched feature point pairs in combination with a preset standard obstacle avoidance scale to determine the distribution range of the obstacles includes: When the scale ratio is greater than or equal to the preset standard obstacle avoidance scale ratio, the matching feature point pair is divided into a preset dangerous feature point pair group; When the number of matching feature point pairs in the preset dangerous feature point pair group is greater than a preset value, the vertical and horizontal distribution of the feature points corresponding to the obstacles are counted; The distribution range of the obstacle is determined according to the vertical and horizontal distribution, wherein the distribution range includes the left and right boundaries and the upper and lower boundaries of the obstacle in the visual image.

6. The obstacle avoidance method based on drone inspection according to claim 5 is characterized in that: The vertical and horizontal distribution of the characteristic points corresponding to the statistical obstacles includes: Dividing the visual image into a plurality of longitudinal regions and transverse regions, wherein each longitudinal region and each transverse region is provided with a corresponding distribution array; According to the longitudinal coordinates of the feature points in the preset dangerous feature point pair group in the visual image, performing element frequency accumulation on the longitudinal distribution array corresponding to the longitudinal area containing the longitudinal coordinates; Traversing the preset dangerous feature point pairs to obtain a longitudinal distribution; According to the horizontal coordinate of the feature point in the visual image, performing element frequency accumulation on the horizontal distribution array corresponding to the horizontal area containing the horizontal coordinate; The preset dangerous feature point pairs are traversed to obtain the horizontal distribution.

7. The obstacle avoidance method based on drone inspection according to claim 6 is characterized in that: Determining the distribution range of the obstacles according to the longitudinal and transverse distribution includes: Traversing the longitudinal distribution array, taking the first element in the longitudinal distribution array whose value is greater than a preset threshold as the first element; The element whose last element value in the longitudinal distribution array is greater than the preset threshold is used as the second element; Using the first column of the longitudinal area corresponding to the first element as the left limit of the obstacle in the visual image; Using the last column of the longitudinal area corresponding to the second element as the right limit of the obstacle in the visual image; Traversing the horizontal distribution array, taking the first element in the horizontal distribution array whose value is greater than the preset threshold as the third element; The element whose last element value in the horizontal distribution array is greater than the preset threshold is used as the fourth element; using the first row of the lateral area corresponding to the third element as the upper limit of the obstacle in the visual image; The fourth element corresponds to the last row of the horizontal area as the lower limit of the obstacle in the visual image.

8. The obstacle avoidance method based on drone inspection according to claim 7, characterized in that: The planning of the obstacle avoidance path based on the preset safety window area and the distribution range includes: When the distribution range intersects with the preset safety window area, the left distance value and the right distance value of the drone to the left and right boundaries are determined according to the distribution range; Comparing the left distance value with the right distance value, and selecting the side with the smaller distance value as the flight direction according to the comparison result; When the obstacle is located on one side of the visual image, the other side of the visual image is used as the flight direction.

9. The obstacle avoidance method based on drone inspection according to any one of claims 1 to 8, characterized in that: Also includes: Based on the triangle similarity theorem and the camera imaging principle, the actual obstacle avoidance physical width is determined as the safe obstacle avoidance distance of the UAV.

10. An obstacle avoidance processing system based on drone inspection, characterized in that: include: A data acquisition module is used to acquire visual images through a monocular camera carried by the drone; A screening and matching module is used to preprocess the two adjacent frames of the visual image and determine matching feature point pairs based on the preprocessing results; an obstacle distribution range processing module, configured to calculate the scale ratio of the matching feature point pairs, and perform vertical and horizontal distribution statistics on the matching feature point pairs in combination with a preset standard obstacle avoidance scale ratio to determine the distribution range of the obstacles; A judgment module, configured to judge whether the height of the obstacle is within a preset flight altitude range based on the distribution range; The output obstacle avoidance module is used to plan an obstacle avoidance path based on a preset safety window area and the distribution range when the height of the obstacle is within the preset flight altitude range, and to perform obstacle avoidance flight control on the UAV.

Images