Unmanned aerial vehicle autonomous inspection method and device based on precise re-shooting and medium

Abstract

The invention discloses an unmanned aerial vehicle autonomous inspection method and device based on precise re-shooting, and a medium, and the method comprises the steps: constructing a point cloud model, carrying out the optimization learning of the point cloud model, constructing a nonlinear scale space pyramid, carrying out the feature extraction of detected device points of different scales, and carrying out the matching learning based on description vectors. According to the method, feature points of different scales in an image can be captured, the expression ability of the features is enhanced, under the condition that accurate detected equipment points are obtained, the cruise line of the minimum enclosing rectangle of the detected equipment points is extracted according to the point set contour line, all key areas are effectively covered, omission is reduced, the comprehensiveness of target detection is improved, and the detection efficiency is improved. By performing defect detection on the preprocessed image, the image quality is improved, so that the defect detection algorithm can more accurately identify and classify defects, thereby improving the accuracy of target identification, effectively improving the route accuracy of the unmanned aerial vehicle in the inspection task and the target identification accuracy, and improving the inspection efficiency and reliability.

Description

Technical Field

[0001] The present invention relates to the field of intelligent inspection technology, and in particular to an autonomous inspection method, equipment and medium for unmanned aerial vehicles based on precise re-shooting. Background Art

[0002] In recent years, multi-rotor drone autonomous inspection technology has been widely used in power transmission, especially in the detailed photography of 35kV~220kV poles and towers. This technology has greatly improved the perception ability of the pole tower hardware, and gradually replaced the traditional power outage inspection work.

[0003] Although the existing drone autonomous inspection methods have improved the efficiency and quality of inspections, there are still some defects and challenges in practical applications:

[0004] The existing accurate re-shooting methods often require pilots to control the UAV to simulate the operation trajectory during the route learning stage, and record the position point status during the flight. This operation is easy to forget, resulting in large deviations in the acquired trajectory. During the autonomous inspection process of the UAV, due to its positioning accuracy, it is difficult to meet the needs of some fine inspections. During the inspection process, since the UAV needs to acquire images of the equipment under test, based on the environmental complexity of the entire inspection area, it is difficult for the UAV to accurately identify the target object, or the image acquisition of the target object is not comprehensive, resulting in reduced inspection efficiency. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the route deviation is large and the target recognition is inaccurate. The purpose is to provide a method, device and medium for autonomous inspection of unmanned aerial vehicles based on precise re-shooting. By constructing a point cloud model, optimizing and learning the point cloud model, constructing a nonlinear scale space pyramid to extract features of inspected equipment points of different scales, and based on description vector matching learning, it is possible to capture feature points of different scales in the image, enhance the expression ability of features, and help improve the accuracy of target recognition. When accurate equipment points are obtained, the cruise route of the minimum circumscribed rectangle of the inspected equipment points is extracted according to the contour line of the point set, effectively covering all key areas, reducing omissions, and improving the comprehensiveness of target detection. By performing defect detection on the preprocessed image, the image quality is improved, so that the defect detection algorithm can more accurately identify and classify defects, thereby improving the accuracy of target recognition, and can effectively improve the route accuracy and target recognition accuracy of the unmanned aerial vehicle in the inspection task, thereby improving the efficiency and reliability of the inspection.

[0006] The present invention is achieved through the following technical solutions:

[0007] The first aspect of the present invention provides a method for autonomous inspection of a drone based on precise re-shooting, comprising the following specific steps:

[0008] Obtain the point distribution data of the inspected equipment in the target detection area, and load the point cloud model of the inspected equipment point distribution in the target detection area based on the point cloud layering;

[0009] Construct a nonlinear scale space pyramid to extract features and perform scale space analysis on the inspected equipment points of different scales;

[0010] Performing main direction analysis on the extracted features to obtain feature points, defining a local area with the feature point as the center, using the Sobel operator to calculate the gradient of the image in the local area, weighting the gradient size in each direction interval according to the gradient size, obtaining a direction histogram, and selecting the direction with the largest gradient size in the direction histogram as the main direction of the feature point;

[0011] Based on the main direction of the feature points, determine the N-dimensional description vector of the first image, and determine the optimized point cloud model of the point distribution of the inspected equipment;

[0012] Based on the optimized point cloud model of the inspected equipment point distribution, determine and output the point set contour line of the inspected equipment points;

[0013] Plan the cruise route and image shooting points according to the point set contour line, and acquire images;

[0014] The acquired image is preprocessed, and defect detection is performed on the preprocessed image to obtain a detection result.

[0015] Furthermore, the point cloud model of the point distribution of the equipment under inspection in the target inspection area is loaded based on the point cloud layering, including:

[0016] Preprocess the original point cloud data to obtain point cloud data;

[0017] Performing hierarchical segmentation on the point cloud data, and visualizing the point cloud data according to the hierarchical segmentation to obtain a first image;

[0018] Point cloud modeling is performed based on the first image to generate a point cloud model of the point distribution of the inspected device.

[0019] Furthermore, the construction of a nonlinear scale space pyramid to perform feature extraction and scale space analysis on the inspected device points of different scales includes:

[0020] Performing grayscale processing on the acquired first image;

[0021] Construct a nonlinear diffusion function to model the image and generate a nonlinear scale space pyramid;

[0022] The nonlinear scale space pyramid includes multiple scale levels, each level corresponds to a scale, and is used to detect features of different scales of the image;

[0023] The first image feature points at multiple scales are obtained based on a nonlinear scale space pyramid.

[0024] Further, based on the main direction of the feature point, an N-dimensional description vector of the first image is determined, including:

[0025] Based on the main direction of the feature point, a window centered on the feature point is defined in the gradient image;

[0026] Subdivide the window into m sub-regions, each sub-region has a size of n×n;

[0027] The first sub-region is weighted using the first Gaussian kernel to obtain a sub-region description vector with a length of n1;

[0028] The second sub-region is weighted using the second Gaussian kernel to obtain a sub-region description vector with a length of n2;

[0029] The two sub-region description vectors are iterated and normalized in turn to obtain a description vector of n×n1×n2=N.

[0030] Furthermore, the cruise route extraction of the minimum circumscribed rectangle of the detected equipment points based on the optimized point cloud model of the detected equipment points includes:

[0031] The adjacency relationship of the triangulated mesh of the point set is obtained, and the outer contour of the point set of the detected equipment line in the target detection area is determined according to the adjacency relationship.

[0032] Furthermore, the step of determining the outer contour of the point set of the detected equipment line in the target detection area according to the adjacency relationship includes:

[0033] Traverse the triangulation network. If the number of adjacent triangles of a triangle is less than K, traverse each edge of the triangle:

[0034] If the current edge E1 only belongs to the current triangle, mark the current edge E1 as the contour boundary;

[0035] If the current edge E2 is a candidate contour boundary, obtain the starting point A and the end point B of the current edge E2, then add point A to the point chain, and search for other triangle edges starting from point A;

[0036] If the current edge E3 is a candidate contour boundary, obtain the starting point A and the end point C of the current edge E3, then add point C to the point chain, and use point C as the starting point to search for other triangle edges;

[0037] If the current edge E2 or E3 is a non-contour boundary, search other triangle edges.

[0038] Repeat the traversal of the triangulated network until the search endpoint returns to the starting point B, and output the point set contour line of the detected device point.

[0039] Furthermore, the cruise line extraction of the minimum circumscribed rectangle of the detected equipment point is performed according to the point set contour line, including:

[0040] The coordinates of the center point of the contour are obtained as (x0, y0), the coordinates of the contour point P1 are (x1, y1), the coordinates of P2 are (x2, y2), and the angle between the line segment P1P2 and the positive direction of the X-axis is θ;

[0041] Rotate the original contour by θ around the contour center point to obtain the new coordinates of the contour point (x1', y1');

[0042] Get the coordinate range of each contour point after rotation and get the circumscribed rectangle;

[0043] Rotate the circumscribed rectangle in the opposite direction by θ around the contour center to obtain the minimum area circumscribed rectangle of the contour point set;

[0044] Through the minimum area circumscribed moment, the direction and size of the object are determined and the cruise route is constructed.

[0045] Furthermore, the preprocessing of the acquired image includes:

[0046] Screening the image data to remove the image data whose pixel value is less than a set threshold;

[0047] Perform frame extraction processing on the filtered image data;

[0048] Extracting the timestamp of the frame-decoded image includes: obtaining the image scale feature and constructing the scale space; discretizing the scale space and obtaining the scale factor of each sub-layer according to the discretized scale space; based on the scale factor, converting the pixels of the scale factor into time to obtain the timestamp of the image;

[0049] The ambient brightness at the corresponding timestamp is obtained according to the timestamp of the image, and the image data is subjected to light compensation processing.

[0050] A second aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, an autonomous inspection method for unmanned aerial vehicles based on precise re-shooting is implemented.

[0051] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an autonomous inspection method for unmanned aerial vehicles based on precise re-shooting.

[0052] Compared with the prior art, the present invention has the following advantages and beneficial effects: by constructing a point cloud model, optimizing and learning the point cloud model, constructing a nonlinear scale space pyramid to extract features of the inspected equipment points of different scales, based on description vector matching learning, it is possible to capture feature points of different scales in the image, enhance the feature expression ability, and help improve the accuracy of target recognition; when accurate equipment points are obtained, the cruise route of the minimum circumscribed rectangle of the inspected equipment points is extracted according to the contour line of the point set, effectively covering all key areas, reducing omissions, and improving the comprehensiveness of target detection; by performing defect detection on the preprocessed image, the image quality is improved, so that the defect detection algorithm can more accurately identify and classify defects, thereby improving the accuracy of target recognition, and can effectively improve the route accuracy and target recognition accuracy of the UAV in the inspection task, thereby improving the efficiency and reliability of the inspection. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without creative work. In the drawings:

[0054] Figure 1 is a flow chart of the inspection method in an embodiment of the present invention;

[0055] Figure 2 A comparative data diagram of waypoint detection between the traditional shooting method in an embodiment of the present invention and the present solution;

[0056] Figure 3 4 is a comparison chart of imaging quality in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0058] As a possible implementation, Figure 1As shown, the present embodiment provides an autonomous inspection method of a drone based on precise re-shooting, comprising the following specific steps: obtaining the distribution data of the inspected equipment points in the target detection area, and loading the inspected equipment point distribution point cloud model in the target detection area based on the point cloud layer; constructing a nonlinear scale space pyramid to extract features and perform scale space analysis on the inspected equipment points of different scales; performing main direction analysis on the extracted features, obtaining feature points, defining a local area, the local area is centered on the feature point, using the sobel operator to calculate the gradient of the image in the local area, weightedly counting the gradient size in each direction interval according to the size of the gradient, obtaining a direction histogram, and selecting the direction with the largest gradient size in the direction histogram as the main direction of the feature point; determining the N-dimensional description vector of the first image based on the main direction of the feature point, and determining the optimized inspected equipment point distribution point cloud model; determining the point set contour line of the output inspected equipment point based on the optimized inspected equipment point distribution point cloud model; planning the cruise route and image shooting points according to the point set contour line, and acquiring the image; preprocessing the acquired image, performing defect detection on the preprocessed image, and obtaining the detection result. This embodiment obtains the distribution data of the inspected equipment points in the target detection area, and loads the point cloud model of the inspected equipment points in the target detection area based on the point cloud layer, so as to ensure that the drone can fly according to the predetermined path during the inspection process, thereby improving the accuracy of the route and reducing the route deviation caused by environmental changes or inaccurate map information. By constructing a nonlinear scale space pyramid, the features of the inspected equipment points of different scales are extracted. The method can capture the feature points of different scales in the image, enhance the expression ability of the features, and help improve the accuracy of target recognition. Then, the main direction analysis of the extracted features is performed to help locate the precise position of the feature points in the image, and further enhance the accuracy of the feature description. Feature matching learning is performed based on the description vector to obtain an optimized point cloud model of the inspected equipment points. By learning to optimize feature matching, the accuracy of feature matching can be improved. The cruise route extraction of the minimum circumscribed rectangle of the inspected equipment points is performed according to the contour line of the point set, which can ensure that the drone can effectively cover all key areas during the inspection, reduce omissions, and improve the comprehensiveness of target detection. The acquired image is preprocessed, and the preprocessed image is subjected to defect detection to obtain the detection result. Image preprocessing can improve image quality, allowing defect detection algorithms to more accurately identify and classify defects, and can effectively improve the accuracy of drone routes and target recognition in inspection tasks, thereby improving inspection efficiency and reliability.

[0059] In some possible embodiments, before point cloud modeling, the original point cloud data needs to be preprocessed, including denoising, filtering, coordinate transformation, etc., so as to eliminate the error of the airborne point cloud system and improve the overall accuracy of the data. Point cloud stratification refers to dividing the point cloud data into multiple levels according to certain rules or standards for easy management and loading. This can be achieved by setting different thresholds or using specific algorithms. After point cloud data preprocessing and stratification, point cloud modeling can be performed.

[0060] In some possible embodiments, the step of constructing a nonlinear scale space pyramid to perform feature extraction and scale space analysis on detected device points of different scales includes:

[0061] First, you need to get the original image data. Because color information may increase the complexity of processing, the image is usually converted to a grayscale image first, which can reduce the amount of calculation and focus on the texture and shape characteristics of the image;

[0062] Construct a nonlinear diffusion function to model the image and generate a nonlinear scale space pyramid. The nonlinear diffusion function can adaptively adjust the diffusion process according to the image content, which helps to retain the edge information of the image while denoising. This diffusion process can be implemented by a nonlinear partial differential equation, in which the value of the conduction function is determined by the differential structure, so that the diffusion can be adaptively filtered according to the image structure. When performing nonlinear diffusion processing on an image, it can be implemented through an iterative process, in which each iteration updates the image pixel value according to the gradient information of the current image;

[0063] In the nonlinear scale space pyramid, there are multiple scale levels, each level corresponds to a scale, which is used to detect the features of different scales of the image. The Fast Explicit Diffusion (FED) method can be used to accelerate the construction of the nonlinear scale space. This method can reduce the amount of calculation while maintaining performance, making the algorithm more suitable for real-time applications;

[0064] At each level of the nonlinear scale space pyramid, feature points can be detected by calculating the determinant of the Hessian matrix of the image to find the feature points. These feature points correspond to corners, edges or other significant texture features in the image. In order to improve the reproducibility and robustness of the feature points, a main direction can be assigned to each feature point, which is usually achieved by calculating the gradient direction of the pixels in the neighborhood of the feature point. In this way, the consistency of the feature points can be maintained even when the image is rotated or the scale changes.

[0065] In some possible embodiments, performing main direction analysis on the extracted features includes:

[0066] Based on the feature point, a local area is defined, and the local area is centered on the feature point;

[0067] The Sobel operator is used to calculate the gradient of the image in the local area to obtain the direction and magnitude of the gradient;

[0068] According to the size of the gradient, the gradient size in each direction interval is weighted and counted to obtain the direction histogram;

[0069] The direction with the largest gradient in the direction histogram is selected as the main direction of the feature point.

[0070] The main direction of the feature point is calculated to give the feature point rotation invariance. By assigning a main direction to the feature point, it is ensured that the feature descriptor remains consistent under rotation transformation. This means that even if the image is rotated, the descriptor of the feature point can still match, so that the feature point can be correctly identified and matched. In the feature point matching process, it is necessary to compare the feature points in different images. If the feature point is rotation invariant, then the feature points on the same object surface can be matched even in images taken from different perspectives. The main direction of the feature point provides additional information about the structure around the feature point. This information can be used to improve the accuracy of feature matching because it can help distinguish feature points that are close in position and scale but have different directions. Including direction information in the feature point descriptor can increase the discrimination of the descriptor, making the feature points easier to distinguish, thereby improving the accuracy of feature matching.

[0071] In some possible embodiments, determining an N-dimensional description vector of the first image based on the main direction of the feature point includes:

[0072] First, a window centered on the feature point is defined in the gradient image based on the main direction of the feature point. This window is usually a square or circular area that includes the pixels around the feature point.

[0073] The window is subdivided into m sub-regions, each of which is n × n in size. Subdividing the window helps capture the local structural information around the feature points.

[0074] Two Gaussian kernels of different scales are used to weight the sub-regions. The first Gaussian kernel is used to weight the first sub-region, and a sub-region description vector of length n1 is obtained. The second Gaussian kernel is used to weight the second sub-region, and a sub-region description vector of length n2 is obtained. The scale of the Gaussian kernel determines the sensitivity of the description vector to scale changes.

[0075] The two sub-region description vectors are iterated and normalized in turn. This usually involves dividing each element in the description vector by the sum or Euclidean norm of the vector to ensure that the description vector is consistent across different scales and directions.

[0076] The description vectors of all sub-regions are combined to obtain the final N-dimensional description vector, where n×n1×n2=N. This vector contains detailed information about the local structure around the feature point and is robust to rotation, scale changes, and illumination changes. It can capture the local structural information of feature points in the image, and through Gaussian weighting and normalization, the description vector is insensitive to scale and orientation changes of the image, thereby improving the accuracy of feature matching.

[0077] In some possible embodiments, the cruise route extraction of the minimum circumscribed rectangle of the detected equipment point is performed based on the optimized point cloud model of the detected equipment point position distribution, including: obtaining the adjacency relationship of the triangulated mesh of the point set, and determining the point set outer contour of the detected equipment line in the target detection area according to the adjacency relationship, including: traversing the triangulated mesh, if the number of adjacent triangles of a triangle is less than K, then traversing each edge of the triangle: if the current edge E1 only belongs to the current triangle, then marking the current edge E1 as a contour boundary; if the current edge E2 is a candidate contour boundary, obtaining the starting point A and the end point B of the current edge E2, then adding point A to the point chain, and starting from point A, searching for other triangle edges; if the current edge E3 is a candidate contour boundary, obtaining the starting point A and the end point C of the current edge E3, then adding point C to the point chain, and starting from point C, searching for other triangle edges; if the current edge E2 or E3 is a non-contour boundary, then searching for other triangle edges. Repeat the traversal of the triangulated mesh until the search endpoint returns to the starting point B, and output the point set contour line of the detected equipment point.

[0078] The cruise route extraction of the minimum circumscribed rectangle of the detected equipment point is performed according to the contour line of the point set, including: obtaining the coordinates of the contour center point as (x0, y0), the coordinates of the contour point P1 as (x1, y1), the coordinates of P2 as (x2, y2), and the angle between the line segment P1P2 and the positive direction of the X-axis as θ; rotating the original contour around the contour center point by θ to obtain the new coordinates of the contour point as (x1', y1'); obtaining the range of the coordinates of each contour point after rotation to obtain the circumscribed rectangle; rotating the circumscribed rectangle around the contour center in the opposite direction by θ to obtain the minimum area circumscribed rectangle of the contour point set; determining the direction and size of the object through the minimum area circumscribed moment, and constructing the cruise route.

[0079] Through the adjacency relationship of triangulated meshes, the characteristic lines of the inspected equipment in the target detection area can be more accurately identified and extracted. By tracking the characteristic points and determining the main direction, a complete description vector can be generated to ensure the continuity and integrity of the characteristic contour. Through the adjacency relationship of triangulated meshes, the target detection area can be quickly located, reducing unnecessary calculations and searches, thereby improving the detection efficiency. After determining the outer contour of the point set of the inspected equipment points, a more optimized cruise route can be designed accordingly. Such a route can ensure that the drone or inspection equipment can effectively cover all key areas while reducing omissions and repeated detections. Image acquisition based on the cruise route of the minimum circumscribed rectangle can ensure that the acquired image accurately covers the target detection area and provide high-quality data for subsequent image processing and defect detection. Defect detection can be performed after preprocessing the acquired image to improve the accuracy of defect detection. Accurate point set outer contours help improve the effect of image processing algorithms, thereby improving the accuracy of defect detection.

[0080] In some possible embodiments, the acquired image is preprocessed, including: screening the image data, filtering out the image data with pixel values ​​less than a set threshold, and setting a threshold to exclude the image data with pixel values ​​lower than the threshold, so as to remove noise or unimportant image information, thereby improving the efficiency and accuracy of subsequent processing; performing frame extraction on the screened image data, and selectively retaining key frames to reduce the amount of data; extracting the timestamp of the frame-extracted image, including: obtaining image scale features and constructing a scale space; discretizing the scale space, and obtaining the scale factor of each sub-layer according to the discretized scale space, each sub-layer represents the approximation of the image at different scales, which is helpful for multi-scale analysis; based on the scale factor, converting the pixels of the scale factor into time to obtain the timestamp of the image; obtaining the ambient brightness at the corresponding timestamp according to the timestamp of the image, and performing light compensation processing on the image data. By obtaining the ambient brightness at the corresponding timestamp according to the timestamp of the image, the lighting conditions when the image was taken can be understood. Performing light compensation processing on the image data can adjust the brightness and contrast of the image, so that the image can maintain a good visual effect under different lighting conditions, improve the image quality, and facilitate subsequent image analysis and understanding.

[0081] As a possible implementation method, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, an autonomous inspection method for drones based on precise re-shooting is implemented.

[0082] As a possible implementation, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, an autonomous inspection method for drones based on precise re-shooting is implemented.

[0083] As a possible implementation method, this embodiment provides a practical application of a drone autonomous inspection method based on precise re-shooting:

[0084] The power transmission operation and inspection center of our company has carried out precise re-flying practice on four types of towers, including 220kV single-circuit straight tower, 220kV single-circuit tension tower, 220kV double-circuit straight tower, and 220kV double-circuit tension tower. A total of 1,275 towers have been flown, verifying the applicability of precise re-shooting autonomous inspection technology, solidifying the re-shooting route, and completing the optimization and efficiency improvement of autonomous inspection work. Inspection tests were carried out in accordance with the inspection shooting requirements of the "Guide to the Shooting of Overhead Transmission Line UAV Inspection Images". The test targets included different types of towers with a voltage level of 220kV, including multi-point autonomous inspections of single-circuit straight towers, single-circuit tension towers, double-circuit straight towers, and double-circuit tension towers. The test was carried out in accordance with the specifications of the "Guide to the Shooting of Overhead Transmission Line UAV Inspection Images". DJI Phantom 4 was used to shoot in a close-up manner, and DJI Latitude M30T was used to manually record the route through precise re-shooting. The efficiency and time difference were evaluated through route re-flying. Based on high requirements, the following test results were obtained.

[0085] (1) Route comparison

[0086] After multiple field tests, the number of waypoints required for accurate re-shooting was reduced by flying patrols on different towers. Accurate waypoint selection was performed. The number of waypoints required for accurate re-shooting was at least 20.5% of the waypoints required for traditional automatic inspections, and at most 25.5%. For more complex towers, the number of waypoints was reduced by about 47 waypoints on the highest double-circuit tension tower. The test results are as follows Figure 2 As shown. After collecting the route, we conducted a go-around for different types of towers. By measuring and comparing the time required for the route go-around, we found that the inspection time of the two types of aircraft was not much different in the straight tower inspection, but there was a significant difference in the tension tower inspection. In particular, in the double-return tension tower inspection, the precise re-shooting took 6 minutes and 31 seconds, while the traditional close-up shooting method took 13 minutes and 15 seconds, and the inspection efficiency was increased by more than 50%.

[0087] (2) Imaging comparison

[0088] According to the technical requirements of the "Guide to Overhead Transmission Line UAV Inspection Image Shooting", the image shooting should ensure that the camera parameters are set reasonably and the focus is accurate. At the same time, the target device should be located in the middle of the image, and the pin-type targets and defects should be clearly visible when magnified. Take the pictures of a double-circuit straight tower taken in different ways on different dates and time periods as an example. The image quality is as follows Figure 3 shown. Figure 3The middle left column is the traditional shooting, and the right column is the precise re-shooting under the method adopted in this embodiment. It can be seen that the traditional autonomous inspection technology has problems such as shooting target offset and defocusing, and the inability to reasonably expose when shooting in a backlit environment. The use of precise re-shooting technology can ensure that the quality of each picture meets the standard, and ensure that the collected photos are taken accurately and clearly, thereby achieving quality improvement. After full-text data analysis, the use of precise re-shooting technology optimizes autonomous routes through long-distance optical zoom shooting, reduces nearly 80% of waypoints, improves inspection efficiency by more than 50%, and saves manpower and material resources. At the same time, the precise re-shooting technology combined with the drone AI algorithm can quickly find shooting targets, search for matching targets of interest from the wide-angle picture, correct the deviation of the drone position and the camera shooting angle, ensure that the shooting object is in the middle every time, effectively guarantee the accuracy of autonomous patrol zoom photography, and achieve all hanging point bolts, nuts, and pins of the tower without dead angles. Shooting improves the quality of picture shooting. Drones equipped with infrared dual-light equipment can use precise flight routes to simultaneously start visible light detection and infrared detection, combining the two tasks to further promote the expansion of the business of refined autonomous inspections of transmission line towers.

[0089] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A drone autonomous inspection method based on precise re-shooting, characterized in that: The specific steps include: Obtain the point distribution data of the inspected equipment in the target detection area, and load the point cloud model of the inspected equipment point distribution in the target detection area based on the point cloud layering; Construct a nonlinear scale space pyramid to extract features and perform scale space analysis on the inspected equipment points of different scales; Performing main direction analysis on the extracted features to obtain feature points, defining a local area with the feature point as the center, using the Sobel operator to calculate the gradient of the image in the local area, weighting the gradient size in each direction interval according to the gradient size, obtaining a direction histogram, and selecting the direction with the largest gradient size in the direction histogram as the main direction of the feature point; Based on the main direction of the feature points, determine the N-dimensional description vector of the first image, and determine the optimized point cloud model of the point distribution of the inspected equipment; Based on the optimized point cloud model of the inspected equipment point distribution, determine and output the point set contour line of the inspected equipment points; Plan the cruise route and image shooting points according to the point set contour line, and acquire images; The acquired image is preprocessed, and defect detection is performed on the preprocessed image to obtain a detection result.

2. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 1 is characterized in that: The point cloud model of the distribution of the points of the equipment under inspection in the target inspection area based on the point cloud layered loading includes: Preprocess the original point cloud data to obtain point cloud data; Performing hierarchical segmentation on the point cloud data, and visualizing the point cloud data according to the hierarchical segmentation to obtain a first image; Point cloud modeling is performed based on the first image to generate a point cloud model of the point distribution of the inspected device.

3. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 2 is characterized in that: The construction of a nonlinear scale space pyramid to perform feature extraction and scale space analysis on the inspected device points of different scales includes: Performing grayscale processing on the acquired first image; Construct a nonlinear diffusion function to model the image and generate a nonlinear scale space pyramid; The nonlinear scale space pyramid includes multiple scale levels, each level corresponds to a scale, and is used to detect features of different scales of the image; The first image feature points at multiple scales are obtained based on a nonlinear scale space pyramid.

4. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 1 is characterized in that: Based on the main direction of the feature point, an N-dimensional description vector of the first image is determined, including: Based on the main direction of the feature point, a window centered on the feature point is defined in the gradient image; Subdivide the window into m sub-regions, each sub-region has a size of n×n; The first sub-region is weighted using the first Gaussian kernel to obtain a sub-region description vector with a length of n1; The second sub-region is weighted using the second Gaussian kernel to obtain a sub-region description vector with a length of n2; The two sub-region description vectors are iterated and normalized in turn to obtain a description vector of n×n1×n2=N.

5. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 1 is characterized in that: The cruise route extraction of the minimum circumscribed rectangle of the inspected equipment point based on the optimized inspected equipment point distribution point cloud model includes: The adjacency relationship of the triangulated mesh of the point set is obtained, and the outer contour of the point set of the detected equipment line in the target detection area is determined according to the adjacency relationship.

6. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 5 is characterized in that: The adjacency relationship of the triangulated mesh of the obtained point set includes: Traverse the triangulation network. If the number of adjacent triangles of a triangle is less than K, traverse each edge of the triangle: If the current edge E1 only belongs to the current triangle, mark the current edge E1 as the contour boundary; If the current edge E2 is a candidate contour boundary, obtain the starting point A and the end point B of the current edge E2, then add point A to the point chain, and search for other triangle edges starting from point A; If the current edge E3 is a candidate contour boundary, obtain the starting point A and the end point C of the current edge E3, then add point C to the point chain, and use point C as the starting point to search for other triangle edges; If the current edge E2 or E3 is a non-contour boundary, search other triangle edges. Repeat the traversal of the triangulated network until the search endpoint returns to the starting point B, and output the point set contour line of the detected device point.

7. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 1 is characterized in that: According to the point set contour line, the cruise line of the minimum circumscribed rectangle of the inspected equipment point is extracted, including: The coordinates of the center point of the contour are obtained as (x0, y0), the coordinates of the contour point P1 are (x1, y1), the coordinates of P2 are (x2, y2), and the angle between the line segment P1P2 and the positive direction of the X-axis is θ; Rotate the original contour by θ around the contour center point to obtain the new coordinates of the contour point (x1', y1'); Get the coordinate range of each contour point after rotation and get the circumscribed rectangle; Rotate the circumscribed rectangle in the opposite direction by θ around the contour center to obtain the minimum area circumscribed rectangle of the contour point set; Through the minimum area circumscribed moment, the direction and size of the object are determined and the cruise route is constructed.

8. The method for autonomous inspection by unmanned aerial vehicles based on precise re-shooting according to claim 1 is characterized in that: The preprocessing of the acquired image comprises: Screening the image data to remove the image data whose pixel value is less than a set threshold; Perform frame extraction processing on the filtered image data; Extracting the timestamp of the frame-decoded image includes: obtaining the image scale feature and constructing the scale space; discretizing the scale space and obtaining the scale factor of each sub-layer according to the discretized scale space; based on the scale factor, converting the pixels of the scale factor into time to obtain the timestamp of the image; The ambient brightness at the corresponding timestamp is obtained according to the timestamp of the image, and the image data is subjected to light compensation processing.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the drone autonomous inspection method based on precise re-shooting as described in any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the drone autonomous inspection method based on precise re-shooting as described in any one of claims 1 to 8 is implemented.

Images