A method and system for unmanned aerial vehicle (UAV) visual inspection and control for power grid operation and maintenance.

Abstract

This invention relates to the field of visual inspection technology, specifically to a UAV visual inspection control method and system for power grid operation and maintenance. The method includes: acquiring a 3D point cloud map of the power grid equipment area; controlling the visual sensor mounted on the UAV to collect image information of key equipment marker points; and adjusting the UAV's flight attitude based on the image information to align the center of the visual sensor's field of view with the center position of the power grid equipment in the 3D point cloud map, ensuring that the optical axis of the visual sensor is aligned with the normal direction of the power grid equipment, thus obtaining an initial alignment attitude. This invention detects the peak consistency of the sharpness of each edge marker point as pitch and roll angles change, ensuring that the imaging plane of the visual sensor is parallel to the equipment surface, thereby guaranteeing that the optical axis is aligned with the normal direction of the equipment. It does not rely on high-precision inertial navigation or external ranging equipment, but only utilizes its own visual information to complete attitude alignment, eliminating field-of-view distortion caused by UAV hovering attitude deviations or equipment surface tilt.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, specifically to a UAV visual inspection control method and system for power grid operation and maintenance. Background Technology

[0002] Currently, traditional methods typically rely solely on the UAV's own inertial navigation attitude angle or the alignment of a single marker center, without using the sharpness peak consistency of multiple edge markers to correct the parallelism between the imaging plane and the target surface. When the equipment surface is tilted or the UAV's hovering attitude has slight deviations, although the field of view center may be aligned with the equipment center, the optical axis is not perpendicular to the equipment surface, resulting in perspective distortion in the acquired image. This stretches or compresses the contour size and defect features of the key detection area in the image. Furthermore, because the field of view coverage is not verified through the distribution of edge markers, key detection areas are prone to deviating from the field of view center or even being located at the edge of the field of view. This leads to insufficient resolution or complete omission of images in that area during subsequent analysis, requiring repeated flight acquisitions and reducing inspection efficiency.

[0003] Furthermore, traditional methods do not pre-establish a mapping table between pixel deviation and gimbal angle. Instead, they approximate the target center by solving spatial geometric relationships online or using image servo iteration during each inspection. This real-time solution method requires repeated 3D coordinate transformations and matrix inversion operations, consuming significant onboard computing resources and extending response time. Simultaneously, traditional methods typically only perform one gimbal adjustment, without acquiring correction images, detecting residual deviations, or performing cyclic compensation. When the gimbal has transmission clearances, mechanical hysteresis, or experiences minor fuselage sway due to airflow disturbances, the target component may still deviate from the center by several pixels in the final image, and this deviation cannot be eliminated. This inconsistent imaging position makes images acquired across different flights or with different equipment lack spatial comparability, affecting the stability and accuracy of subsequent defect identification algorithms based on fixed-area pixel analysis. Summary of the Invention

[0004] To achieve the above objectives, the present invention provides the following technical solution: a UAV visual inspection and control method for power grid operation and maintenance, comprising:

[0005] Obtain a 3D point cloud map of the power grid equipment area;

[0006] The visual sensor on the control drone collects image information of the key equipment identification points, and adjusts the flight attitude of the drone according to the image information so that the center of the field of view of the visual sensor is aligned with the center position of the power grid equipment in the three-dimensional point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thus obtaining the initial alignment attitude.

[0007] The first offset distance is determined based on the center position of the field of view of the visual sensor under the initial alignment posture and the center position of the power grid equipment in the three-dimensional point cloud map. The second offset distance is determined based on the center point of the key detection area identified by the equipment outline edge information in the three-dimensional point cloud map and the center position of the power grid equipment. The deflection angle is calculated based on the ratio of the first offset distance to the second offset distance. The inspection flight path of the UAV to the preset hovering position of the tower to be inspected is planned with the initial alignment posture as the starting point.

[0008] The drone is controlled to fly along the inspection flight path to the preset hovering position. When the gimbal is in the zero position, the inspection camera is controlled to capture the initial image of the target component of the tower. The pixel coordinates of the target component and the vertical pixel deviation value of the center of the image are extracted from the initial image. The target component is located in the key detection area.

[0009] The vertical pixel deviation value is substituted into a pre-established pixel deviation-gimbal rotation angle lookup table to calculate the target rotation angle of the gimbal. The gimbal is then driven to rotate around the deflection axis by the target rotation angle so that the target component is located in the center of the image in subsequent frames.

[0010] Preferably, the deflection axis is perpendicular to the plane containing the angle between the line of sight of the inspection camera and the horizontal plane. The pixel deviation-pan-tilt angle reference table is obtained in advance through ground calibration tests, describing the correspondence between the vertical pixel offset of the target component in the image and the required rotation angle of the pan-tilt. The target rotation angle is the angle that the line of sight of the inspection camera needs to rotate from the direction aligned with the center position of the power grid equipment in the three-dimensional point cloud map to the direction aligned with the current actual position of the target component. The three-dimensional point cloud map is formed by fusing the power grid equipment point cloud data obtained by LiDAR scanning with multiple key equipment identification points.

[0011] Preferably, the method for establishing the pixel deviation-gimbal angle comparison table includes:

[0012] A calibration device is set on the ground, and the calibration device is equipped with multiple calibration marks at different vertical heights, including a reference calibration mark;

[0013] The drone is controlled to hover at the calibration hovering position, the gimbal is in the zero position, and the inspection camera is controlled to collect images of each calibration mark to obtain the calibration image;

[0014] For any of the calibration marks, determine the actual rotation angle of the gimbal corresponding to the height difference between the calibration mark and the reference calibration mark, and obtain the vertical pixel offset between the pixel coordinates of the calibration mark and the pixel coordinates of the reference calibration mark in the calibration screen;

[0015] Based on the actual rotation angle of the gimbal and the vertical pixel offset corresponding to the multiple calibration marks, the pixel deviation-gimbal rotation angle comparison table is obtained by fitting.

[0016] Preferably, the pixel deviation-gimbal angle comparison table includes an upper deviation comparison sub-table and a lower deviation comparison sub-table;

[0017] The above offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target component is located above the center of the image. The target rotation angle is the angle at which the gimbal rotates clockwise.

[0018] The lower offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target component is located below the center of the image. The target rotation angle is the angle at which the gimbal rotates counterclockwise.

[0019] Accordingly, the vertical pixel deviation value is substituted into a pre-established pixel deviation-gimbal angle lookup table to calculate the target rotation angle of the gimbal, and the gimbal is driven to rotate around the deflection axis by the target rotation angle, including:

[0020] If the target component is located above the center of the screen in the initial image, the vertical pixel offset is substituted into the upper offset sub-table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle clockwise.

[0021] If the target component is located below the center of the screen in the initial image, the vertical pixel offset is substituted into the lower offset sub-table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle counterclockwise.

[0022] Preferably, the method further includes:

[0023] After driving the pan-tilt unit to rotate the target by the rotation angle, the inspection camera is controlled to re-capture the image of the target component of the tower to obtain a corrected image;

[0024] Extract the residual deviation value between the pixel coordinates of the target component and the center of the image from the calibrated image;

[0025] If the residual deviation value is greater than the preset tolerance threshold, the residual deviation value is substituted back into the pixel deviation-gimbal angle lookup table to calculate the correction angle, and the gimbal is driven to rotate by the correction angle until the residual deviation value is not greater than the preset tolerance threshold.

[0026] Preferably, identifying the center point of the key detection area based on the device outline edge information in the 3D point cloud map includes:

[0027] Based on the correspondence between the projected contour of the 3D point cloud map from the perspective of the UAV and the actual geometric shape of the power grid equipment, the key detection area of ​​the power grid equipment is determined by edge matching.

[0028] The key detection area is the area where the components of the power grid equipment that need to be inspected are located, and the center point of the key detection area is the geometric center of the key detection area. The center point is used to determine the second offset distance.

[0029] Preferably, the deflection angle is calculated based on the ratio of the first offset distance to the second offset distance, and the inspection flight path of the UAV to the preset hovering position of the tower to be inspected is planned starting from the initial alignment attitude, including:

[0030] The ratio of the second offset distance to the first offset distance is used as a scaling factor to scale and adjust the position of the UAV in the initial alignment posture to obtain the preset hovering position;

[0031] The inspection flight path is a straight line from the UAV position in the initial alignment attitude to the preset hovering position. The UAV maintains the same flight altitude in the initial alignment attitude while flying along the inspection flight path.

[0032] Preferably, the control drone carries a visual sensor that acquires image information of the key equipment identification points, and adjusts the drone's flight attitude based on the image information, so that the center of the visual sensor's field of view is aligned with the center position of the power grid equipment in the 3D point cloud map, and the optical axis of the visual sensor is aligned with the normal direction of the power grid equipment, thus obtaining an initial alignment attitude, including:

[0033] The multiple key equipment identification points include a central equipment identification point and multiple edge equipment identification points, and the position of the central equipment identification point coincides with the equipment center point in the corresponding three-dimensional point cloud map;

[0034] The image information of the central device identifier point is acquired, and the flight attitude of the UAV is adjusted accordingly until the central device identifier point is imaged at the center of the image of the visual sensor;

[0035] Image information of each edge device identifier point is collected, and the flight attitude of the UAV is adjusted according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is parallel to the surface of the power grid device.

[0036] Preferably, before acquiring the image information of each of the edge device identifier points, the method further includes:

[0037] Using a ground reference station, the takeoff position of the UAV is adjusted to a horizontal position;

[0038] Accordingly, adjusting the flight attitude of the UAV based on the peak data of the corresponding sharpness variation curve until the field of view of the visual sensor is parallel to the surface of the power grid equipment includes:

[0039] The flight attitude of the UAV is adjusted according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is also adjusted to the horizontal state. At this time, the optical axis of the visual sensor in the initial alignment attitude is consistent with the normal direction of the power grid device.

[0040] A drone-based visual inspection and control system for power grid operation and maintenance, applicable to the aforementioned drone-based visual inspection and control method for power grid operation and maintenance, includes:

[0041] The map acquisition unit is configured to acquire a 3D point cloud map of the power grid equipment area;

[0042] The center alignment unit is configured to control the visual sensor carried by the UAV to collect image information of the key equipment identification points, and adjust the flight attitude of the UAV according to the image information so that the center of the field of view of the visual sensor is aligned with the center position of the power grid equipment in the three-dimensional point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thereby obtaining the initial alignment attitude.

[0043] The path planning unit is configured to determine a first offset distance based on the center position of the field of view of the visual sensor under the initial alignment posture and the center position of the power grid equipment in the three-dimensional point cloud map, and to determine a second offset distance based on the center point of the key detection area identified by the equipment outline edge information in the three-dimensional point cloud map and the center position of the power grid equipment, to calculate the deflection angle based on the ratio of the first offset distance to the second offset distance, and to plan the inspection flight path of the UAV to the preset hovering position of the tower to be inspected, starting from the initial alignment posture;

[0044] The central detection unit is configured to control the UAV to fly along the inspection flight path to the preset hovering position, and control the inspection camera to collect the initial image of the target component of the tower when the gimbal is in the zero position. The unit extracts the pixel coordinates of the target component and the vertical pixel deviation value between the center of the image and the center of the image from the initial image. The target component is located in the key detection area.

[0045] The gimbal adjustment unit is configured to substitute the vertical pixel deviation value into a pre-established pixel deviation-gimbal rotation angle lookup table, calculate the target rotation angle of the gimbal, and drive the gimbal to rotate around the deflection axis by the target rotation angle so that the target component is located in the center of the image in the subsequent image.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] (1) This invention achieves high-precision acquisition of the initial alignment attitude of the UAV by adjusting the two-level visual feedback based on the center marker point and the edge marker point. By making the center marker point imaged in the center of the screen to ensure that the center of the field of view is aligned with the center of the device, and by detecting the peak consistency of the sharpness of each edge marker point as pitch and roll angle change, the imaging plane of the visual sensor is made parallel to the surface of the device, thereby ensuring that the optical axis is consistent with the normal direction of the device. This process does not rely on high-precision inertial navigation or external ranging equipment, and can complete the attitude alignment using only its own visual information. It effectively eliminates the field of view distortion caused by UAV hovering attitude deviation or device surface tilt, and ensures that subsequent shooting always obtains target images from a standardized perspective, avoiding missed shots or image distortion in key areas.

[0048] (2) This invention achieves precise centering control of the target component in the image by establishing a pixel deviation-gimbal rotation angle comparison table and introducing a closed-loop iterative correction mechanism. The comparison table is obtained in advance based on ground calibration tests, and the vertical pixel offset is directly mapped to the gimbal rotation angle. The table lookup operation is fast and does not require real-time calculation of spatial geometric relationships, which is suitable for scenarios with limited airborne computing resources. At the same time, after the initial rotation, the correction image is collected and the residual deviation is calculated. If it exceeds the tolerance, the table is looked up and compensated in a loop until the target component is accurately located in the center of the image. This closed-loop correction mechanism effectively compensates for the fitting error of the comparison table, the gimbal transmission gap and the influence of external airflow disturbance, ensuring that the target component is always imaged at the same pixel position, providing a highly consistent standard image for subsequent defect identification or status detection, and improving the comparability and analysis reliability of inspection data. Attached Figure Description

[0049] Figure 1 This is a schematic flowchart of the overall method in one embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the overall system architecture in one embodiment of the present invention.

[0051] In the diagram: 1. Map acquisition unit; 2. Center alignment unit; 3. Path planning unit; 4. Center detection unit; 5. Gimbal adjustment unit. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1, please refer to Figure 1 This invention provides a technical solution: a UAV visual inspection control method for power grid operation and maintenance, comprising:

[0054] S1. Obtain a 3D point cloud map of the power grid equipment area;

[0055] S2. Control the visual sensor on the drone to collect image information of key equipment identification points, and adjust the drone's flight attitude according to the image information so that the center of the visual sensor's field of view is aligned with the center position of the power grid equipment in the 3D point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thus obtaining the initial alignment attitude.

[0056] S3. Determine the first offset distance based on the center position of the field of view of the visual sensor under the initial alignment posture and the center position of the power grid equipment in the three-dimensional point cloud map. Determine the second offset distance based on the center point of the key detection area and the center position of the power grid equipment by identifying the equipment outline edge information in the three-dimensional point cloud map. Calculate the deflection angle based on the ratio of the first offset distance to the second offset distance. Plan the inspection flight path of the UAV to the preset hovering position of the tower to be inspected, starting from the initial alignment posture.

[0057] S4. Control the drone to fly along the inspection flight path to the preset hovering position. When the gimbal is in the zero position, control the inspection camera to collect the initial image of the target component of the tower. Extract the pixel coordinates of the target component from the initial image and the vertical pixel deviation value between the target component and the center of the image. The target component is located in the key detection area.

[0058] S5. Substitute the vertical pixel deviation value into the pre-established pixel deviation-gimbal rotation angle reference table, calculate the target rotation angle of the gimbal, drive the gimbal to rotate around the deflection axis by the target rotation angle, so that the target component is located in the center of the image in the subsequent image.

[0059] It should be noted that the UAV is equipped with a gimbal and a vision sensor to pre-acquire a 3D point cloud map of the inspection target area and store it on the airborne. The UAV is controlled to ascend to the initial observation altitude, and the vision sensor is used to identify key markers in the area. The yaw and pitch angles of the UAV are adjusted according to the image coordinates of the markers so that the center of the sensor's field of view is aligned with the center of the equipment in the point cloud map. At the same time, the optical axis is adjusted to be consistent with the normal direction of the equipment surface to complete the initial alignment attitude.

[0060] In the initial alignment posture, the coordinates of the field of view center in the 3D map are read, and the Euclidean distance between the center coordinates and the device center coordinates is calculated as the first offset distance. Then, the device outline edge is extracted from the point cloud map, and the center point of the key detection area is identified using an edge detection algorithm. The Euclidean distance between the center point and the device center is calculated as the second offset distance. The first offset distance is divided by the second offset distance, and the arctangent is taken to obtain the deflection angle. This deflection angle reflects the angular difference between the actual orientation of the device and the initial field of view direction. Starting from the coordinates and orientation of the initial alignment posture, an arc-shaped inspection flight path is planned in combination with the deflection angle. This path enables the UAV to safely fly to the preset hovering position and ensures that the main body of the device does not fly out of the field of view during subsequent shooting.

[0061] After the UAV flies along the planned path to the preset hovering position, the gimbal remains at zero position, and the inspection camera captures the initial image. The center pixel coordinates of the smallest bounding rectangle of the target component are extracted from the initial image, and the vertical pixel deviation value between this center and the center of the image is calculated. A one-to-one correspondence table between pixel deviation and gimbal pitch angle is established in advance through ground calibration experiments. This table covers the angles corresponding to various deviation values. The current vertical deviation value is used as an index to look up the table. If the deviation value is between two calibration points, linear interpolation is performed to obtain the target rotation angle of the gimbal around the pitch axis. Then, the gimbal is driven to perform this angle rotation so that the target component is located in the center of the image in the subsequently acquired images, completing fine alignment and providing a standardized observation perspective for subsequent high-precision condition detection.

[0062] In an optional embodiment, the deflection axis is perpendicular to the plane containing the angle between the inspection camera's line of sight and the horizontal plane. The pixel deviation-pan-tilt angle reference table is obtained in advance through ground calibration tests, describing the correspondence between the vertical pixel offset of the target component in the image and the required rotation angle of the pan-tilt. The target rotation angle is the angle that the inspection camera's line of sight needs to rotate from the direction aligned with the center position of the power grid equipment in the 3D point cloud map to the direction aligned with the current actual position of the target component. The 3D point cloud map is formed by fusing the power grid equipment point cloud data obtained by LiDAR scanning with multiple key equipment identification points.

[0063] It should be noted that the drone is equipped with a LiDAR, gimbal, and visual sensor. First, the LiDAR scans the equipment area and identifies multiple pre-deployed marker points. The LiDAR point cloud is then fused with the spatial coordinates of the marker points to generate a 3D point cloud map containing the equipment outline and the marker points. Based on the center position of the equipment and its normal direction in the map, the drone's position and the gimbal angle are adjusted so that the center of the visual sensor's field of view is aligned with the center of the equipment and the optical axis is consistent with the normal, serving as the initial alignment posture.

[0064] In the initial alignment posture, the coordinates of the field of view center in the map are read, and the three-dimensional distance between the center of the field of view and the device center is calculated as the first offset distance. The device outline edge is extracted from the point cloud map, and the center point of the key detection area is identified by contour analysis. The distance between the center point and the device center is calculated as the second offset distance. The first offset distance is divided by the second offset distance, and the arctangent is taken to obtain the deflection angle, which reflects the deviation between the actual orientation of the device and the line of sight. Based on the initial position, the deflection angle, and the target hovering position, a smooth flight path is planned to enable the UAV to safely fly to the predetermined hovering point.

[0065] Upon reaching the hover position, the gimbal remains at zero, and the camera captures the initial image. The target component is detected, and its pixel coordinates are calculated to determine the vertical pixel deviation from the center of the image. This deviation is used to look up a pre-made pixel deviation-gimbal rotation angle table. This table, obtained through ground calibration experiments, records the required rotation angle of the gimbal around the deflection axis corresponding to different vertical pixel offsets. The deflection axis is perpendicular to the plane containing the angle between the camera's line of sight and the horizontal plane, which is the gimbal's pitch axis. The target rotation angle is defined as the angle required to rotate the line of sight from the current direction of the aligned device center to the direction of the actual position of the aligned target component. The target rotation angle is obtained by looking up the table based on the current vertical deviation value; if the deviation is within the calibration points, linear interpolation is performed. After the gimbal executes this angle, the target component is precisely centered in the subsequent images, achieving fine-grained alignment.

[0066] In an optional embodiment, the method for establishing the pixel deviation-gimbal angle lookup table includes:

[0067] A calibration device is set up on the ground, and the calibration device is equipped with multiple calibration marks at different vertical heights, including a reference calibration mark;

[0068] Control the drone to hover at the designated hovering position, with the gimbal in the zero position, and control the inspection camera to collect images of each calibration mark to obtain the calibration image;

[0069] For any calibration mark, determine the actual rotation angle of the gimbal corresponding to the height difference between the calibration mark and the reference calibration mark, and obtain the vertical pixel offset between the pixel coordinates of the calibration mark and the pixel coordinates of the reference calibration mark in the calibration screen.

[0070] Based on the actual rotation angle of the gimbal and the vertical pixel offset corresponding to multiple calibration marks, a pixel deviation-gimbal rotation angle comparison table is obtained.

[0071] It should be noted that a calibration device is set up on a flat ground. The device is a vertical column with multiple calibration marks fixed at equal intervals along the height direction. Each mark has the same visual characteristics and the same size. The lowest mark is used as the reference calibration mark. The UAV flies to the preset calibration hovering position. The horizontal distance between this position and the calibration device is fixed. The gimbal is kept at zero position, that is, the pitch angle and yaw angle are both zero. The inspection camera is facing the calibration device to take pictures and obtain a calibration image containing all the marks.

[0072] For each non-reference mark, based on its actual height difference with the reference mark and the horizontal distance from the UAV to the calibration device, the arctangent function is used to calculate the angle that the gimbal needs to rotate around the pitch axis. This angle is the actual rotation angle of the gimbal, so that the camera optical axis changes from being aligned with the reference mark to being aligned with this mark. At the same time, the center pixel coordinates of the mark and the reference mark are extracted from the calibration image, and the difference between the two in the image row direction is calculated to obtain the vertical pixel offset. This calculation is performed sequentially for all marks to obtain multiple sets of data points corresponding to the actual rotation angle and the vertical pixel offset.

[0073] Using the vertical pixel offset as the independent variable and the actual rotation angle of the gimbal as the dependent variable, a linear fit is performed on these data points using the least squares method to obtain the fitted line equation. When the offset is small, the linear relationship is good; if the offset range is large, a quadratic polynomial fit is used to improve accuracy. The fitting results are made into a lookup table, which contains discrete offset nodes and their corresponding angle values. The node interval is determined according to the distribution of actual measurement points. This lookup table is stored onboard and used in subsequent operations to quickly interpolate and obtain the target rotation angle based on the real-time detected pixel deviation.

[0074] In an optional embodiment, the pixel deviation-gimbal angle reference table includes an upper deviation reference table and a lower deviation reference table;

[0075] The upper offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target part is located above the center of the image. The target rotation angle is the angle of the gimbal rotation in the clockwise direction.

[0076] The lower offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target part is located below the center of the image. The target rotation angle is the angle of the gimbal rotation in the counterclockwise direction.

[0077] Accordingly, the vertical pixel deviation value is substituted into a pre-established pixel deviation-gimbal rotation angle lookup table to calculate the target rotation angle of the gimbal, and the gimbal is driven to rotate around the deflection axis to the target rotation angle, including:

[0078] If the target component is located above the center of the screen in the initial image, the vertical pixel offset is substituted into the upper offset reference table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle clockwise.

[0079] If the target component is located below the center of the screen in the initial image, the vertical pixel offset is substituted into the lower offset sub-table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle counterclockwise.

[0080] It should be noted that two independent reference tables were established during the calibration phase. The upper reference table records the correspondence between the vertical upward offset and the clockwise rotation angle of the gimbal's pitch axis when the pixel coordinates of the target component are located above the center of the image. The lower reference table records the correspondence between the vertical downward offset and the counterclockwise rotation angle of the gimbal's pitch axis when the target component is located below the center of the image. Both reference tables were obtained through ground calibration experiments, each fitted with a linear or piecewise linear function, and stored onboard.

[0081] In formal inspection operations, after the UAV flies to the preset hovering position and the gimbal is at zero position, the inspection camera captures the initial image. The center pixel coordinates of the target component are extracted from the initial image and compared with the center pixel coordinates of the image. If the row coordinates of the target component's center are less than the row coordinates of the image center, it indicates that the target component is above the image center. In this case, the absolute value of the vertical pixel offset is used as an index to look up the upper offset lookup table, and the corresponding clockwise rotation angle value is obtained through linear interpolation. The gimbal is then driven to rotate clockwise around the pitch axis by this angle, causing the line of sight to rotate upward and pulling the target component back to the center of the image. If the row coordinates of the target component's center are greater than the row coordinates of the image center, it indicates that the target component is below the image center. The lower offset lookup table is then looked up to obtain the counterclockwise rotation angle value, and the gimbal is driven to rotate counterclockwise by the corresponding angle, causing the line of sight to rotate downward and centering the target component. This directional lookup table and directional rotation mechanism ensures that no matter how the target component deviates from its direction, the gimbal always rotates along the shortest path towards the center, avoiding accidental reverse rotation, thereby improving alignment efficiency and accuracy.

[0082] In an optional embodiment, the method further includes:

[0083] After the target rotation angle is rotated by driving the gimbal, the inspection camera is controlled to re-acquire images of the target components of the tower to obtain a corrected image;

[0084] Extract the residual deviation between the pixel coordinates of the target component and the center of the image from the calibrated image;

[0085] If the residual deviation value is greater than the preset tolerance threshold, the residual deviation value is substituted back into the pixel deviation-gimbal angle lookup table to calculate the correction angle, and the gimbal is driven to rotate to correct the angle until the residual deviation value is no greater than the preset tolerance threshold.

[0086] It should be noted that after the gimbal executes the target rotation angle, it triggers the vision sensor again to acquire the current image, obtaining a calibrated image. The center pixel coordinates of the target component are extracted from the calibrated image, and the difference between these coordinates and the pixel row coordinates of the image center is calculated to obtain the residual deviation value. The absolute value of the residual deviation value is compared with a preset tolerance threshold, which is set as an acceptable centering error, such as two pixels. If the residual deviation value is greater than the threshold, it indicates that the initial lookup angle was not perfectly centered due to calibration fitting error or mechanical backlash. In this case, the residual deviation value is used as a new vertical pixel offset. Depending on whether the target component is above or below the image center, it is substituted into the upper or lower offset lookup table, respectively, and the corresponding correction angle is calculated through interpolation. The gimbal is then driven to continue rotating in the same direction by this correction angle. After each correction, the image is reacquired and a new residual deviation is calculated until the residual deviation value is no greater than the tolerance threshold. This closed-loop iterative mechanism can compensate for lookup table calibration deviations, gimbal transmission gaps, and external disturbances, ensuring that the target component is ultimately stably located in the image center, providing a precise and consistent observation benchmark for subsequent fine-tuning. Each corrected angle and residual deviation can be recorded in a log for subsequent online calibration and optimization of the lookup table.

[0087] In an optional embodiment, identifying the center point of the key detection area based on the device outline edge information in the 3D point cloud map includes:

[0088] Based on the correspondence between the projected contour of the 3D point cloud map from the perspective of the UAV and the actual geometric shape of the power grid equipment, the key detection areas of the power grid equipment are determined by edge matching.

[0089] Among them, the key detection area is the area where the components of the power grid equipment that need to be inspected are located, and the center point of the key detection area is the geometric center of the key detection area. The center point is used to determine the second offset distance.

[0090] It should be noted that the visible point cloud from the current viewpoint of the UAV is extracted from the 3D point cloud map, and orthogonally projected along the line of sight to obtain a 2D contour projection map of the device from that viewpoint. Simultaneously, based on the device's standard 3D model or a typical contour template defined offline, the actual geometric edges of each key detection component are extracted. A shape context matching algorithm is used to register the edge features in the projection map with the template edges, calculate the similarity score, and select the matching region with the highest score as the key detection region. This region corresponds to the location of the component on the device that needs to be observed. Subsequently, the circumscribed rectangle or minimum circumscribed ellipse of this matching region in the 2D projection map is calculated, and its geometric center is taken as the center point of the key detection region. The 3D coordinates of this center point are obtained by back-projecting onto the point cloud map and used to calculate the second offset distance between it and the device center. This process does not depend on a specific device type, but is based solely on the matching of point cloud projection and shape template, and is applicable to various inspection targets with fixed shape characteristics.

[0091] In an optional embodiment, the deflection angle is calculated based on the ratio of the first offset distance to the second offset distance, and an inspection flight path is planned from the initial alignment attitude to the preset hovering position of the UAV to be inspected, including:

[0092] The ratio of the second offset distance to the first offset distance is used as a scaling factor to scale and adjust the position of the UAV in the initial alignment attitude to obtain the preset hovering position.

[0093] The inspection flight path is a straight line from the UAV's position in the initial alignment attitude to the preset hovering position. The UAV maintains the same flight altitude as in the initial alignment attitude while flying along the inspection flight path.

[0094] It should be noted that, in the initial alignment posture, the UAV position coordinates and device center coordinates are obtained, and the Euclidean distance between the field of view center and the device center is calculated as the first offset distance; at the same time, the center point coordinates of the key detection area are extracted from the point cloud map, and the Euclidean distance between the center point and the device center is calculated as the second offset distance; the second offset distance is divided by the first offset distance to obtain the scaling factor;

[0095] Using the device center as the scaling center, the displacement vector of the UAV position relative to the device center under the initial alignment attitude is multiplied by the scaling factor to obtain a new displacement vector of the preset hovering position relative to the device center. This vector is then added to the coordinates of the device center to obtain the coordinates of the preset hovering position. During this process, the flight altitude remains unchanged, that is, only the position in the horizontal plane is adjusted, while the vertical coordinate remains unchanged.

[0096] A straight flight path is then planned from the drone's initial alignment position to the preset hovering position. As the drone flies along this straight line, the optical axis of the visual sensor remains unchanged from the initial alignment posture, and the flight altitude is constant. Through this scaling adjustment, the direction of the line connecting the center of the drone's field of view and the center of the device at the preset hovering position corresponds exactly to the direction of the center of the critical detection area relative to the center of the device, thus ensuring that the critical detection area is located near the center of the field of view during subsequent hovering photography. This path planning is based solely on geometric proportions and is applicable to various flight missions that require alignment with specific target areas from different distances.

[0097] In one optional embodiment, the visual sensor mounted on the UAV is controlled to acquire image information of key equipment identification points, and the flight attitude of the UAV is adjusted according to the image information so that the center of the visual sensor's field of view is aligned with the center position of the power grid equipment in the 3D point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thus obtaining an initial alignment attitude, including:

[0098] Multiple key equipment identification points include a central equipment identification point and multiple edge equipment identification points, and the location of the central equipment identification point coincides with the equipment center point in the corresponding 3D point cloud map;

[0099] The image information of the central equipment marker is collected, and the flight attitude of the UAV is adjusted accordingly until the central equipment marker is imaged at the center of the image of the visual sensor.

[0100] Image information of each edge device marker point is collected, and the flight attitude of the drone is adjusted according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is parallel to the surface of the power grid equipment.

[0101] It should be noted that multiple key equipment markers include one central marker and at least three edge markers. The central marker is located at the geometric center of the equipment surface, and its position coincides with the coordinates of the equipment's center point in the 3D point cloud map. The edge markers are evenly distributed at the four corners or edge areas of the equipment surface. After the UAV flies near the equipment, the vision sensor begins to acquire images. First, the imaging position of the central marker is detected. By adjusting the UAV's horizontal position and yaw angle, the imaging point is gradually moved towards the central area of ​​the image sensor until its pixel coordinates coincide with the center of the image. At this point, the field of view center is aligned with the equipment center.

[0102] Subsequently, the image sharpness of each edge marker point is detected sequentially. At the current altitude and distance of the UAV, the pitch and roll angles of the UAV are fine-tuned. After each adjustment, one frame of image is acquired, and the image gradient magnitude of each edge marker point region is calculated as a sharpness evaluation index. The curve of this index changing with the adjustment angle is recorded. When the sharpness of a certain edge marker point reaches its peak, it indicates that the point is in the optimal focus plane. The attitude is adjusted further so that the adjustment angles corresponding to the peak points of the sharpness curves of all edge marker points tend to be consistent, that is, all edge marker points simultaneously reach the highest sharpness, indicating that the imaging plane of the vision sensor is parallel to the device surface, and at this time the center of the field of view is consistent with the direction of the surface normal. Finally, the UAV maintains this attitude, that is, the initial alignment attitude, which satisfies both center alignment and optical axis perpendicularity. This process does not depend on the specific shape of the device, but is based only on the spatial distribution of marker points and image quality feedback, and is applicable to the attitude alignment of various planar or near-planar targets.

[0103] In an optional embodiment, before acquiring image information of each edge device identifier point, the method further includes:

[0104] Using a ground reference station, the takeoff position of the UAV is adjusted to be horizontal;

[0105] Accordingly, adjusting the drone's flight attitude based on the peak data of the corresponding sharpness variation curve until the field of view of the visual sensor is parallel to the surface of the power grid equipment includes:

[0106] Adjust the drone's flight attitude according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is also adjusted to a horizontal state. At this time, the optical axis of the visual sensor in the initial alignment attitude is consistent with the normal direction of the power grid equipment.

[0107] It should be noted that before the drone takes off, the ground reference station uses the level calibration module to adjust the takeoff platform to an absolute level, ensuring that the initial roll and pitch angles of the drone are accurate. After takeoff, the drone hovers in front of the target equipment, the visual sensor collects the image of the center marker point, and the drone's horizontal position and yaw angle are adjusted so that the center marker point is imaged in the center of the screen, thus completing the center alignment.

[0108] Subsequently, the sensors sequentially acquire images of edge marker points distributed in the four corner areas of the device surface. During imaging at each edge marker point, the pitch and roll angles of the UAV are fine-tuned. For each adjustment step, the image gradient amplitude within the marker point area is recorded as a sharpness index, obtaining a sharpness curve that varies with angle. When the sharpness curves of all edge marker points reach their respective peak values, and the angle adjustment amount corresponding to each peak value is the same, it indicates that the imaging plane is parallel to the device surface. Since the takeoff platform has been calibrated to be horizontal by the reference station, this parallel state is equivalent to the field of view of the visual sensor being horizontal. Combined with the fact that the center marker point is already centered, the optical axis is perpendicular to the device surface, that is, the optical axis is consistent with the device normal direction, completing the initial alignment attitude. This process uses the ground horizontal reference to eliminate attitude zero-position error, and then achieves precise parallel alignment by judging the consistency of sharpness peak values ​​of multiple edge points, without depending on the specific shape of the device.

[0109] Example 2, please refer to Figure 2 This invention provides a technical solution: a UAV visual inspection and control system for power grid operation and maintenance, applicable to the aforementioned UAV visual inspection and control method for power grid operation and maintenance, comprising:

[0110] Map acquisition unit 1 is configured to acquire a 3D point cloud map of the power grid equipment area;

[0111] Center alignment unit 2 is configured to control the visual sensor carried by the UAV to collect image information of key equipment identification points, and adjust the flight attitude of the UAV according to the image information so that the center of the field of view of the visual sensor is aligned with the center position of the power grid equipment in the three-dimensional point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thus obtaining the initial alignment attitude.

[0112] The path planning unit 3 is configured to determine the first offset distance based on the center position of the field of view of the visual sensor and the center position of the power grid equipment in the three-dimensional point cloud map under the initial alignment attitude, and to determine the second offset distance based on the center point of the key detection area and the center position of the power grid equipment identified by the equipment outline edge information in the three-dimensional point cloud map. The deflection angle is calculated based on the ratio of the first offset distance to the second offset distance, and the inspection flight path of the UAV to the preset hovering position of the tower to be inspected is planned with the initial alignment attitude as the starting point.

[0113] The central detection unit 4 is configured to control the UAV to fly along the inspection flight path to the preset hovering position, and control the inspection camera to collect the initial image of the target component of the tower when the gimbal is in the zero position. The pixel coordinates of the target component and the vertical pixel deviation value of the center of the image are extracted from the initial image, and the target component is located in the key detection area.

[0114] The gimbal adjustment unit 5 is configured to substitute the vertical pixel deviation value into a pre-established pixel deviation-gimbal rotation angle reference table, calculate the target rotation angle of the gimbal, drive the gimbal to rotate around the deflection axis by the target rotation angle, so that the target component is located in the center of the image in the subsequent image.

[0115] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A UAV visual inspection control method for power grid operation and maintenance, characterized in that, include: Obtain a 3D point cloud map of the power grid equipment area; The visual sensor on the control drone collects image information of the key equipment identification points, and adjusts the flight attitude of the drone according to the image information so that the center of the field of view of the visual sensor is aligned with the center position of the power grid equipment in the three-dimensional point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thus obtaining the initial alignment attitude. The first offset distance is determined based on the center position of the field of view of the visual sensor under the initial alignment posture and the center position of the power grid equipment in the three-dimensional point cloud map. The second offset distance is determined based on the center point of the key detection area identified by the equipment outline edge information in the three-dimensional point cloud map and the center position of the power grid equipment. The deflection angle is calculated based on the ratio of the first offset distance to the second offset distance. The inspection flight path of the UAV to the preset hovering position of the tower to be inspected is planned with the initial alignment posture as the starting point. The drone is controlled to fly along the inspection flight path to the preset hovering position. When the gimbal is in the zero position, the inspection camera is controlled to capture the initial image of the target component of the tower. The pixel coordinates of the target component and the vertical pixel deviation value of the center of the image are extracted from the initial image. The target component is located in the key detection area. The vertical pixel deviation value is substituted into a pre-established pixel deviation-gimbal rotation angle lookup table to calculate the target rotation angle of the gimbal. The gimbal is then driven to rotate around the deflection axis by the target rotation angle so that the target component is located in the center of the image in subsequent frames.

2. The UAV visual inspection control method for power grid operation and maintenance according to claim 1, characterized in that, The deflection axis is perpendicular to the plane containing the angle between the line of sight of the inspection camera and the horizontal plane. The pixel deviation-pan-tilt angle reference table is obtained in advance through ground calibration tests and describes the correspondence between the vertical pixel offset of the target component in the image and the required rotation angle of the pan-tilt. The target rotation angle is the angle that the line of sight of the inspection camera needs to rotate from the direction aligned with the center position of the power grid equipment in the three-dimensional point cloud map to the direction aligned with the current actual position of the target component. The three-dimensional point cloud map is formed by fusing the power grid equipment point cloud data obtained by LiDAR scanning with multiple key equipment identification points.

3. The UAV visual inspection control method for power grid operation and maintenance according to claim 2, characterized in that, The method for establishing the pixel deviation-gimbal angle comparison table includes: A calibration device is set on the ground, and the calibration device is equipped with multiple calibration marks at different vertical heights, including a reference calibration mark; The drone is controlled to hover at the calibration hovering position, the gimbal is in the zero position, and the inspection camera is controlled to collect images of each calibration mark to obtain the calibration image; For any of the calibration marks, determine the actual rotation angle of the gimbal corresponding to the height difference between the calibration mark and the reference calibration mark, and obtain the vertical pixel offset between the pixel coordinates of the calibration mark and the pixel coordinates of the reference calibration mark in the calibration screen; Based on the actual rotation angle of the gimbal and the vertical pixel offset corresponding to the multiple calibration marks, the pixel deviation-gimbal rotation angle comparison table is obtained by fitting.

4. The UAV visual inspection control method for power grid operation and maintenance according to claim 3, characterized in that, The pixel deviation-gimbal angle comparison table includes an upper deviation comparison table and a lower deviation comparison table; The above offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target component is located above the center of the image. The target rotation angle is the angle at which the gimbal rotates clockwise. The lower offset sub-table describes the correspondence between the target rotation angle and the vertical pixel offset when the target component is located below the center of the image. The target rotation angle is the angle at which the gimbal rotates counterclockwise. Accordingly, the vertical pixel deviation value is substituted into a pre-established pixel deviation-gimbal angle lookup table to calculate the target rotation angle of the gimbal, and the gimbal is driven to rotate around the deflection axis by the target rotation angle, including: If the target component is located above the center of the screen in the initial image, the vertical pixel offset is substituted into the upper offset sub-table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle clockwise. If the target component is located below the center of the screen in the initial image, the vertical pixel offset is substituted into the lower offset sub-table to obtain the target rotation angle, and the gimbal is driven to rotate the target rotation angle counterclockwise.

5. A UAV visual inspection control method for power grid operation and maintenance according to claim 4, characterized in that, The method further includes: After driving the pan-tilt unit to rotate the target by the rotation angle, the inspection camera is controlled to re-capture the image of the target component of the tower to obtain a corrected image; Extract the residual deviation value between the pixel coordinates of the target component and the center of the image from the calibrated image; If the residual deviation value is greater than the preset tolerance threshold, the residual deviation value is substituted back into the pixel deviation-gimbal angle lookup table to calculate the correction angle, and the gimbal is driven to rotate by the correction angle until the residual deviation value is not greater than the preset tolerance threshold.

6. The UAV visual inspection control method for power grid operation and maintenance according to claim 5, characterized in that, The center point of the key detection area is identified based on the device outline edge information in the 3D point cloud map, including: Based on the correspondence between the projected contour of the 3D point cloud map from the perspective of the UAV and the actual geometric shape of the power grid equipment, the key detection area of ​​the power grid equipment is determined by edge matching. The key detection area is the area where the components of the power grid equipment that need to be inspected are located, and the center point of the key detection area is the geometric center of the key detection area. The center point is used to determine the second offset distance.

7. A UAV visual inspection control method for power grid operation and maintenance according to claim 6, characterized in that, The deflection angle is calculated based on the ratio of the first offset distance to the second offset distance, and the inspection flight path of the UAV to the preset hovering position of the tower to be inspected is planned starting from the initial alignment attitude, including: The ratio of the second offset distance to the first offset distance is used as a scaling factor to scale and adjust the position of the UAV in the initial alignment posture to obtain the preset hovering position; The inspection flight path is a straight line from the UAV position in the initial alignment attitude to the preset hovering position. The UAV maintains the same flight altitude in the initial alignment attitude while flying along the inspection flight path.

8. A UAV visual inspection control method for power grid operation and maintenance according to claim 7, characterized in that, The system controls the visual sensor mounted on the drone to acquire image information of the key equipment identification points, and adjusts the drone's flight attitude based on the image information, so that the center of the visual sensor's field of view is aligned with the center position of the power grid equipment in the 3D point cloud map, and the optical axis of the visual sensor is aligned with the normal direction of the power grid equipment, thus obtaining an initial alignment attitude, including: The multiple key equipment identification points include a central equipment identification point and multiple edge equipment identification points, and the position of the central equipment identification point coincides with the equipment center point in the corresponding three-dimensional point cloud map; The image information of the central device identifier point is acquired, and the flight attitude of the UAV is adjusted accordingly until the central device identifier point is imaged at the center of the image of the visual sensor; Image information of each edge device identifier point is collected, and the flight attitude of the UAV is adjusted according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is parallel to the surface of the power grid device.

9. A UAV visual inspection control method for power grid operation and maintenance according to claim 8, characterized in that, Before acquiring image information of each of the edge device identifier points, the method further includes: Using a ground reference station, the takeoff position of the UAV is adjusted to a horizontal position; Accordingly, adjusting the flight attitude of the UAV based on the peak data of the corresponding sharpness variation curve until the field of view of the visual sensor is parallel to the surface of the power grid equipment includes: The flight attitude of the UAV is adjusted according to the peak data of the corresponding sharpness change curve until the field of view of the visual sensor is also adjusted to the horizontal state. At this time, the optical axis of the visual sensor in the initial alignment attitude is consistent with the normal direction of the power grid device.

10. A UAV visual inspection and control system for power grid operation and maintenance, applicable to the UAV visual inspection and control method for power grid operation and maintenance as described in any one of claims 1-9, characterized in that, include: The map acquisition unit is configured to acquire a 3D point cloud map of the power grid equipment area; The center alignment unit is configured to control the visual sensor carried by the UAV to collect image information of the key equipment identification points, and adjust the flight attitude of the UAV according to the image information so that the center of the field of view of the visual sensor is aligned with the center position of the power grid equipment in the three-dimensional point cloud map, and the optical axis of the visual sensor is consistent with the normal direction of the power grid equipment, thereby obtaining the initial alignment attitude. The path planning unit is configured to determine a first offset distance based on the center position of the field of view of the visual sensor under the initial alignment posture and the center position of the power grid equipment in the three-dimensional point cloud map, and to determine a second offset distance based on the center point of the key detection area identified by the equipment outline edge information in the three-dimensional point cloud map and the center position of the power grid equipment, to calculate the deflection angle based on the ratio of the first offset distance to the second offset distance, and to plan the inspection flight path of the UAV to the preset hovering position of the tower to be inspected, starting from the initial alignment posture; The central detection unit is configured to control the UAV to fly along the inspection flight path to the preset hovering position, and control the inspection camera to collect the initial image of the target component of the tower when the gimbal is in the zero position. The unit extracts the pixel coordinates of the target component and the vertical pixel deviation value between the center of the image and the center of the image from the initial image. The target component is located in the key detection area. The gimbal adjustment unit is configured to substitute the vertical pixel deviation value into a pre-established pixel deviation-gimbal rotation angle lookup table, calculate the target rotation angle of the gimbal, and drive the gimbal to rotate around the deflection axis by the target rotation angle so that the target component is located in the center of the image in the subsequent image.

Images