A shooting control method and device of a gimbal camera carried by a distribution network inspection unmanned aerial vehicle

Abstract

The application relates to the technical field of gimbal camera shooting control, in particular to a shooting control method and device for a network distribution inspection unmanned aerial vehicle carrying a gimbal camera, which comprises the following steps: acquiring a current inspection picture collected by the gimbal camera; obtaining a main target boundary box, detecting a sub-target, and generating a small-scale level component priori; solving an expected optical zoom ratio of the gimbal camera and a gimbal angle correction amount, synchronously adjusting shooting parameters of the gimbal camera and an unmanned aerial vehicle attitude; acquiring the current inspection picture after the shooting parameters and the unmanned aerial vehicle attitude are adjusted, extracting an imaging evaluation index and an imaging quality comprehensive evaluation coefficient of the sub-target, performing closed-loop imaging feedback control on the gimbal camera, and acquiring qualified imaging; and performing hierarchical decision-making to complete network distribution inspection shooting control. The application can effectively reduce the missing shooting, missing inspection and rework shooting probability of small-scale network distribution components by performing shooting control on the unmanned aerial vehicle and the gimbal camera carried thereby, and significantly improves the network distribution inspection efficiency.

Description

Technical Field

[0001] This application relates to the field of gimbal camera shooting control technology, specifically to a shooting control method and device for a power distribution network inspection drone equipped with a gimbal camera. Background Technology

[0002] Existing power distribution network inspection drones typically carry a three-axis gimbal camera to autonomously inspect and photograph components such as towers, insulator strings, and conductors along a preset route, and perform identification and defect detection. Current shooting control methods usually involve: first identifying the target in the image, then controlling the gimbal rotation to center the target based on the deviation between the target and the image center before shooting; or adjusting shooting parameters based on image-level statistics; or using an inertial measurement unit and inter-image information to correct the gimbal attitude.

[0003] However, the objects to be inspected in power distribution network inspection scenarios often have obvious multi-level structures, such as the insulator string as a whole and its local pins, hardware, and other small-scale components. Existing single-level target detection methods are insufficient to provide accurate prior information for such small-scale components. Furthermore, existing gimbal angle and zoom control are disconnected from UAV attitude control, and shooting parameter adjustments are mostly performed in a one-time manner, lacking a closed-loop feedback correction mechanism based on image quality. In addition, existing shooting control and defect identification processes are independent of each other, failing to form a closed loop at the UAV's onboard end for initial screening, targeted reshooting, and on-demand data transmission.

[0004] The aforementioned defects mean that when faced with drone flight vibrations, gusts of wind, and zoom adjustments, small-scale components such as pins and fittings are easily deflected from the center of the image or even overflow the field of view, resulting in insufficient image coverage or blurriness. Due to the lack of quantitative judgment and closed-loop feedback on the shooting results, the actual inspection images often fail to meet the requirements for defect identification of small-scale components. This ultimately leads to missed shots, missed inspections, and reshoots of small-scale power distribution network components containing defects, severely reducing the efficiency of power distribution network inspections. Summary of the Invention

[0005] In view of the above, it is necessary to provide a shooting control method and device for a distribution network inspection drone equipped with a gimbal camera. By controlling the shooting of the drone and its onboard gimbal camera during the distribution network inspection process, the probability of missing or under-inspecting small-scale distribution network components and requiring rework shooting can be effectively reduced, significantly improving the efficiency of distribution network inspection.

[0006] In a first aspect, embodiments of this application provide a shooting control method for a distribution network inspection drone equipped with a gimbal camera, the method comprising the following steps:

[0007] Acquire the current inspection image captured by the PTZ camera and perform image enhancement processing on the current inspection image;

[0008] Multi-level component perception is performed on the current inspection screen after image enhancement. The main target is detected to obtain the main target bounding box. Sub-targets are detected within the main target bounding box, and a small-scale component prior is generated, which includes the sub-target bounding box, the sub-target center coordinates, and the sub-target imaging ratio.

[0009] Based on the prior knowledge of the small-scale components, the expected optical zoom ratio and gimbal angle correction of the gimbal camera are solved, and the shooting parameters of the gimbal camera and the attitude of the UAV are adjusted simultaneously.

[0010] The system acquires the current inspection image after the shooting parameters and the attitude adjustment of the UAV, extracts the imaging evaluation index of the sub-target, and then calculates the comprehensive evaluation coefficient of the imaging quality of the sub-target. This is used to perform closed-loop imaging feedback control on the gimbal camera in order to obtain qualified imaging.

[0011] Defects are identified in the qualified images to obtain the defect confidence level. Based on the defect confidence level, a graded decision is made to complete the distribution network inspection and shooting control.

[0012] Preferably, when the main target bounding box is not detected or no sub-target is detected within the main target bounding box, and the target is not in a directional reshooting state, a centering and zoom retraction minimum magnification reset command is sent to the gimbal camera, and the inspection image is reacquired.

[0013] Preferably, the desired optical zoom ratio is calculated based on the ratio between the sub-target imaging ratio and the preset desired sub-target image ratio, as well as the current optical zoom ratio, and the desired optical zoom ratio cannot exceed the preset maximum optical zoom ratio of the gimbal camera.

[0014] Preferably, the gimbal angle correction is calculated based on the pixel offset of the sub-target center coordinates relative to the image center coordinates. When the pixel offset is less than a preset offset threshold, the gimbal angle correction is set to zero.

[0015] Preferably, when the gimbal angle correction exceeds a preset gimbal movable angle threshold, the excess portion is subjected to rotational transformation between the UAV body coordinate system and the gimbal coordinate system to extract the yaw angle component and pitch angle component as the UAV attitude correction amount.

[0016] Preferably, the imaging evaluation index includes a sub-target clarity feature value, a sub-target proportion feature value, and a sub-target centering feature value. The sub-target clarity feature value is positively correlated with the clarity of the current inspection image. The sub-target proportion feature value is negatively correlated with the degree to which the sub-target imaging proportion in the current inspection image deviates from the preset expected sub-target image proportion. The sub-target centering feature value is positively correlated with the degree of centering of the sub-target in the current inspection image.

[0017] Preferably, the comprehensive evaluation coefficient of imaging quality is the weighted sum of the normalized value of the sub-target clarity feature value, the sub-target proportion feature value, and the sub-target centering feature value.

[0018] Preferably, the closed-loop imaging feedback control of the gimbal camera includes:

[0019] If the comprehensive evaluation coefficient of imaging quality is greater than or equal to the preset imaging quality compliance threshold, it is determined to be a compliant image.

[0020] Otherwise, the imaging quality closed-loop feedback adjustment is triggered, and the process of acquiring the inspection image and detecting compliance is re-executed, with a maximum number of feedback adjustments set.

[0021] Preferably, the step of making a graded decision based on the defect confidence level includes:

[0022] When the confidence level of the defect is greater than or equal to the preset high confidence threshold, the gimbal camera is triggered to perform directional reshoot and real-time transmission.

[0023] When the defect confidence level is greater than or equal to a preset low confidence threshold and less than a preset high confidence threshold, only the thumbnail and defect coordinates are returned.

[0024] When the confidence level of the defect is less than a preset low confidence threshold, the images are only cached on the air and uploaded to the ground receiving station in batches after the inspection mission is completed.

[0025] Secondly, embodiments of this application also provide a shooting control device for a distribution network inspection drone equipped with a gimbal camera, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the above-described shooting control methods for a distribution network inspection drone equipped with a gimbal camera.

[0026] This application has at least the following beneficial effects:

[0027] By acquiring the current inspection image captured by the gimbal camera and performing image enhancement processing, the impact of uneven brightness, decreased contrast, and loss of detail information under complex outdoor lighting conditions on subsequent target perception and defect identification can be suppressed.

[0028] Multi-level perception of components is performed on the current inspection screen after image enhancement. The main target bounding box is detected and sub-targets are detected within the main target bounding box. Small-scale component priors are generated. This can make full use of the multi-level structural features of distribution network components and detect small-scale sub-targets based on large-scale main targets. This effectively overcomes the limitation of single-level target detection in accurately perceiving small-scale components such as pins and hardware.

[0029] Based on the priors of small-scale components, the desired optical zoom ratio and gimbal angle correction of the gimbal camera are solved, and the shooting parameters of the gimbal camera and the attitude of the UAV are adjusted simultaneously. This enables the optical zoom control, gimbal angle control, and UAV attitude control to be executed in a coordinated manner, so that the small-scale components remain centered and the image is complete during the magnification process, effectively suppressing the sub-targets from deviating from the center of the image or even overflowing the field of view due to flight disturbances.

[0030] The system acquires the current inspection image after adjusting the shooting parameters, extracts the imaging evaluation index of the sub-target, and then calculates the comprehensive evaluation coefficient of the imaging quality of the sub-target. This enables closed-loop imaging feedback control of the gimbal camera, avoiding the uncertainty of imaging quality caused by one-time parameter adjustment, and ensuring that the imaging of small-scale components simultaneously meets the requirements for defect identification in terms of clarity and integrity.

[0031] By identifying defects in compliant images to obtain defect confidence levels, and then making graded decisions, the imaging control and defect identification process can be deeply integrated at the airborne end, forming a closed-loop collaboration between imaging and detection. This effectively reduces the probability of missed images, missed detections, and rework, and significantly improves the efficiency of power distribution network inspection. Attached Figure Description

[0032] Figure 1 A flowchart illustrating the steps of a shooting control method for a power distribution network inspection drone equipped with a gimbal camera, provided in one embodiment of this application;

[0033] Figure 2 This is a schematic diagram illustrating the process of obtaining the comprehensive evaluation coefficient for image quality. Detailed Implementation

[0034] The following, in conjunction with the accompanying drawings, details the specific scheme of the shooting control method and device for the distribution network inspection drone equipped with a gimbal camera provided in this application.

[0035] In this application, the power distribution network inspection drone is equipped with a three-axis gimbal camera, which is mounted on the belly of the drone as an airborne payload via a gimbal.

[0036] Please see Figure 1 The diagram illustrates a step-by-step flowchart of a shooting control method for a distribution network inspection drone equipped with a gimbal camera, according to an embodiment of this application. The method includes the following steps:

[0037] Step 1: Acquire the current inspection image captured by the PTZ camera and perform image enhancement processing.

[0038] Since power distribution network inspections are usually carried out in complex outdoor environments, the inspection footage may have complex lighting conditions such as backlight, shadows, fog, and low light. This can cause uneven brightness, reduced contrast, and loss of detail in the images captured by the PTZ camera, which in turn affects the accuracy of subsequent target perception and defect identification.

[0039] Therefore, in order to suppress the interference of complex lighting environment on the sensing process, image enhancement processing is performed on the current inspection image output by the gimbal camera.

[0040] In this embodiment, a histogram equalization algorithm is used to enhance the image of the current inspection screen. The histogram equalization algorithm is a well-known technology and will not be described in detail in this application. As other implementation methods, based on the ability to enhance the image of the current inspection screen, the implementer may choose other existing feasible technologies, such as the Retinex image enhancement algorithm, etc. This application does not impose any special limitations on this.

[0041] Step 2: Perform multi-level component perception on the current inspection screen after image enhancement, detect the main target to obtain the main target bounding box, detect sub-targets within the main target bounding box, and generate a small-scale component prior that includes the sub-target bounding box, the sub-target center coordinates, and the sub-target imaging proportion.

[0042] Because the main targets in the inspection images, such as the tower body, the insulator string as a whole, and the conductor section, are large in scale and structurally stable, they usually occupy the main part of the image and their bounding box features are clearly identifiable. On the other hand, the smaller-scale components such as pins, hardware, and hanging points in the distribution network are smaller in scale and occupy a smaller proportion of the image. They usually do not have the obvious bounding box features of the main target in the whole image, and the sub-targets are usually located within the main target area.

[0043] Therefore, in order to better perceive the imaging status of small-scale components such as pins, hardware, and mounting points in the current inspection screen, the following processing is performed:

[0044] (1) Use an image target detection algorithm to detect the main target in the current inspection screen after image enhancement, and obtain the main target bounding box in the current inspection screen of the gimbal camera.

[0045] Specifically, it determines whether the current state is in directional reshoot mode. If it is not in directional reshoot mode and the main target bounding box has not been detected, a centering and zoom-to-minimum-magnification reset command is sent to the gimbal camera to prevent the gimbal camera from continuing to shoot at high magnification or in blind spots where the target has been lost. The process then returns to step one to reacquire the image and execute the above steps again, i.e., reacquire the inspection image and perform image enhancement processing and main target bounding box detection. Further, if the current state is in directional reshoot mode and the main target bounding box has not been detected, the entire current inspection image area is temporarily designated as the main target bounding box, and the process directly proceeds to subsequent sub-target detection.

[0046] In this embodiment, the YOLOv8 algorithm is used to detect the main target in the current inspection image after image enhancement. The YOLOv8 algorithm is a well-known technology and will not be described in detail in this application. As other implementation methods, based on the ability to detect the main target in the current inspection image after image enhancement, the implementer may choose other existing feasible technologies, such as the YOLO series algorithms, Faster R-CNN algorithm, SSD algorithm, etc. This application does not make any special limitations on this.

[0047] (2) Use an image target detection algorithm to perform sub-target detection on the image within the main target bounding box in the current inspection screen of the gimbal camera, and obtain the sub-target bounding box in the current inspection screen of the gimbal camera; record the center point coordinates of the image within the sub-target bounding box as the sub-target center coordinates in the current inspection screen of the gimbal camera, which is used to characterize the pixel coordinates of the center of the sub-target in the gimbal camera screen; record the ratio of the area of ​​the sub-target bounding box to the area of ​​the current inspection screen as the sub-target imaging proportion in the current inspection screen of the gimbal camera, which is used to characterize the imaging scale proportion of the sub-target in the gimbal camera screen. The smaller the calculated target imaging proportion, the lower the imaging scale proportion of the sub-target in the gimbal camera screen, and the more necessary it is to increase the optical zoom ratio of the gimbal camera to enlarge the sub-target imaging scale in the output screen of the gimbal camera.

[0048] Specifically, if no sub-target is detected within the main target bounding box and the camera is not in a directional reshooting state, a centering and zoom retraction minimum magnification reset command is sent to the gimbal camera, and the camera returns to step one to reacquire the image and execute the above steps, namely, reacquire the inspection image and perform image enhancement processing, main target bounding box detection, sub-target bounding box detection, sub-target center coordinate acquisition, and sub-target imaging ratio calculation.

[0049] Step 3: Based on the prior knowledge of the small-scale components, solve for the expected optical zoom ratio and gimbal angle correction of the gimbal camera, and simultaneously adjust the shooting parameters of the gimbal camera and the attitude of the UAV.

[0050] In general, during drone inspection flights, there are often flight disturbances such as gusts, heading drift, and flight shaking. These disturbances cause instability in the line of sight of the gimbal camera on the drone, resulting in pixel deviation between the line of sight and the center of the sub-target. The larger the pixel deviation, the further the sub-target is from the center of the gimbal camera's image. The more likely the sub-target is to overflow the field of view or move out of the frame when the gimbal camera is magnified by optical zoom. At the same time, the greater the deviation of the sub-target's image proportion from the preset expected image proportion, the more likely the sub-target will be too large, resulting in incomplete imaging, or too small, resulting in unclear imaging. This makes it difficult for the sub-target imaging to simultaneously meet the requirements of image clarity and completeness for defect identification.

[0051] Therefore, in order to prevent the inspection screen from overflowing and to improve the clarity and integrity of sub-target imaging when the UAV's flight disturbances and the optical zoom of the gimbal camera change, the following processing is performed:

[0052] (1) Based on the sub-target imaging ratio of the current inspection image of the gimbal camera, calculate the expected optical zoom ratio of the current inspection image of the gimbal camera, which will be used as the optical zoom ratio for subsequent shooting by the gimbal camera. Adjust the optical zoom ratio used in the current inspection image of the gimbal camera according to the expected optical zoom ratio. The calculation formula is as follows:

[0053] In the formula, Minimum value operation; This indicates the preset maximum optical zoom ratio of the gimbal camera, which can be set by the user according to the camera model. This represents the floor function; This indicates the optical zoom level used in the current inspection view of the gimbal camera; This indicates the preset desired image proportion of sub-targets for the gimbal camera, which can be set by the implementer according to the requirements for the clarity and completeness of the sub-target defect identification imaging. In this embodiment... The value is 0.2; This indicates the operation of retrieving the maximum value; This indicates a preset minimum positive number, used to avoid a denominator of 0; This indicates the percentage of sub-target images captured in the current inspection view of the gimbal camera.

[0054] Since the actual imaging area resulting from optical zoom typically changes according to the square of the zoom ratio, it is necessary to... Square root processing is implemented to prevent excessive focus adjustment from causing sub-targets to overflow the field of view.

[0055] In this embodiment, The value is 0.001, and the implementer can set it according to the actual situation. The value of is not specifically limited in this application.

[0056] (2) Based on the center coordinates (x, y) of the sub-target in the current inspection screen of the PTZ camera, and the center point coordinates of the current inspection screen. Calculate the sub-target pixel offset (dx, dy) in the current inspection image of the gimbal camera, where the horizontal offset is... Vertical offset dx and dy are used to characterize the number of pixels in the current inspection image of the gimbal camera that are deviated from the center of the image. The larger the calculated dx and dy are, the farther the horizontal and vertical distances of the sub-targets are from the center of the image, and the more the correction amount of the gimbal angle needs to be increased. The gimbal angle includes the horizontal angle and the vertical angle.

[0057] In this embodiment, the image coordinate system in the output screen of the gimbal camera has the lower left corner as the origin, the horizontal rightward direction as the positive X-axis, and the vertical upward direction as the positive Y-axis.

[0058] Based on the sub-target pixel offset (dx, dy), calculate the gimbal angle correction amount of the current inspection image of the gimbal camera. This is used to adjust the gimbal angle during subsequent shooting by the gimbal camera. This indicates the amount of horizontal angle correction. This represents the vertical angle correction amount; and based on the gimbal angle correction amount, the current gimbal angle of the camera is adjusted; the formula for calculating the gimbal angle correction amount is as follows:

[0059] ; In the formula, dx and dy represent the horizontal and vertical offsets of the sub-target pixel offsets in the current inspection image of the gimbal camera, respectively. , These represent the horizontal and vertical angles corresponding to each pixel, respectively. They can be set according to the camera parameters used in the current inspection screen of the PTZ camera. The camera parameters include the horizontal field of view, screen width, vertical field of view, and screen height. , These represent the preset horizontal offset threshold and the preset vertical offset threshold, respectively. This indicates the absolute value operation.

[0060] In this embodiment, the preset horizontal offset threshold is 3% of the width of the current inspection screen, and the preset vertical offset threshold is 3% of the height of the current inspection screen. The implementer can set the specific values ​​of the preset horizontal offset threshold and the preset vertical offset threshold according to the actual situation. This application does not impose any special restrictions on this.

[0061] (3) When the gimbal angle correction exceeds the preset gimbal movable angle threshold, the angle difference between the horizontal angle correction and the vertical angle correction exceeding the preset gimbal movable angle threshold is obtained respectively. At the same time, the rotation transformation matrix between the UAV body coordinate system and the gimbal coordinate system is obtained. Based on the rotation transformation matrix, the obtained angle difference is transformed from the gimbal coordinate system to the UAV body coordinate system. The yaw angle component and pitch angle component are extracted from the angle difference component transformed to the UAV body coordinate system. The obtained yaw angle component and pitch angle component are used as the target offset of the yaw angle and pitch angle of the UAV attitude correction. Based on the target offset of the yaw angle and pitch angle, the attitude angle tracking control is performed through the flight control system of the UAV to adjust the flight attitude of the UAV to cooperate with the gimbal angle correction, so as to ensure that the sub-target does not overflow the inspection screen under the condition of UAV flight disturbance and gimbal camera optical zoom magnification.

[0062] In this embodiment, the preset gimbal movable angle threshold is set according to the maximum movable angle hardware parameters provided by the gimbal equipment manufacturer.

[0063] Step 4: Obtain the shooting parameters and the current inspection image after the UAV attitude adjustment, extract the imaging evaluation index of the sub-target, and then calculate the comprehensive evaluation coefficient of the imaging quality of the sub-target. This is used to perform closed-loop imaging feedback control on the gimbal camera to obtain qualified imaging.

[0064] After adjusting the shooting control of the gimbal camera, the imaging quality of sub-targets such as pins, hardware, and hanging points in the distribution network components is usually affected by three factors in the inspection screen: image sharpness, target centering, and target imaging proportion. If any of these factors fails to meet the standards, the sub-target imaging may not meet the defect identification requirements. Therefore, by quantitatively evaluating the imaging quality of sub-targets in the current inspection screen of the gimbal camera, feedback adjustment of the gimbal camera's shooting control is made.

[0065] (1) After adjusting the shooting parameters of the gimbal camera and the attitude of the UAV, the current inspection image of the gimbal camera after adjusting the shooting parameters and the attitude of the UAV is obtained. After calling the multi-level perception detection of the component to generate the prior of small-scale components in the new image, the imaging evaluation index of the sub-target is extracted. The imaging evaluation index includes the sub-target clarity feature value, the sub-target proportion feature value and the sub-target centering feature value. The specific calculation process is as follows:

[0066] 1) The clarity feature value of the sub-target is positively correlated with the clarity of the current inspection image.

[0067] It should be noted that positive correlation refers to variables changing in the same direction, either increasing or decreasing simultaneously.

[0068] In this embodiment, the Laplacian operator is used to calculate the image sharpness within the bounding box of the sub-target in the current inspection image, and this is used as the sub-target sharpness feature value of the current inspection image of the gimbal camera. This feature value is used to characterize the image sharpness of the sub-target in the current inspection image of the gimbal camera. The larger the calculated sub-target sharpness feature value, the sharper the image of the sub-target. The Laplacian operator is a known technology, and the calculation process will not be described in detail in this application.

[0069] 2) The sub-target proportion feature value is negatively correlated with the degree to which the sub-target imaging proportion in the current inspection image deviates from the preset expected sub-target image proportion.

[0070] It should be noted that negative correlation refers to variables changing in opposite directions; if one increases, the other decreases.

[0071] In this embodiment, the absolute difference between the sub-target imaging ratio in the current inspection image of the gimbal camera and the preset expected sub-target imaging ratio is calculated. The difference between 1 and the absolute difference is used as the sub-target imaging ratio feature value in the current inspection image of the gimbal camera. This feature value is used to characterize the degree of closeness between the sub-target imaging ratio in the current inspection image of the gimbal camera and the preset expected sub-target imaging ratio. The larger the calculated sub-target imaging ratio feature value, the closer the sub-target imaging ratio is to its expected imaging ratio.

[0072] 3) The centering feature value of the sub-target is positively correlated with the centering degree of the sub-target in the current inspection screen.

[0073] In this embodiment, the coordinates of the center point of the current inspection image of the gimbal camera are obtained, the Euclidean distance between the center point coordinates and the center coordinates of the sub-targets in the current inspection image of the gimbal camera is calculated, and the ratio of the Euclidean distance to the number of pixels on the diagonal in the output image of the gimbal camera is calculated. The difference between 1 and the ratio is used as the centering feature value of the sub-targets in the current inspection image of the gimbal camera, which is used to characterize the centering degree of the sub-targets in the current inspection image of the gimbal camera. The larger the calculated centering feature value of the sub-targets, the closer the sub-targets are to the center of the image.

[0074] (2) Normalize the sub-target clarity feature values ​​of the current inspection screen of the PTZ camera so as to map the sub-target clarity feature values ​​to the same value range as the sub-target proportion feature value and the sub-target centering feature value.

[0075] In this embodiment, the Min-Max normalization method is used to normalize the sharpness feature values ​​of the sub-targets. The process of obtaining the maximum and minimum values ​​is as follows: images of the same type of gimbal camera in a focused state are pre-acquired, and the maximum value among the sharpness feature values ​​of the sub-targets in all images is taken as the maximum value during normalization; images of the same type of gimbal camera in a defocused state are pre-acquired, and the minimum value among the sharpness feature values ​​of the sub-targets in all images is taken as the minimum value during normalization. The Min-Max normalization method is a known technology, and the calculation process will not be described in detail in this application.

[0076] Furthermore, the comprehensive evaluation coefficient of the imaging quality of the sub-targets in the current inspection image of the gimbal camera is calculated using the following formula:

[0077] In the formula, Q represents the comprehensive evaluation coefficient of the imaging quality of the sub-target in the current inspection image of the gimbal camera; denoted by , D, representing the normalized value of the image clarity of the sub-target in the current inspection image of the gimbal camera; denoted by , D, representing the sub-target proportion feature value in the current inspection image of the gimbal camera; denoted by , R, representing the sub-target centering feature value in the current inspection image of the gimbal camera. , , Both represent preset weighting coefficients greater than 0, and . Figure 2 This is a schematic diagram illustrating the process of obtaining the comprehensive evaluation coefficient for image quality.

[0078] In this embodiment, since the clarity of the sub-target image, the proportion of the sub-target image, and the centering of the sub-target all have the same impact on the sub-target imaging quality, therefore... , , All set to Implementers can set it themselves according to the actual situation. , , The specific value of is not specifically limited in this application.

[0079] (3) Compare the comprehensive evaluation coefficient of imaging quality with the preset imaging quality threshold. When the comprehensive evaluation coefficient of imaging quality is greater than or equal to the preset imaging quality threshold, it is determined that the imaging quality of the current inspection screen of the gimbal camera meets the standard. When the comprehensive evaluation coefficient of imaging quality is less than the preset imaging quality threshold, return directly to step one to reacquire the inspection screen and execute the above steps, that is, reacquire the inspection screen and perform image enhancement processing, main target bounding box detection, sub-target bounding box detection, sub-target center coordinate acquisition, sub-target imaging ratio calculation, and imaging quality comprehensive evaluation coefficient calculation to form imaging quality closed-loop feedback adjustment, and set the maximum number of feedback adjustments.

[0080] Specifically, when the comprehensive evaluation coefficient of imaging quality is less than the preset imaging quality threshold and the maximum number of feedback adjustments is reached, the imaging quality adjustment of the current inspection image of the gimbal camera is determined to have failed, and subsequent defect identification processing is directly performed on the current inspection image of the gimbal camera.

[0081] In this embodiment, the preset imaging quality threshold is set to 0.6. The implementer can set the required imaging clarity and integrity based on the imaging images of small-scale components such as pins, hardware, and hanging points in the power distribution network.

[0082] In this embodiment, the maximum number of feedback adjustments is 3. The maximum number of feedback adjustments can be set by the implementer according to the inspection time, and this application does not impose any special limitations on this.

[0083] Step 5: Perform defect identification on the qualified images to obtain the defect confidence level, and make a graded decision based on the defect confidence level to complete the distribution network inspection and shooting control.

[0084] (1) Because real defects in distribution network components, such as loose pins, spontaneous explosion of insulators, hanging foreign objects, and damage, usually have characteristics such as clear edge structure, obvious gray difference with the surrounding background, and relatively concentrated morphological distribution, while normal areas and pseudo defects in normal distribution network components usually do not have the above-mentioned characteristics due to the randomness of lighting and viewing angle. Among them, pseudo defects mainly include image misjudgment caused by shadows, reflections, or occlusion.

[0085] Based on the above analysis, a defect detection algorithm is used to identify defects in the compliant images, and the defect category and defect confidence score are output. The defect confidence score is used to characterize the probability that the compliant images contain defects. The higher the calculated defect confidence score, the greater the probability that the compliant images contain defects.

[0086] In this embodiment, the defect detection algorithm is specifically the YOLOv8 algorithm. As other implementation methods, based on the ability to identify defects in compliant images, the implementer may choose other existing feasible technologies, such as the YOLO series algorithms, PaDim algorithm, SimpleNet algorithm, etc. This application does not impose any special limitations on this.

[0087] (2) The defect confidence level is compared with the preset defect classification threshold to make a classification decision for the gimbal camera, as follows:

[0088] The preset defect classification thresholds include a preset high confidence threshold and a preset low confidence threshold, and the preset low confidence threshold is less than the preset high confidence threshold;

[0089] When the confidence level of the defect is greater than or equal to the preset high confidence threshold, it is determined that the current inspection screen of the PTZ camera contains a defect, triggering the PTZ camera to take directional reshoots and transmit them back in real time, and generating real-time alarm information containing defect coordinates, defect category and image evidence and transmitting it to the ground receiving station.

[0090] When the defect confidence level is greater than or equal to the preset low confidence threshold and less than the preset high confidence threshold, the defect is judged to be of medium suspicion, and only the thumbnail and defect coordinates are returned.

[0091] When the defect confidence level is less than the preset low confidence threshold, it is determined that the current inspection image of the gimbal camera does not contain a defect. The image is only used for onboard local caching and is then uploaded to the ground receiving station in batches with low priority after the UAV inspection mission is completed.

[0092] In this embodiment, the preset high confidence threshold and the preset low confidence threshold are 0.85 and 0.30, respectively, and the values ​​of the preset high confidence threshold and the preset low confidence threshold are calculated from experimental data.

[0093] (3) When the gimbal camera is triggered to shoot in a directional manner, the preset sub-target expected image ratio used in the original calculation of the expected optical zoom ratio is replaced with the preset fine shooting expected value. The preset fine shooting expected value needs to be set to a value greater than the preset sub-target expected image ratio in order to obtain a higher resolution and more detailed image.

[0094] Using the replaced preset high-precision shooting expectation value, a larger expected optical zoom ratio is recalculated directly according to the calculation logic in step three, and the gimbal camera is controlled to perform the magnification operation.

[0095] After the gimbal camera completes the magnification operation, control the gimbal camera to capture a new image, and execute steps one through four sequentially on the new image. During this process, if step four successfully outputs a qualified image, or if step four forces the output of a substandard image due to the closed-loop feedback adjustment reaching its upper limit, the output image will be re-evaluated for defects according to step five.

[0096] This application pre-sets an upper limit on the number of targeted reshoots. When the cumulative number of targeted reshoots reaches the upper limit, the targeted reshoot process is forcibly exited. No further parameter modifications or reshoots are performed. The result of the last defect determination is taken as the final conclusion, and corresponding alarms or cache upload operations are executed.

[0097] In this embodiment, the preset expected value of precision shooting is 0.4. The implementer can set its specific value according to the actual situation. This application does not make any special limitation on this.

[0098] In this embodiment, the preset upper limit for targeted reshoots is 2 times. The implementer can set its specific value according to the actual situation. This application does not impose any special restrictions on this.

[0099] Based on the same inventive concept as the above methods, this application also provides a shooting control device for a distribution network inspection drone equipped with a gimbal camera, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described shooting control methods for a distribution network inspection drone equipped with a gimbal camera.

[0100] In summary, this application acquires the current inspection image captured by the gimbal camera and performs image enhancement processing, which can suppress the impact of uneven brightness, decreased contrast and loss of detail information on subsequent target perception and defect recognition under complex outdoor lighting conditions.

[0101] Multi-level perception of components is performed on the current inspection screen after image enhancement. The main target bounding box is detected and sub-targets are detected within the main target bounding box. Small-scale component priors are generated. This can make full use of the multi-level structural features of distribution network components and detect small-scale sub-targets based on large-scale main targets. This effectively overcomes the limitation of single-level target detection in accurately perceiving small-scale components such as pins and hardware.

[0102] Based on the priors of small-scale components, the desired optical zoom ratio and gimbal angle correction of the gimbal camera are solved, and the shooting parameters of the gimbal camera and the attitude of the UAV are adjusted simultaneously. This enables the optical zoom control, gimbal angle control, and UAV attitude control to be executed in a coordinated manner, so that the small-scale components remain centered and the image is complete during the magnification process, effectively suppressing the sub-targets from deviating from the center of the image or even overflowing the field of view due to flight disturbances.

[0103] The system acquires the current inspection image after adjusting the shooting parameters, extracts the imaging evaluation index of the sub-target, and then calculates the comprehensive evaluation coefficient of the imaging quality of the sub-target. This enables closed-loop imaging feedback control of the gimbal camera, avoiding the uncertainty of imaging quality caused by one-time parameter adjustment, and ensuring that the imaging of small-scale components simultaneously meets the requirements for defect identification in terms of clarity and integrity.

[0104] By identifying defects in compliant images to obtain defect confidence levels, and then making graded decisions, the imaging control and defect identification process can be deeply integrated at the airborne end, forming a closed-loop collaboration between imaging and detection. This effectively reduces the probability of missed images, missed detections, and rework, and significantly improves the efficiency of power distribution network inspection.

Claims

1. A shooting control method for a power distribution network inspection drone equipped with a gimbal camera, characterized in that, The method includes the following steps: Acquire the current inspection footage captured by the PTZ camera and perform image enhancement processing; Multi-level component perception is performed on the current inspection screen after image enhancement. The main target is detected to obtain the main target bounding box. Sub-targets are detected within the main target bounding box, and a small-scale component prior is generated, which includes the sub-target bounding box, the sub-target center coordinates, and the sub-target imaging ratio. Based on the prior knowledge of the small-scale components, the expected optical zoom ratio and gimbal angle correction of the gimbal camera are solved, and the shooting parameters of the gimbal camera and the attitude of the UAV are adjusted simultaneously. The system acquires the current inspection image after the shooting parameters and the attitude adjustment of the UAV, extracts the imaging evaluation index of the sub-target, and then calculates the comprehensive evaluation coefficient of the imaging quality of the sub-target. This is used to perform closed-loop imaging feedback control on the gimbal camera in order to obtain qualified imaging. Defects are identified in the qualified images to obtain the defect confidence level. Based on the defect confidence level, a graded decision is made to complete the distribution network inspection and shooting control.

2. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, When the main target bounding box is not detected or no sub-target is detected within the main target bounding box, and the device is not in a directional reshooting state, a centering and zoom retraction minimum magnification reset command is sent to the gimbal camera, and the inspection image is reacquired.

3. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, The desired optical zoom ratio is calculated based on the ratio between the sub-target imaging ratio and the preset desired sub-target image ratio, as well as the current optical zoom ratio. The desired optical zoom ratio cannot exceed the preset maximum optical zoom ratio of the gimbal camera.

4. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, The gimbal angle correction is calculated based on the pixel offset of the sub-target center coordinates relative to the image center coordinates. When the pixel offset is less than a preset offset threshold, the gimbal angle correction is set to zero.

5. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 4, characterized in that, When the gimbal angle correction exceeds the preset gimbal movable angle threshold, the excess portion is transformed between the UAV body coordinate system and the gimbal coordinate system to extract the yaw angle component and pitch angle component as the UAV attitude correction amount.

6. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, The imaging evaluation indicators include sub-target clarity feature value, sub-target proportion feature value, and sub-target centering feature value. The sub-target clarity feature value is positively correlated with the clarity of the current inspection image. The sub-target proportion feature value is negatively correlated with the degree to which the sub-target imaging proportion in the current inspection image deviates from the preset expected sub-target image proportion. The sub-target centering feature value is positively correlated with the degree of centering of the sub-target in the current inspection image.

7. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 6, characterized in that, The comprehensive evaluation coefficient for imaging quality is the weighted sum of the normalized value of the sub-target clarity feature value, the sub-target proportion feature value, and the sub-target centering feature value.

8. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, The closed-loop imaging feedback control of the gimbal camera includes: If the comprehensive evaluation coefficient of imaging quality is greater than or equal to the preset imaging quality compliance threshold, it is determined to be a compliant image. Otherwise, the imaging quality closed-loop feedback adjustment is triggered, and the process of acquiring the inspection image and detecting compliance is re-executed, with a maximum number of feedback adjustments set.

9. The shooting control method for a power distribution network inspection drone equipped with a gimbal camera as described in claim 1, characterized in that, The step of making a graded decision based on the defect confidence level includes: When the confidence level of the defect is greater than or equal to the preset high confidence threshold, the gimbal camera is triggered to perform directional reshoot and real-time transmission. When the defect confidence level is greater than or equal to a preset low confidence threshold and less than a preset high confidence threshold, only the thumbnail and defect coordinates are returned. When the confidence level of the defect is less than a preset low confidence threshold, the images are only cached on the air and uploaded to the ground receiving station in batches after the inspection mission is completed.

10. A shooting control device for a power distribution network inspection drone equipped with a gimbal camera, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the shooting control method of the distribution network inspection drone equipped with a gimbal camera as described in any one of claims 1-9.

Images