Unmanned aerial vehicle inspection image acquisition and automatic optimization method and system

Abstract

The invention discloses an unmanned aerial vehicle inspection image acquisition and automatic optimization method and system, and relates to the technical field of unmanned aerial vehicle inspection image automatic optimization, and the method comprises the steps: carrying out the optical correction of an image collected by an unmanned aerial vehicle through a local gray histogram equalization algorithm based on gamma transformation; determining the position and size of the target in the image according to a target detection algorithm; and adjusting the attitude and focal length of the holder according to the position and size of the target. According to the method disclosed by the invention, the problem of overexposure or underexposure caused by the change of illumination conditions is solved through a local gray histogram equalization algorithm based on gamma transformation; through a target detection algorithm, the key target of the power line in the image can be accurately identified, and the attitude of the holder is adjusted according to the position and size of the target, so that the accuracy of defect identification is improved; through a size estimation algorithm and a reverse consistency check mechanism, the focal length of the holder can be automatically adjusted according to the change of the relative distance between the target and the unmanned aerial vehicle, and the image definition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic optimization of drone inspection images, and specifically to a method and system for drone inspection image acquisition and automatic optimization. Background Art

[0002] With the continuous expansion of the scale of China's power system, the inspection work of transmission lines has become increasingly important. Traditional inspection of transmission lines mainly relies on manual labor, which has problems such as low efficiency, high labor intensity, and high risk coefficient. In recent years, the development of drone technology has provided a new solution for the inspection of transmission lines. Drone inspection has advantages such as high efficiency, low cost, and strong safety, and has become an important means for the inspection of transmission lines.

[0003] However, the existing drone inspection technology still has some deficiencies. First, due to changes in lighting conditions, the images captured by the drone may be overexposed or underexposed, affecting the image quality and being unfavorable for subsequent defect identification and analysis. Second, most of the existing drone inspection systems adopt preset flight paths and shooting angles, and cannot be dynamically adjusted according to the position and size of the target, resulting in possible target deviation or incomplete shooting of the captured images, affecting the accuracy of defect identification. In addition, the existing drone inspection systems lack an automatic zoom function and cannot adjust the focal length according to the change in the relative distance between the target and the drone, resulting in unstable target size in the image and affecting the image clarity. Therefore, how to improve the quality of drone inspection images and achieve automatic target tracking and analysis is an urgent problem to be solved in the current technical field of transmission line inspection. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed.

[0005] Therefore, the technical problems solved by the present invention are: the existing drone inspection image acquisition methods have problems such as unstable image quality, inaccurate target tracking, and non-automatic focal length adjustment, and how to achieve the goal of drone inspection image acquisition and automatic optimization.

[0006] To solve the above technical problems, the present invention provides the following technical solution: A method for drone inspection image acquisition and automatic optimization, including optically correcting the images captured by the drone by using a local gray histogram equalization algorithm based on gamma transformation; identifying the key targets of the power line in the image according to a target detection algorithm, and determining the position and size of the target in the image; calculating the horizontal and vertical rotation angles of the pan-tilt according to the position and size of the target, adjusting the pan-tilt attitude, and calculating the adjustment amount of the pan-tilt focal length, and adjusting the pan-tilt focal length to an appropriate ratio.

[0007] As a preferred solution of the drone inspection image acquisition and automatic optimization method described in the present invention, wherein: the optical correction of the images collected by the drone includes converting the images collected by the drone into grayscale images and calculating the average grayscale value of the grayscale images; assuming that the resolution of the drone inspection images is m×n, and converting the images from the three-color space to the grayscale space, then the average grayscale value is expressed as:

[0008]

[0009] wherein, f(x,y) represents the grayscale value at the image pixel point (x,y); determining the gamma coefficient according to the average grayscale value and the gamma value correspondence table, performing gamma transformation on the grayscale image, and performing optical correction on the image; performing local histogram equalization processing on the image after gamma transformation to further improve the image quality.

[0010] As a preferred solution of the drone inspection image acquisition and automatic optimization method described in the present invention, wherein: determining the position and size of the target in the image includes using a pre-trained deep learning model to perform target detection on the image and identifying the key targets of the power line in the image; determining the position and size of the target in the image according to the output result of the target detection; the position and size include the center point coordinates of the target and the length and width of the target area.

[0011] As a preferred solution of the drone inspection image acquisition and automatic optimization method described in the present invention, wherein: adjusting the pan-tilt focal length to an appropriate ratio includes calculating the horizontal and vertical rotation angles of the pan-tilt according to the difference between the target center point coordinates and the image center point coordinates; calculating the adjustment amount of the pan-tilt focal length according to the ratio of the length and width of the target area to the image size; controlling the pan-tilt to rotate horizontally and vertically to make the target located at the center of the image; controlling the pan-tilt to adjust the focal length to make the target occupy an appropriate ratio in the image.

[0012] As a preferred solution of the drone inspection image acquisition and automatic optimization method described in the present invention, wherein: calculating the horizontal and vertical rotation angles of the pan-tilt includes setting the horizontal and vertical threshold ranges of the target center point coordinates and the image center point coordinates; the shooting field of view of the pan-tilt camera is m×n, the center point coordinates (x p ,y p ) of the target detected by the target detection algorithm, the detected window size P L ×P W , and the range of the threshold area S The center point coordinates (x s ,y s ) of the S area;

[0013] The center point coordinates of the S area are also the center point coordinates of the image; according to the horizontal and vertical differences between the target center point coordinates and the image center point coordinates, it is determined whether the target center point exceeds the threshold range. The judgment of the target center point threshold range is expressed as:

[0014]

[0015] According to the horizontal and vertical differences between the target center point coordinates and the image center point coordinates and the preset pan-tilt rotation speed, calculate the horizontal and vertical rotation angles of the pan-tilt; in the horizontal direction, if then the pan-tilt camera turns to the right, if then the pan-tilt camera turns to the left; in the vertical direction, if then the pan-tilt camera turns up, if then the pan-tilt camera turns down.

[0016] As a preferred solution of the drone inspection image acquisition and automatic optimization method described in the present invention, wherein: the calculation of the adjustment amount of the pan-tilt focal length includes that during the tracking process, the size of the target in each frame of the image changes. Analyze and estimate according to the size ratio of the target, and calculate and correct through the size estimation algorithm to realize the zoom adjustment parameters of the pan-tilt; estimate the proportion of the target in the picture through the size estimation algorithm. The target is represented by an elliptical area in the image, and use ξ(x i ,y i ,h) to represent the elliptical equation of the target area. The equation is expressed as:

[0017]

[0018] Among them, ξ(x i ,y i ,h) represents the target area, x i represents the pixel coordinates in the sample box, y represents the center position of the target area, h represents the scale parameter of the target candidate area, and a and b respectively represent the long semi-axis and short semi-axis parameters of the ellipse; then the first-frame target model is expressed as:

[0019]

[0020] Among them, C represents the normalization constant, {x i} i＝1,...n represents the pixel positions in the sample box, represents the indicator function used to count the pixel information in the target area. If the pixel point x i belongs to the u-th color feature, then the pixel point x i value is 1, otherwise it is 0; the current-frame target candidate is expressed as:

[0021]

[0022] Among them, y = (y 1 -y 2 ) T represents the center position of the target candidate in the current frame; the candidate target region is defined by Riemann integral and calculated through multiple iterations using the Bhattacharyya coefficient. Starting from the current position along direction, iteratively move to obtain a new position y1 and a new target scale parameter h1; through the probability density along the gradient method, obtain the updated position y1 and the scale parameter h1 of the target, expressed as:

[0023]

[0024] Among them, y1 represents the target position after iterative update, h1 represents the scale parameter after iterative update, m k represents the displacement vector, w i represents the weight coefficient, g(·) represents the contour function, and G represents the kernel function; introduce a correction and detection mechanism. Considering the situation where the estimated scale gradually becomes larger, add a regularization term rs(y, h) for reverse change; considering that the target image has a small size and small scale change relative to the ground view, while the scale of self-similar objects is uncertain, resulting in an underestimation of the scale, force the search window to include a part of the background pixels to expand the sample frame scale, and add a regularization term rb(y, h); obtain the final scale update formula, expressed as:

[0025]

[0026] The final scale result is used for the scale change of the target during the inspection process.

[0027] As a preferred solution of the method for collecting and automatically optimizing UAV inspection images according to the present invention, wherein: the calculation of the adjustment amount of the pan-tilt focal length further includes performing scale update based on the scale change evaluation standard when there is no scale change or the tracking frame changes greatly and contains obvious background clutter; the scale change evaluation standard includes obtaining the position y t-1 , S t-1 ) of the current frame from the position and scale information (y t and the estimated scale parameter h t of the previous frame, performing reverse tracking to obtain the estimated position y black and the estimated scale parameter h black of the previous frame, verifying the consistency of h black and h t , and determining the final estimated scale according to different situations of the consistency coefficient; the consistency verification coefficient is expressed as:

[0028] θ c = |log2(ht ·h black )

[0029] Among them, θ c The threshold of is set to 0.1; if it is detected that the front and rear scale estimation ratios are the same, that is, θ c < 0.1, then the scale is updated according to If it is detected that the front and rear scale estimation ratios are different, that is, θ c < 0.1, then the tracking target scale combines the size of the previous frame, the new estimated size, and the size of the target at the first frame in a weighted manner, expressed as:

[0030]

[0031] Among them, β and σ respectively represent the parameters obtained by testing on a subset of the test sequence; as the target moves relative to the pan-tilt camera, considering the loss of target detection caused by the movement in the depth direction, a threshold is set for the focal length, expressed as:

[0032]

[0033] If The pan-tilt camera focal length performs a zoom-out action; if The pan-tilt camera focal length performs a zoom-in action.

[0034] Another object of the present invention is to provide an unmanned aerial vehicle inspection image acquisition and automatic optimization system, which can solve the technical problem that the current unmanned aerial vehicle inspection image acquisition technology cannot automatically adjust the focal length through the pan-tilt attitude and focal length automatic adjustment algorithm.

[0035] As a preferred solution of the unmanned aerial vehicle inspection image acquisition and automatic optimization system described in the present invention, it includes: an optical correction module, an image recognition module, and a pan-tilt adjustment module; the optical correction module is used to perform optical correction on the images collected by the unmanned aerial vehicle by using the local gray histogram equalization algorithm based on gamma transformation; the image recognition module is used to identify the key targets of the power line in the image according to the target detection algorithm and determine the position and size of the target in the image; the pan-tilt adjustment module is used to calculate the horizontal and vertical rotation angles of the pan-tilt according to the position and size of the target, adjust the pan-tilt attitude, and calculate the adjustment amount of the pan-tilt focal length to adjust the pan-tilt focal length to an appropriate ratio.

[0036] A computer device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the steps of the unmanned aerial vehicle inspection image acquisition and automatic optimization method.

[0037] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, it implements the steps of an unmanned aerial vehicle (UAV) inspection image acquisition and automatic optimization method.

[0038] Advantages of the present invention: The UAV inspection image acquisition and automatic optimization method provided by the present invention can effectively improve the quality of UAV inspection images through a local gray-level histogram equalization algorithm based on gamma transformation, solve the problems of overexposure or underexposure due to changes in lighting conditions, and provide a clear image basis for subsequent defect identification and analysis; through a deep learning object detection algorithm, it can accurately identify key targets of power lines in the image, and adjust the pan-tilt attitude according to the position and size of the targets to ensure that the targets are always located at the center of the image, improving the accuracy of defect identification; through a size estimation algorithm and a reverse consistency check mechanism, it can automatically adjust the pan-tilt focal length according to the change in the relative distance between the target and the UAV, ensure the stability of the target size in the image, and improve the image clarity; through the automation of image quality improvement, target tracking, and focal length adjustment, it can significantly improve the efficiency of UAV inspection, reduce manual intervention, and lower the inspection cost; through UAV inspection, it can avoid the safety risks of manual inspection and improve the safety of inspection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0040] Figure 1 It is the overall flowchart of a UAV inspection image acquisition and automatic optimization method provided by the first embodiment of the present invention.

[0041] Figure 2 It is the corresponding table diagram of the average gray value and γ value of a UAV inspection image acquisition and automatic optimization method provided by the first embodiment of the present invention.

[0042] Figure 3 It is the field of view diagram of the pan-tilt camera of a UAV inspection image acquisition and automatic optimization method provided by the first embodiment of the present invention.

[0043] Figure 4 It is the overall flowchart of a UAV inspection image acquisition and automatic optimization system provided by the third embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0045] Example 1. Referring to Figures 1 - 3 , which is an embodiment of the present invention, provides a method for collecting and automatically optimizing drone inspection images, including:

[0046] S1: Use the local gray histogram equalization algorithm based on gamma transformation to perform optical correction on the images collected by the drone.

[0047] Furthermore, the optical correction of the images collected by the drone includes converting the images collected by the drone into grayscale images and calculating the average gray value of the grayscale images; assuming that the resolution of the drone inspection images is m×n, and converting the images from the three-color space to the grayscale space, then the average gray value is expressed as:

[0048]

[0049] where f(x,y) represents the gray value at the image pixel point (x,y); determine the gamma coefficient according to the average gray value and gamma value correspondence table, perform gamma transformation on the grayscale image, and perform optical correction on the image; perform local histogram equalization processing on the gamma-transformed image to further improve the image quality.

[0050] It should be noted that the histogram equalization process satisfies two conditions. Condition 1: Regardless of which function is selected for the mapping relationship, it is necessary to ensure that the size of the image does not change. Condition 2: For the image after equalization, the pixel values of each point cannot exceed the gray range, that is, the pixel values of the output image should be between 0 and 255.

[0051] It should also be noted that during local histogram equalization, to avoid being unable to adaptively select the processed image, the adapthisteq function is called in MATLAB to implement local histogram equalization of the image.

[0052] S2: Identify the key targets of the power lines in the image according to the target detection algorithm, and determine the position and size of the targets in the image.

[0053] Further, determining the position and size of the target in the image includes using a pre-trained deep learning model to perform object detection on the image to identify key power line objects in the image; determining the position and size of the target in the image according to the output result of the object detection; the position and size include the center point coordinates of the target and the length and width of the target area.

[0054] S3: According to the position and size of the target, calculate the horizontal and vertical rotation angles of the pan-tilt, adjust the pan-tilt attitude, and calculate the adjustment amount of the pan-tilt focal length, and adjust the pan-tilt focal length to an appropriate ratio.

[0055] Further, adjusting the pan-tilt focal length to an appropriate ratio includes calculating the horizontal and vertical rotation angles of the pan-tilt according to the difference between the center point coordinates of the target and the center point coordinates of the image; calculating the adjustment amount of the pan-tilt focal length according to the ratio of the length and width of the target area to the image size; controlling the pan-tilt to rotate horizontally and vertically to make the target located at the center of the image; controlling the pan-tilt to adjust the focal length to make the target occupy an appropriate ratio in the image.

[0056] It should be noted that calculating the horizontal and vertical rotation angles of the pan-tilt includes setting the threshold ranges in the horizontal and vertical directions between the center point coordinates of the target and the center point coordinates of the image; the shooting field of view of the pan-tilt camera is m×n, the center point coordinates (x p , y p ) of the target recognized by the object detection algorithm, the size P L ×P W of the detected window, and the range of the threshold area S The center point coordinates (x s , y s ) of the S area; the center point coordinates of the S area are also the center point coordinates of the image; according to the differences in the horizontal and vertical directions between the center point coordinates of the target and the center point coordinates of the image, determine whether the center point of the target exceeds the threshold range. The judgment of the target center point threshold range is expressed as:

[0057]

[0058] Calculate the horizontal and vertical rotation angles of the pan-tilt according to the differences in the horizontal and vertical directions between the center point coordinates of the target and the center point coordinates of the image and the preset pan-tilt rotation speed; in the horizontal direction, if then the pan-tilt camera turns to the right, if then the pan-tilt camera turns to the left; in the vertical direction, if then the pan-tilt camera turns up, if then the pan-tilt camera turns down.

[0059] It should also be noted that calculating the adjustment amount of the pan-tilt focal length includes that during the tracking process, the size of the target in each frame of the image changes. The size ratio of the target is analyzed and estimated, and then corrected through the size estimation algorithm to achieve the zoom adjustment parameters of the pan-tilt; the size ratio of the target in the picture is estimated through the size estimation algorithm. The target is represented by an elliptical area in the image, and ξ(x i ,y i ,h) represents the elliptical equation of the target area, and the equation is expressed as:

[0060]

[0061] Among them, ξ(x i ,y i ,h) represents the target area, x i represents the pixel coordinates within the sample box, y represents the center position of the target area, h represents the scale parameter of the target candidate area, and a and b respectively represent the long semi-axis and short semi-axis parameters of the ellipse; then the target model of the first frame is expressed as:

[0062]

[0063] Among them, C represents the normalization constant, {x i} i＝1,...n represents the pixel positions within the sample box, represents the indicator function used to count the pixel information in the target area. If the pixel point x i belongs to the u-th color feature, then the value of the pixel point x i is 1, otherwise it is 0; the target candidate of the current frame is expressed as:

[0064]

[0065] Among them, y = (y 1 -y 2 ) T represents the center position of the target candidate of the current frame; the candidate target area is defined through the Riemann integral and the Bhattacharyya coefficient, and after multiple iterations of calculation, it moves iteratively from the current position along direction to obtain the new position y1 and the new target scale parameter h1;

[0066] Through the probability density along the gradient method, the updated position y1 and the scale parameter h1 of the target are obtained, which are expressed as:

[0067]

[0068] Among them, y1 represents the target position after iterative update, h1 represents the scale parameter after iterative update, m kdenotes the displacement vector, w i denotes the weight coefficient, g(·) denotes the contour function, and G denotes the kernel function; a correction and detection mechanism is introduced. Considering the case where the estimated scale gradually increases, a regularization term rs(y, h) is added for reverse variation; considering that the target image is small in size and has small scale changes relative to the ground perspective, while the scale of self-similar objects is uncertain, resulting in an underestimation of the scale, the search window is forced to include a part of the background pixels to expand the scale of the sample box, and a regularization term rb(y, h) is added; the final scale update formula is obtained and expressed as:

[0069]

[0070] The final scale result is used for the scale change of the target during the inspection process.

[0071] It should also be noted that calculating the adjustment amount of the pan-tilt focal length also includes scale updating based on the scale change evaluation criteria when there is no scale change or the tracking box changes greatly and contains obvious background clutter; the scale change evaluation criteria include obtaining the position y t-1 , S t-1 ) of the current frame from the position and scale information (y t and the estimated scale parameter h t of the previous frame, performing reverse tracking to obtain the estimated position y black and the estimated scale parameter h black of the previous frame, verifying the consistency of h black and h t , and determining the final estimated scale according to different situations of the consistency coefficient; the consistency verification coefficient is expressed as:

[0072] θ c = |log2(h t ·h black )|

[0073] where the threshold of θ c is set to 0.1; if it is detected that the estimated scale ratio before and after is consistent, that is, θ c < 0.1, then the scale is updated according to ; if it is detected that the estimated scale ratio before and after is inconsistent, that is, θ c ≥ 0.1, then the target scale is weighted and combined with the size of the previous frame, the new estimated size, and the size of the target at the first frame, expressed as:

[0074]

[0075] where β and σ respectively represent the parameters obtained by testing on a subset of the test sequence; as the target and the pan-tilt camera move relative to each other, considering the loss of target detection caused by the movement in the depth direction, a threshold setting is performed on the focal length, which is expressed as:

[0076]

[0077] If the pan-tilt camera performs a focal length reduction action; if the pan-tilt camera performs a focal length enlargement action.

[0078] It should also be noted that the candidate target area is defined by Riemann integral and the Bhattacharyya coefficient through multiple iterative calculations. Specifically, C h can be obtained by letting n1 represent the number of pixels in the target model area, and n h represent the number of pixels in the target candidate area with size h, and n h = h 2 n1. According to the definition of Riemann integral, and for any two values h0, h1, there is By using the Bhattacharyya coefficient to measure the difference between the tracking target probability distribution q = {q u} u＝1,...,m and the probability distribution {p u (h)} u＝1,...,m of the candidate target; based on the fact that the similarity between the two probability distributions is the highest when the Bhattacharyya coefficient between them is the largest, the maximum value of the ρ(y, h) function is obtained along the gradient ascent direction.

[0079] It should also be noted that the displacement vector m k , which is expressed as:

[0080]

[0081] The weight coefficient w i , which is expressed as:

[0082]

[0083] The kernel function G, which is expressed as:

[0084]

[0085] Embodiment 2 is an embodiment of the present invention, which provides a method for unmanned aerial vehicle inspection image acquisition and automatic optimization. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0086] Verify the image quality improvement ability, target detection accuracy, pan-tilt attitude adjustment efficiency, and focal length adaptive accuracy of the UAV inspection image acquisition and automatic optimization method proposed by the present invention under complex lighting conditions, and compare the performance differences with the prior art. The prior art has no optical correction, manual pan-tilt adjustment, and fixed focal length.

[0087] First, select the UAV model DJI Matrice 300RTK, equipped with a 30x optical zoom pan-tilt camera with a resolution of 4096×2160 pixels. The control group is equipped with the same model of UAV with the automatic optical correction and pan-tilt adjustment functions turned off. Install the system of the present invention and the control system of the prior art on the UAVs respectively.

[0088] Select a certain high-voltage transmission line area as the test scene, and select 3 typical lighting conditions: strong light at noon, weak light on cloudy days, and backlight environment. Each group of tests is repeated 10 times, and the average value is taken.

[0089] Among them, for the experimental steps, the UAV of the present invention group flies to a fixed height of 50 meters. After collecting the original image, the system automatically converts it into a grayscale image, calculates the average grayscale value, selects the best value according to the preset gamma coefficient table for gamma transformation, and then performs local histogram equalization on the image in blocks; performs power equipment detection on the corrected image according to the pre-trained target detection model. The power equipment includes but is not limited to insulators and wire joints, outputs the coordinates of the target center point and the size of the bounding box, calculates the horizontal and vertical rotation angles of the pan-tilt according to the deviation of the target center area from the image center, and at the same time dynamically adjusts the focal length based on the target size ratio to make the target ratio stable at 12.5% - 25%; the prior art group uses global histogram equalization processing to perform optical correction on the collected images, locates the target through Aanny edge detection + template matching, outputs the target position, the operator manually adjusts the pan-tilt angle and focal length, records the adjustment time consumption and the final target ratio, and records the experimental data.

[0090] Finally, based on the experimental data, in terms of the image quality improvement effect, in the contrast data, the present invention reaches 68.9, 52.7, and 47.3 (standard deviation) under strong light, weak light, and backlight conditions respectively, which is higher than 35.2, 18.7, and 12.4 of the prior art without correction and 45.6, 28.3, and 20.1 of the prior art with only gamma correction. This shows that the combination of gamma transformation and local histogram equalization can effectively enhance image details; in the brightness uniformity data, the regional grayscale variance of the present invention is 150 - 210, which is lower than the lowest 320 of the prior art, indicating that local equalization avoids the overexposure or underexposure problems caused by the global algorithm and realizes light-adaptive compensation.

[0091] In terms of object detection and tracking performance, in the accuracy data, the average detection accuracy of the present invention is 91.6%. Compared with 62.1% and 72.0% of the prior art, the accuracy is relatively high, mainly due to the robustness of the object detection model and the suppression of noise by optical correction. In the false detection rate data, the false detection rate of the present invention is as low as 2.1% - 5.4%, while the prior art is as high as 38.7% and 26.5% respectively under backlight, indicating that traditional edge detection is vulnerable to light interference, and the object detection model combined with high-quality input images can effectively reduce misjudgment.

[0092] In terms of the control efficiency of the pan-tilt and focal length, in the adjustment time data, the pan-tilt adjustment of the present invention takes 1.2 - 1.8 seconds, and the efficiency is improved by more than 80% compared with the 8.5 - 10.1 seconds of manual adjustment in the prior art. The present invention realizes fast closed-loop control by calculating the deviation amount in real time through the PID algorithm. In the focal length stability data, the fluctuation range of the target proportion of the present invention is ±4.5% - ±7.2%, which is better than ±12.8% - ±30.5% of the prior art. This indicates that the size estimation algorithm effectively suppresses background interference and scale drift by dynamically correcting scale parameters, namely the regularization term and consistency verification.

[0093] In summary, the present invention has technological innovation in image quality, object detection, and dynamic adjustment, especially in complex inspection scenarios with variable lighting and dynamically changing object scales.

[0094] Example 3, referring to Figure 4 , which is an embodiment of the present invention, provides an unmanned aerial vehicle inspection image acquisition and automatic optimization system, including an optical correction module, an image recognition module, and a pan-tilt adjustment module.

[0095] Among them, the optical correction module is used to perform optical correction on the images collected by the unmanned aerial vehicle by using the local gray histogram equalization algorithm based on gamma transformation; the image recognition module is used to identify the key targets of the power line in the image according to the object detection algorithm and determine the position and size of the targets in the image; the pan-tilt adjustment module is used to calculate the horizontal and vertical rotation angles of the pan-tilt according to the position and size of the targets, adjust the pan-tilt attitude, and calculate the adjustment amount of the pan-tilt focal length to adjust the pan-tilt focal length to an appropriate ratio.

[0096] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes of various kinds.

[0097] The logic and / or steps represented in the flowchart or described in other ways herein, for example, can be considered as a defined sequence list of executable instructions for implementing a logical function, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processor, or other systems that can fetch and execute instructions from the instruction execution system, apparatus, or device), or in combination with these instruction execution systems, apparatus, or devices. For the purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in combination with an instruction execution system, apparatus, or device.

[0098] More specific examples (non-exhaustive list) of computer-readable media include the following: electrical connection parts with one or more wirings (electronic devices), portable computer disk cartridges (magnetic devices), random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), optical fiber devices, and portable compact disc read-only memories (CDROMs). Additionally, a computer-readable medium can even be paper or other suitable media on which a program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpretation, or other suitable processing when necessary, and then stored in a computer memory.

[0099] It should be understood that various parts of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following techniques well known in the art can be used: discrete logic circuits having logic gate circuits for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), etc. It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

[0100] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for collecting and automatically optimizing unmanned aerial vehicle inspection images, characterized in that: include: The images collected by the UAV are optically corrected using the local grayscale histogram equalization algorithm based on gamma transform; Identify key power line targets in the image based on the target detection algorithm, and determine the location and size of the target in the image; According to the position and size of the target, calculate the horizontal and vertical rotation angles of the gimbal, adjust the gimbal posture, calculate the adjustment amount of the gimbal focal length, and adjust the gimbal focal length to the appropriate ratio.

2. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images according to claim 1, characterized in that: The optical correction of the image collected by the drone includes converting the image collected by the drone into a grayscale image and calculating the average grayscale value of the grayscale image; Assuming that the resolution of the drone inspection image is m×n, the image is converted from the three-color space to the grayscale space, then the average grayscale value It is expressed as: Among them, f(x,y) represents the gray value of the image pixel (x,y); Determine the gamma coefficient according to the correspondence table between the average gray value and the γ value, perform gamma transformation on the gray image, and perform optical correction on the image; The gamma transformed image is subjected to local histogram equalization to further improve the image quality.

3. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images according to claim 2, characterized in that: Determining the position and size of the target in the image includes using a pre-trained deep learning model to perform target detection on the image and identify key targets of the power line in the image; According to the target detection output results, determine the position and size of the target in the image; The position and size include the coordinates of the center point of the target and the length and width of the target area.

4. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images according to claim 3, characterized in that: The adjusting the focal length of the gimbal to a suitable ratio includes calculating the horizontal and vertical rotation angles of the gimbal according to the difference between the coordinates of the target center point and the coordinates of the image center point; Calculate the adjustment amount of the pan / tilt focal length based on the ratio of the length and width of the target area to the image size; Control the gimbal to rotate horizontally and vertically so that the target is located in the center of the image; Control the gimbal to adjust the focus so that the target occupies an appropriate proportion of the image.

5. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images as claimed in claim 4, characterized in that: The calculating of the horizontal and vertical rotation angles of the gimbal includes setting the horizontal and vertical direction threshold ranges of the target center point coordinates and the image center point coordinates; The pan / tilt camera captures a field of view of m×n, and the target center coordinates (x p ,y p ), the detection window size P L ×P W , the range of the threshold region S The center point coordinates of the S region (x s ,y s ); The center point coordinates of the S area are also the center point coordinates of the image; According to the horizontal and vertical differences between the target center coordinates and the image center coordinates, it is determined whether the target center exceeds the threshold range. The target center threshold range determination is expressed as: Calculate the horizontal and vertical rotation angles of the gimbal according to the horizontal and vertical differences between the target center coordinates and the image center coordinates and the preset gimbal rotation speed; In the horizontal direction, if Then the gimbal camera turns right. Then the gimbal camera turns left; In the vertical direction, if Then the gimbal camera turns upwards. The gimbal camera turns downward.

6. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images according to claim 5, characterized in that: The calculation of the adjustment amount of the pan-tilt focal length includes: during the tracking process, the size of the target in each frame image changes, and the size ratio of the target is analyzed and estimated, and the correction is calculated by the size estimation algorithm to realize the zoom adjustment parameter of the pan-tilt; The size estimation algorithm is used to estimate the proportion of the target in the picture. The target is represented by an elliptical area in the image, denoted by ξ(x i ,y i ,h) represents the ellipse equation of the target area, which is expressed as: Among them, ξ(x i ,y i ,h) represents the target area, x i represents the pixel coordinates in the sample frame, y represents the center position of the target area, h represents the scale parameter of the target candidate area, a and b represent the major and minor semi-axis parameters of the ellipse respectively; Then the target model of the first frame is expressed as: Where C represents the normalization constant, {x i ) i＝1,...n represents the pixel position within the sample frame, Represents the indicator function used to count the pixel information in the target area. If the pixel point x i Belongs to the uth color feature, then the pixel x i The value is 1, otherwise it is 0; The current frame target candidate is expressed as: Where y=(y 1 -y 2 ) T Indicates the center position of the target candidate in the current frame; The candidate target area is defined by Riemann integral and the Bhattacharyya coefficient is calculated through multiple iterations from the current position along Move iteratively in the direction to obtain a new position y1 and a new target scale parameter h1; Through the probability density along the gradient method, the updated position y1 and the scale parameter h1 of the target are obtained, which can be expressed as: Among them, y1 represents the target position after iterative update, h1 represents the scale parameter after iterative update, and m k represents the displacement vector, w i represents the weight coefficient, g(·) represents the contour function, and G represents the kernel function; Introduce a correction and detection mechanism, consider the situation where the estimation scale gradually increases, and add a regular term rs(y,h) to perform reverse changes; Considering that the target image is small in size and scale change relative to the ground perspective, and the scale of the self-similar object is uncertain, which leads to the underestimation of the scale, the search window is forced to include a part of the background pixels to expand the sample frame scale, and a regular term rb(y,h) is added; The final scale update formula is obtained, expressed as: The final scale result is used when the scale of the target changes during the inspection process.

7. The method for collecting and automatically optimizing unmanned aerial vehicle inspection images according to claim 6, characterized in that: The calculating of the adjustment amount of the pan / tilt focal length also includes updating the scale based on a scale change evaluation standard when there is no scale change or the tracking frame changes greatly and contains obvious background clutter; The scale change evaluation criteria include the position and scale information of the previous frame (y t-1 ,S t-1 )Get the position y of the current frame t and estimate the scale parameter h t , perform reverse tracking to obtain the estimated position y of the previous frame black and estimate the scale parameter h black , h black and h t Conduct consistency verification and determine the final estimation scale based on different consistency coefficients; The consistency verification coefficient is expressed as: θ c =|log2(h t ·h black )| Among them, θ c The threshold of is set to 0.1; If it is detected that the ratio of the estimated scales before and after is consistent, that is, θ c <0.1, then according to Update the scale; If the estimated ratios of the previous and next scales are inconsistent, θ c <0.1, the tracking target scale is a weighted combination of the size of the previous frame, the new estimated size, and the size of the target in the first frame, expressed as: in, β and σ represent the parameters obtained by testing on a subset of the test sequence; As the target moves relative to the PTZ camera, considering the loss of target detection caused by the movement in the depth direction, the focal length threshold is set, which is expressed as: like The pan / tilt camera zooms out; like The pan / tilt camera focus is zoomed in.

8. A system using the unmanned aerial vehicle inspection image acquisition and automatic optimization method according to any one of claims 1 to 7, characterized in that: Including optical correction module, image recognition module, pan / tilt adjustment module; The optical correction module is used to perform optical correction on the image collected by the drone using a local grayscale histogram equalization algorithm based on gamma transform; The image recognition module is used to identify key targets of the power line in the image according to the target detection algorithm, and determine the position and size of the target in the image; The gimbal adjustment module is used to calculate the horizontal and vertical rotation angles of the gimbal according to the position and size of the target, adjust the gimbal posture, and calculate the adjustment amount of the gimbal focal length to adjust the gimbal focal length to an appropriate ratio.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the drone inspection image acquisition and automatic optimization method described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the drone inspection image acquisition and automatic optimization method described in any one of claims 1 to 7 are implemented.

Images