Camera parameter dynamic adjustment method and system for unmanned aerial vehicle intelligent identification

Abstract

This application provides a method and system for dynamically adjusting camera parameters for intelligent drone recognition, relating to the field of image processing technology. The method includes: predicting scene illumination parameters based on environmental data of the area where the drone inspection task is located, and matching this with an initial shutter speed and initial focus distance; adjusting the initial shutter speed to obtain a coarse shutter speed when the distance between the drone and the inspection target is less than a preset first distance threshold; performing focus search and locking the fine-tuning focus distance when the distance between the drone and the inspection target is less than a preset second distance threshold; unlocking the coarse shutter speed, extracting the region of interest corresponding to the inspection target type for local histogram analysis, and fine-tuning the coarse shutter speed to obtain a fine-tuning shutter speed; and controlling the drone to perform inspection shooting based on the fine-tuning focus distance and fine-tuning shutter speed. This application improves the quality of inspection images under complex lighting conditions.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method and system for dynamic adjustment of camera parameters for intelligent recognition of unmanned aerial vehicles (UAVs). Background Technology

[0002] In high-altitude areas, power transmission and distribution line inspections face complex climatic conditions and unique terrain. These areas often have no or weak signal coverage. Drone inspections, due to their flexibility, efficiency, and lack of ground transportation limitations, have become an important means of ensuring the safe operation of the power grid. The core of drone inspections is acquiring clear and effective inspection images, and image quality directly depends on the precise control of camera parameters. Among these, shutter speed and focusing distance are key parameters determining image brightness and sharpness.

[0003] Currently, camera parameter adjustments for drone inspections mostly rely on fixed preset values ​​or manual adjustments by ground personnel, which is difficult to adapt to the complex and ever-changing inspection environment in high-altitude areas. This region experiences a large dynamic range and drastic changes in lighting conditions, with significant differences in light intensity at different times and under different terrains. Furthermore, the uneven distribution of light is further affected by natural terrain features such as mountains and ravines. Simultaneously, the distance between the drone and the inspection target changes dynamically during flight, and fixed parameters cannot adaptively adjust to these changes, leading to improper image exposure or inaccurate focus, severely impacting image quality and subsequent identification accuracy. Summary of the Invention

[0004] This invention addresses the technical problems in existing technologies for drone power line inspection in high-altitude areas with no or weak signal coverage, where camera parameters cannot adapt to complex lighting and distance changes, focus accuracy is low, and image brightness is mismatched, resulting in poor inspection image quality and affecting the accuracy of subsequent anomaly detection. The invention provides a method and system for dynamically adjusting camera parameters for intelligent drone recognition.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: In a first aspect, the present invention provides a method for dynamically adjusting camera parameters for intelligent recognition of unmanned aerial vehicles (UAVs), comprising: Scene illumination parameters are predicted based on environmental data of the area where the UAV inspection mission is located, and the initial shutter speed and initial focus distance are obtained by matching the scene illumination parameters with the inspection target type. When the distance between the drone and the inspection target is less than a preset first distance threshold, real-time image acquisition is initiated, and the initial shutter speed is adjusted based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain a coarse shutter speed. When the distance between the drone and the inspection target is less than the preset second distance threshold, the coarse shutter speed is locked, the initial focus distance is used as the search starting point, the improved multi-directional Sobel operator is used as the sharpness evaluation function and the focus search is performed, and the fine focus distance is locked at the peak point of the sharpness evaluation value. Unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed based on the analysis results to obtain the fine shutter speed; The drone is controlled to perform inspection and shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

[0006] Secondly, the present invention provides a camera parameter dynamic adjustment system for intelligent recognition of unmanned aerial vehicles, comprising: The parameter initialization module is used to predict scene illumination parameters based on environmental data of the area where the UAV inspection mission is located, and to obtain the initial shutter speed and initial focus distance by matching the scene illumination parameters with the inspection target type. The shutter speed coarse adjustment module is used to start real-time image acquisition when the distance between the UAV and the inspection target is less than a preset first distance threshold, and adjust the initial shutter speed based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain the coarse shutter speed. The focus distance search module is used to lock the coarse shutter speed when the distance between the UAV and the inspection target is less than a preset second distance threshold. It uses the initial focus distance as the search starting point, adopts an improved multi-directional Sobel operator as the sharpness evaluation function and performs focus search, and locks the fine focus distance at the peak point of the sharpness evaluation value. The shutter speed fine-tuning module is used to unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed according to the analysis results to obtain the fine shutter speed. The parameter output module is used to control the UAV to perform inspection shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

[0007] The beneficial effects of this invention are: Compared to existing technologies, this application first predicts scene illumination parameters by acquiring environmental data of the area where the UAV inspection mission is located, and then obtains the initial shutter speed and initial focus distance by matching the inspection target type. This achieves pre-coarse adjustment of camera parameters, providing a reasonable starting point for subsequent dynamic adjustments and avoiding blind searching from default parameters, effectively improving the overall efficiency of parameter adjustment. Secondly, in the first stage of the UAV approaching the target, the initial shutter speed is dynamically adjusted based on the deviation between the global image brightness and the preset target brightness, allowing the image brightness to quickly converge to the sensor's linear response range, obtaining a coarsely adjusted shutter speed. This provides a stable image foundation for subsequent focus search, overcoming the impact of illumination changes during flight on image quality. Thirdly, in the second stage of further approaching the target, after locking the coarsely adjusted shutter speed, the focus search is performed using an improved multi-directional Sobel operator as the sharpness evaluation function, starting from the initial focus distance. Multi-directional gradient information is used to improve the accuracy and noise resistance of sharpness evaluation, and the fine-tuned focus distance is locked at the peak point to ensure sharp target imaging, solving the problem of traditional single-directional gradient evaluation being susceptible to noise interference. Finally, after unlocking the coarse shutter speed, the region of interest for the inspected target type is extracted for local histogram analysis. Fine-tuning is then performed to obtain the fine shutter speed, which optimizes the brightness of the target area and further improves the target recognition success rate, overcoming the defect of appropriate global brightness but poor exposure of the target area.

[0008] Through the above technical solutions, this application constructs a dynamic adjustment mechanism covering the entire process from environmental prediction, coarse brightness adjustment, fine focus adjustment to local brightness fine adjustment. This effectively solves the technical problems of large dynamic range and uneven distribution of light in high-altitude areas, as well as improper image exposure and inaccurate focus caused by dynamic changes in target distance during UAV flight. It improves the brightness uniformity, clarity and target recognition accuracy of inspection images under complex lighting conditions, and provides reliable technical support for intelligent UAV inspection in high-altitude areas. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the method for dynamically adjusting camera parameters for intelligent recognition of unmanned aerial vehicles provided by the present invention. Figure 2 This is a schematic diagram of the camera parameter dynamic adjustment system for intelligent recognition of unmanned aerial vehicles provided by the present invention.

[0010] In the attached diagram, the components represented by each number are as follows: Parameter initialization module 11, shutter speed coarse adjustment module 12, focus distance search module 13, shutter speed fine adjustment module 14, parameter output module 15. Detailed Implementation

[0011] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0013] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0014] Example 1, as Figure 1 As shown, this embodiment of the invention provides a method for dynamically adjusting camera parameters for intelligent recognition of unmanned aerial vehicles (UAVs), including: S10: Based on the environmental data of the area where the UAV inspection mission is located, the scene illumination parameters are predicted, and the initial shutter speed and initial focus distance are obtained by matching the scene illumination parameters with the inspection target type.

[0015] When drones conduct power line inspections at high altitudes, lighting conditions change dynamically with weather, time, and geographical location. Directly using fixed shutter speeds and fixed focus distances is difficult to adapt to different scenarios, resulting in overexposed, underexposed, or blurry images. Furthermore, different types of inspection targets, such as insulators, transmission lines, and towers, have varying light sensitivities and imaging requirements, necessitating differentiated initial parameter settings.

[0016] To address the aforementioned issues, this application predicts scene illumination parameters based on environmental data of the area where the UAV inspection mission is located, and then matches these scene illumination parameters with the inspection target type to obtain the initial shutter speed and initial focus distance. This provides an effective starting point for subsequent dynamic adjustments during flight, avoiding blind searching from scratch and improving the efficiency and accuracy of parameter adjustments.

[0017] Specifically, step S10 in the method includes: The system acquires environmental data for the area where the drone inspection mission is located, including weather forecast data, satellite cloud image data, geographic coordinate information, and current time information. The environmental data is input into a pre-trained illumination prediction model, and scene illumination parameters are output. The scene illumination parameters include illumination dynamic range and illumination principal direction intensity ratio. The illumination dynamic range is used to characterize the brightness difference between the brightest and darkest areas in the area where the UAV inspection task is located. The illumination principal direction intensity ratio is used to characterize the ratio between the illumination intensity in the light source direction and the global average illumination intensity. The training process for the illumination prediction model includes: Collect environmental data samples from historical inspection areas to form an environmental data training sample set; For each environmental data sample in the environmental data training sample set, obtain the corresponding dynamic range of illumination and the intensity ratio of the main direction of illumination to form an illumination parameter annotation set. An initial illumination prediction model is constructed based on a convolutional neural network; Using the environmental data training sample set as input and the illumination parameter annotation set as supervision labels, the initialized illumination prediction model is trained in a supervised manner until the verification convergence is achieved, thus obtaining the pre-trained illumination prediction model.

[0018] In this embodiment, environmental data samples from historical inspection task areas are first collected to form an environmental data training sample set. Specifically, environmental data recorded in multiple past inspection tasks are extracted from historical inspection data, including weather forecast data, satellite cloud image data, geographic coordinate information, and current time information, as environmental data samples. Preferably, to ensure the diversity of training samples, the collected historical inspection task areas can cover typical scenarios with different terrains, climate conditions, seasons, and time periods. For example, the inspection areas of power transmission and distribution lines in high-altitude areas such as Qinghai Province cover various meteorological conditions such as sunny days, cloudy days, foggy days, and snowy days, as well as different solar altitude angles such as sunrise, noon, and sunset. All environmental data samples are combined to form an environmental data training sample set.

[0019] Ideally, for each environmental data sample, actual illumination measurement data is collected synchronously for subsequent annotation. Specifically, during the execution of historical inspection tasks, illumination intensity is recorded in real time using a ground-based illuminance meter array (including multiple measurement points) deployed in the historical inspection task area, or by measuring at key locations using a handheld illuminance meter to obtain illumination intensity values ​​for different areas within the historical inspection task area. This measurement data provides the basis for calculating the dynamic range of illumination and the intensity ratio of the main illumination direction.

[0020] Secondly, for each environmental data sample in the environmental data training sample set, the corresponding dynamic range of illumination (DLL) and the ratio of illumination intensity along the principal direction are obtained to form an illumination parameter annotation set. Specifically, the DLL can be obtained by taking the logarithm of the ratio of maximum to minimum illuminance in the environmental data sample scene (in EV). The ratio of illumination intensity in the principal direction to the labeled value can be obtained by measuring the ratio of the illuminance in the direction of the principal light source (note that in most scenarios, the direction of the principal light source is the direction of the sun) to the average illuminance at multiple measurement points in the environmental data sample scene. The calculated dynamic range and principal direction intensity ratio values ​​of illumination are correlated with the corresponding environmental data samples to form an illumination parameter annotation set.

[0021] For example, if multiple illuminance meters are deployed in the scene of the historical inspection task area, the dynamic range of illumination is calculated by taking the ratio of the maximum illuminance to the minimum illuminance of all illuminance meter readings; at the same time, the illuminance in the direction of direct sunlight is measured by a directional illuminance meter as the illuminance in the direction of the main light source, and compared with the average value of all illuminance meters (i.e., the global average illuminance) to obtain the intensity ratio of the main direction of illumination.

[0022] Next, an initial illumination prediction model is constructed based on a convolutional neural network. Convolutional neural networks have strong feature extraction capabilities, effectively extracting features from environmental data, making them suitable for illumination parameter prediction.

[0023] For example, the initialized illumination prediction model mainly consists of an input layer, an image input branch, and a numerical input branch. The input layer needs to receive multi-source heterogeneous environmental data, including weather forecast data, satellite cloud image data, geographic coordinate information, and current time information. For unified processing, the satellite cloud image data can be scaled to a 64×64 pixel single-channel grayscale image as the image input branch. The weather type (e.g., sunny, cloudy, rainy) in the weather forecast data is uniquely encoded, the latitude and longitude in the geographic coordinate information are normalized to the [0,1] interval, and the current time information is converted into a day of the year (the day of the year) and the number of hours in a day and normalized. These numerical data are concatenated into a feature vector as the numerical input branch.

[0024] The image input branch employs a three-layer convolutional neural network: the first convolutional layer uses 32 3×3 convolutional kernels with a stride of 1, same padding, and ReLU activation function, followed by a 2×2 max pooling layer; the second convolutional layer uses 64 3×3 convolutional kernels with a stride of 1, same padding, and ReLU activation function, followed by a 2×2 max pooling layer; the third convolutional layer uses 128 3×3 convolutional kernels with a stride of 1, same padding, and ReLU activation function, followed by a global average pooling layer, outputting a 128-dimensional image feature vector.

[0025] The numerical input branch uses a two-layer fully connected network: the first fully connected layer contains 64 neurons with the ReLU activation function; the second fully connected layer contains 32 neurons with the ReLU activation function, and outputs a 32-dimensional numerical feature vector.

[0026] The output layer contains two neurons, corresponding to the dynamic range of illumination and the intensity ratio of the principal direction of illumination, respectively: the dynamic range of illumination output neuron uses a linear activation function to directly output the predicted dynamic range of illumination; the principal direction of illumination intensity ratio output neuron uses a sigmoid activation function to constrain the output value to the (0,1) interval, and then maps it to the actual physical range, such as 0.5 to 2.0, through a linear transformation.

[0027] Furthermore, using the environmental data training sample set as input and the illumination parameter annotation set as supervision labels, the initialized illumination prediction model is subjected to supervised training until validation convergence, resulting in a pre-trained illumination prediction model. For example, the training process of the illumination prediction model can be as follows: 1. Data preparation: The environmental data training sample set and the illumination parameter annotation set are randomly divided into a training set, a validation set, and a test set in a ratio of 7:1.5:1.5. The training set is used for model parameter updates, the validation set is used for hyperparameter tuning and convergence judgment, and the test set is used for final model performance evaluation.

[0028] 2. Model Training: Using environmental data samples from the training set as input, and the corresponding dynamic range of illumination and the ratio of illumination intensity in the principal direction as supervision labels, the mean squared error loss function is used to calculate the error between the model's predicted values ​​and the true labels. The optimizer is Adam, with an initial learning rate of 0.001, a batch size of 32, and a maximum of 100 training epochs. After each training epoch, the mean squared error loss is calculated on the validation set. When the validation set loss no longer decreases for 10 consecutive epochs (ideally, the decrease is less than 0.001), the model is considered to have converged, and training is stopped early.

[0029] 3. Model saving: Save the model parameters with the minimum loss on the validation set as the pre-trained illumination prediction model.

[0030] Furthermore, environmental data for the area where the UAV inspection mission is located is acquired. This environmental data includes weather forecast data, satellite cloud image data, geographic coordinate information, and current time information. Weather forecast data refers to real-time and future weather information for the inspection area, such as light intensity and cloud cover. Satellite cloud image data refers to cloud image information of the inspection area obtained via satellite, used to determine cloud cover and analyze light distribution. Geographic coordinate information refers to the latitude, longitude, and altitude of the inspection area, used to analyze lighting conditions in conjunction with topography, such as light obstruction in high-altitude mountainous areas. Current time information refers to the specific time the UAV performs the inspection mission, used to determine the angle of sunlight and analyze light intensity and direction.

[0031] Specifically, weather forecast data for the area where the drone inspection mission is located can be obtained from the meteorological service interface, satellite cloud image data such as real-time cloud coverage can be obtained from satellite cloud images, and then combined with the geographical coordinate information (latitude, longitude, altitude) and current time information (year, month, day, hour) of the area where the drone inspection mission is located to form comprehensive environmental data. These data can comprehensively reflect the macroscopic lighting conditions of the area where the inspection mission is located.

[0032] For example, if the area where the drone inspection mission is located is 40°N, 116°E, the current time is 10:00 AM on June 1, 2024, the weather forecast is clear, and the satellite cloud image shows no clouds, then the environmental data can be comprehensively represented as "clear, no clouds, 40°N, 116°E, 10:00 AM on June 1".

[0033] Finally, the environmental data is input into the pre-trained illumination prediction model, which outputs scene illumination parameters, including the dynamic range of illumination (DLA) and the principal direction intensity ratio. The DLA characterizes the brightness difference between the brightest and darkest areas in the UAV inspection mission area, while the principal direction intensity ratio characterizes the ratio between the illumination intensity along the light source direction and the global average illumination intensity. A larger DLA value indicates a greater brightness difference within the inspection area; a larger principal direction intensity ratio indicates a more concentrated illumination intensity along the light source direction (e.g., sunlight) and a more uneven global illumination distribution.

[0034] For example, if the environmental data "sunny, cloudless, 40°N latitude, 116°E longitude, 10:00 AM on June 1st" is input into the pre-trained illumination prediction model, the output will show an illumination dynamic range of 10 EV and an illumination principal direction intensity ratio of 1.3, indicating that the current scene brightness difference is large and the principal light source direction is 30% stronger than the global average illumination intensity.

[0035] Specifically, the initial shutter speed and initial focus distance are obtained by matching the scene lighting parameters with the inspection target type, including: Pre-store the recommended shutter speed and recommended focus distance corresponding to the combination of dynamic range of illumination, intensity ratio of main direction of illumination and type of inspection target, and construct a camera parameter mapping table; Using the dynamic range of illumination, the ratio of illumination intensity in the main direction, and the type of the target being inspected as joint query conditions, the corresponding recommended shutter speed and recommended focus distance are obtained from the camera parameter mapping table and used as the initial shutter speed and initial focus distance, respectively.

[0036] In this embodiment, a camera parameter mapping table is first pre-stored, containing recommended shutter speeds and recommended focusing distances corresponding to combinations of illumination dynamic range, illumination principal direction intensity ratio, and target type. The camera parameter mapping table is a lookup table storing recommended camera parameters for different combinations of illumination conditions (illumination dynamic range, illumination principal direction intensity ratio) and target type. The recommended shutter speed is the shutter speed, verified through experimental calibration or historical experience, that ensures normal initial exposure of the image under the given illumination conditions and target type. The recommended focusing distance is the focusing distance, verified through experimental calibration or historical experience, that ensures a clear initial image of the target under the given illumination conditions and target type.

[0037] Specifically, by combining the lighting characteristics of high-altitude areas and the types of targets for power transmission and distribution line inspection (e.g., transmission lines, insulators, towers, fittings, etc.), and through the accumulation of a large amount of field experiments or historical experience data, the recommended shutter speed and recommended focus distance for obtaining clear images under different combinations of lighting dynamic range, main direction intensity ratio of lighting and inspection target type are recorded. These are then compiled into a camera parameter mapping table and pre-stored in the UAV's control system for easy quick query and matching.

[0038] It should be noted that the recommended parameters provided by the camera parameter mapping table are only used as initial values ​​for subsequent dynamic adjustments and do not need to reach the final accuracy. Therefore, they can be obtained through a limited number of experiments or empirical statistics, and subsequent steps will be based on these values ​​for fine-tuning.

[0039] For example, a portion of the camera parameter mapping table is as follows: When the dynamic range of illumination is 8EV~8.5EV, the intensity ratio of the main direction of illumination is 1.2~1.3, and the inspection target type is an insulator, the recommended shutter speed is 1 / 500 second and the recommended focusing distance is 15 meters; when the dynamic range of illumination is 6EV~6.5EV, the intensity ratio of the main direction of illumination is 1.3~1.4, and the inspection target type is a power transmission line, the recommended shutter speed is 1 / 300 second and the recommended focusing distance is 20 meters.

[0040] Secondly, the dynamic range of illumination, the ratio of the principal direction of illumination intensity, and the type of the inspected target are used as joint query conditions to obtain the corresponding recommended shutter speed and recommended focusing distance from the camera parameter mapping table, which are then used as the initial shutter speed and initial focusing distance, respectively. For example, based on the predicted dynamic range of illumination (e.g., 8.5 EV), the ratio of the principal direction of illumination intensity (e.g., 1.3), and the inspected target type being an insulator, the recommended shutter speed of 1 / 500 second and the focusing distance of 15 meters are found in the camera parameter mapping table and used as the initial shutter speed and initial focusing distance, respectively.

[0041] In summary, compared to existing technologies, this application predicts scene illumination parameters based on environmental data of the area where the UAV inspection mission is located, and obtains the initial shutter speed and initial focus distance by matching the scene illumination parameters with the inspection target type. This provides targeted initial values ​​for dynamic adjustment of camera parameters during subsequent flight, avoiding blind searching from default parameters and effectively improving the efficiency and accuracy of camera parameter adjustment.

[0042] S20: When the distance between the UAV and the inspection target is less than a preset first distance threshold, real-time image acquisition is started, and the initial shutter speed is adjusted based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain the coarse shutter speed.

[0043] As the drone approaches the inspection target, the ambient lighting conditions may change dynamically, such as cloud cover or changes in the sun's angle. This can cause the initial shutter speed predicted before takeoff to become unsuitable for the actual lighting conditions. If the initial shutter speed is used directly, the image may be overexposed or underexposed, affecting the accuracy of subsequent target recognition. At the same time, when the distance to the target is far, it is not necessary to immediately perform high-precision focusing, but it is necessary to quickly adjust the image brightness to the sensor's linear response range to provide a stable image input for subsequent focusing and searching.

[0044] To address the aforementioned issues, this application initiates real-time image acquisition when the distance between the UAV and the inspection target is less than a preset first distance threshold. The initial shutter speed is adjusted based on the brightness deviation between the global image brightness and the preset target brightness to obtain a coarse shutter speed. By adjusting the shutter speed through closed-loop feedback, the image brightness is rapidly converged to the ideal range, thus obtaining the coarse shutter speed.

[0045] Specifically, step S20 in the method includes: When the distance between the UAV and the inspection target is less than a preset first distance threshold, periodic real-time image acquisition is initiated. Global histogram features are extracted for each frame of real-time image acquired, and the average brightness value of the current image is calculated based on the global histogram features. Obtain the linear response range of the camera image sensor on the drone, calculate the median brightness of the linear response range, and use the median brightness as the preset target brightness; The difference between the average brightness value of the current real-time image and the preset target brightness is calculated to obtain the brightness deviation; Based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed. The next real-time image is re-acquired at the adjusted shutter speed, and the average brightness value and new brightness deviation of the next real-time image are calculated. The step of dynamically adjusting the shutter speed according to the brightness deviation is repeated until the average brightness value converges to the preset target brightness. The shutter speed at which convergence occurs is used as the coarse adjustment shutter speed.

[0046] In this embodiment, when the distance between the UAV and the inspection target is less than a preset first distance threshold, periodic real-time image acquisition is initiated. Global histogram features are extracted for each acquired real-time image frame, and the average brightness value of the current image is calculated based on these features. Specifically, the UAV monitors the distance to the inspection target in real time using its onboard ranging sensor (e.g., LiDAR, millimeter-wave radar, or visual ranging module). When the detected distance is less than the preset first distance threshold, periodic real-time image acquisition begins. For each acquired image frame, it is first converted to a grayscale image. Then, the pixel distribution of all grayscale levels (0-255) is statistically analyzed to generate a global grayscale histogram. Finally, based on the global grayscale histogram, the average brightness value of the current image is calculated using an arithmetic mean formula. .

[0047] The preset first distance threshold is a critical distance value used to trigger the coarse brightness adjustment stage. Its function is to ensure that the drone begins preliminary adjustment of the shutter speed when it is close enough to the target but has not yet entered the fine focusing stage, providing a brightly stable image for subsequent focusing. Determining the preset first distance threshold requires comprehensive consideration of factors such as the typical size of the inspected target, the camera's field of view, the drone's flight speed, and the response time to changes in lighting. Those skilled in the art can dynamically set it according to the specific inspection scenario. For example, for inspecting a 20-meter-high power pole, to ensure the target occupies a certain proportion of the frame and allows sufficient adjustment time, the preset first distance threshold can be set to 50 meters; for smaller targets (such as insulator strings), the preset first distance threshold can be appropriately reduced to ensure the target is sufficiently clear in the image.

[0048] For example, the UAV uses LiDAR to measure distance in real time. When it detects that the distance between itself and the target being inspected is less than a preset first distance threshold of 50 meters, it initiates periodic real-time image acquisition. The first frame of grayscale image acquired is statistically analyzed, and the average brightness value of each grayscale pixel distribution is calculated to be 100 (8-bit grayscale, range 0-255).

[0049] Secondly, the linear response range of the camera image sensor on the UAV is obtained, and the median brightness of the linear response range is calculated. This median brightness is then used as the preset target brightness. Specifically, the camera image sensor on the UAV exhibits linear response characteristics within a certain exposure range, which is called the linear response range. Within the linear response range, the pixel value output by the sensor is proportional to the incident light intensity. The brightness range corresponding to the linear response range can be obtained from the sensor's technical manual or calibration experiments. For example, for an 8-bit depth image sensor, the linear response range is typically [30, 220] (grayscale value). To ensure that the acquired image is neither underexposed (loss of detail in dark areas) nor overexposed (saturation in bright areas), the average brightness of the image should be controlled at the middle position of the linear response range. That is, the arithmetic mean of the minimum and maximum values ​​of the linear response range is taken as the preset target brightness.

[0050] The linear response range refers to the brightness range in which the output of the image sensor maintains a linear relationship with the intensity of the incident light. Beyond this range, it enters the nonlinear region or the saturation region. The median brightness refers to the average value of the two endpoints of the linear response range, which is used to characterize the optimal center point for linear operation of the sensor.

[0051] For example, if the linear response range of a camera image sensor is calibrated as [30, 220], then the preset target brightness = (30 + 220) / 2 = 125.

[0052] Next, the difference between the average brightness value of the current real-time image and the preset target brightness is calculated to obtain the brightness deviation. The brightness deviation is calculated as: Brightness Deviation = Average Brightness Value of the Current Real-Time Image - Preset Target Brightness. The sign of the brightness deviation reflects the direction of the current real-time image's brightness deviation: a positive value indicates the image is too bright; a negative value indicates the image is too dark. The absolute value of the brightness deviation reflects the severity of the deviation.

[0053] For example, if the average brightness of the current real-time image is 100 and the preset target brightness is 125, then the brightness deviation = 100 - 125 = -25, indicating that the current real-time image is too dark.

[0054] Furthermore, the initial shutter speed is dynamically adjusted based on the sign and magnitude of the brightness deviation to obtain the adjusted shutter speed. Specifically, the dynamic adjustment logic is as follows: if the brightness deviation is positive (image too bright), the shutter speed is decreased by a certain adjustment step size to accelerate exposure and thus reduce the amount of light entering the camera; if the brightness deviation is negative (image too dark), the shutter speed is increased by the adjustment step size to slow down exposure and thus increase the amount of light entering the camera. The adjustment step size is positively correlated with the absolute value of the brightness deviation, that is, the larger the deviation, the larger the step size, in order to accelerate the convergence speed.

[0055] Finally, the next real-time image frame is re-acquired at the adjusted shutter speed, and the average brightness value and new brightness deviation of the next real-time image are calculated. This process of dynamically adjusting the shutter speed based on the brightness deviation is repeated until the average brightness value converges to the preset target brightness. The shutter speed at which convergence occurs is then used as the coarse-adjustment shutter speed. Converging to the preset target brightness means that the brightness deviation between the currently calculated average brightness value and the preset target brightness converges within a preset allowable error range. This allowable error range can be dynamically set based on factors such as the linear response range width of the image sensor and the brightness accuracy requirements of the actual application. For example, the allowable error range can be ±5 (for 8-bit grayscale images).

[0056] For example, after several iterations of adjustment, the average brightness value finally reaches 123, with a deviation of -2 from the preset target brightness of 125, which is less than the preset allowable error range of ±5, and is considered convergence. At this point, the recorded shutter speed is 1 / 400 second, which is used as the coarse adjustment shutter speed. In this way, through closed-loop feedback adjustment based on brightness deviation, the image brightness is quickly stabilized within the ideal range, providing a stable image input for subsequent focus search.

[0057] Furthermore, based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed, including: The brightness deviation is used as a feedback quantity, and the current adjustment step size is dynamically determined based on the absolute value of the brightness deviation, wherein the current adjustment step size is positively correlated with the absolute value of the brightness deviation. According to the positive or negative direction of the brightness deviation, the initial shutter speed is adjusted with the current adjustment step size to obtain the adjusted shutter speed: if the brightness deviation is positive, the initial shutter speed is decreased according to the current adjustment step size; if the brightness deviation is negative, the initial shutter speed is increased according to the current adjustment step size.

[0058] In this embodiment, the brightness deviation is first used as a feedback quantity. The current adjustment step size is dynamically determined based on the absolute value of the brightness deviation, where the current adjustment step size is positively correlated with the absolute value of the brightness deviation. Specifically, to accelerate the convergence speed and avoid over-adjustment, a variable step size strategy is adopted: when the absolute value of the brightness deviation is large, it indicates that the image brightness deviates severely, requiring a large step size to quickly approach the target; when the absolute value of the brightness deviation is small, it indicates that the target has been approached, and a small step size should be used for fine adjustment to prevent oscillation. The adjustment step size refers to the amount of change in shutter speed each time it is adjusted, usually expressed in exposure compensation levels (EV). Each 1 EV adjustment is equivalent to doubling or halving the exposure time.

[0059] For example, the adjustment step size can be set using a linear proportional method: adjustment step size = k × |brightness deviation|, where k is an adjustment coefficient that can be dynamically set according to the camera shutter adjustment accuracy and response speed requirements. Those skilled in the art can determine the value of k through experimental calibration and experience setting. Preferably, k can be set to 0.1 level / unit brightness, which means that for every 1 unit deviation from the preset target brightness, the current adjustment step size is 0.1EV, so that the adjustment process is smooth and the convergence speed is moderate.

[0060] For example, the current adjustment step size can also be set directly using a piecewise function based on computing resources or experience: for instance, when the absolute value of the brightness deviation is >50, the current adjustment step size is 1 EV to quickly pull back the brightness; when the absolute value of the brightness deviation is between 20 and 50, the current adjustment step size is 0.5 EV for medium-amplitude adjustment; and when the absolute value of the brightness deviation is <20, the current adjustment step size is 0.2 EV for fine-tuning. This piecewise method can be preset according to the response characteristics of the actual image sensor and the target brightness range, without the need for real-time calculation, and is suitable for embedded platforms with limited computing resources.

[0061] Secondly, the initial shutter speed is adjusted according to the direction of the brightness deviation, using the current adjustment step size, to obtain the adjusted shutter speed: if the brightness deviation is positive, the initial shutter speed is decreased by the current adjustment step size; if the brightness deviation is negative, the initial shutter speed is increased by the current adjustment step size. Specifically, the adjustment logic is as follows: if the brightness deviation is positive, it means the current image is too bright, and the shutter speed needs to be decreased to reduce the amount of light entering the camera; if the brightness deviation is negative, it means the current image is too dark, and the shutter speed needs to be increased to increase the amount of light entering the camera. During adjustment, the shutter speed is adjusted exponentially by the current adjustment step size, for example, decreasing by 1 / 3 stop or increasing by 1 / 2 stop. The final result should be the stop value actually supported by the camera, which is taken as the adjusted shutter speed.

[0062] For example, assuming the brightness deviation is -25 (negative value, the image is too dark), the shutter speed needs to be increased. If the initial shutter speed is 1 / 500 second and the current adjustment step is 0.5 EV, then the theoretical value of the adjusted shutter speed is 1 / 500×2^(0.5)≈1 / 354 second. In practice, the nearest shutter speed supported by the camera, such as 1 / 320 second or 1 / 400 second (depending on the camera's shutter speed accuracy), is taken as the adjusted shutter speed.

[0063] In summary, compared to existing technologies, this application initiates real-time image acquisition when the distance between the UAV and the inspection target is less than a preset first distance threshold. It then adjusts the initial shutter speed based on the brightness deviation between the global image brightness and the preset target brightness to obtain a coarse shutter speed. Thus, by acquiring images in real-time and dynamically adjusting the shutter speed based on the brightness deviation, a rapid response to changes in illumination during flight is achieved. This ensures that the image brightness stably converges to the ideal range before entering the fine focusing stage, providing a moderately bright image foundation for subsequent high-precision focusing.

[0064] S30: When the distance between the UAV and the inspection target is less than the preset second distance threshold, lock the coarse shutter speed, take the initial focus distance as the search starting point, use the improved multi-directional Sobel operator as the sharpness evaluation function and perform focus search, and lock the fine focus distance at the peak point of the sharpness evaluation value.

[0065] After coarse brightness adjustment, the image brightness has stabilized within the sensor's linear response range. However, the focusing distance is still an initial value obtained based on matching illumination parameters and target type, which may deviate from the actual optimal focus point. Directly using the initial focusing distance will result in blurred target imaging, affecting subsequent recognition accuracy.

[0066] Meanwhile, as the drone approaches the target, changes in distance will affect the depth of field and the optimal focus position. Furthermore, the richness of image details will vary depending on the dynamic range of illumination, which will also affect the requirements for the search step size.

[0067] To address the aforementioned issues, this application locks the coarse shutter speed when the distance between the UAV and the inspection target is less than a preset second distance threshold. Using the initial focus distance as the search starting point, an improved multi-directional Sobel operator is employed as the sharpness evaluation function to perform a focus search. At the peak point of the sharpness evaluation value, the fine-tuning focus distance is locked.

[0068] Specifically, step S30 in the method includes: When the distance between the drone and the inspection target is less than the preset second distance threshold, the coarse shutter speed is locked to a fixed value, and the adjustment of the shutter speed is stopped. The real-time distance and dynamic range of illumination between the current UAV and the inspection target are obtained, and the adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination. Using the initial focusing distance as the search starting point, the lens is driven to move along the focusing direction according to the adaptive search step size. A real-time image is acquired at each search position, and the sharpness evaluation value of each real-time image is calculated using the improved multi-directional Sobel operator to form a sharpness evaluation curve. Search for the peak point of the sharpness evaluation value in the sharpness evaluation curve, and lock the focus distance corresponding to the peak point as the fine-tuning focus distance; The adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination, including: The basic search step size is preset based on the stepping accuracy of the lens focusing motor and the focal length of the lens. Calculate the ratio of the real-time distance to the preset second distance threshold to obtain the distance adjustment coefficient; The ratio of the illumination dynamic range to the maximum dynamic range of the UAV camera image sensor is calculated to obtain the illumination adjustment coefficient; The adaptive search step size is obtained by multiplying the basic search step size by the distance adjustment coefficient and the illumination adjustment coefficient.

[0069] In this embodiment, when the UAV flies to a distance less than a preset second distance threshold from the target, the coarse shutter speed is locked to a fixed value, and shutter speed adjustment is stopped. Specifically, when the UAV detects that the distance to the target is less than the preset second distance threshold using the same ranging sensor as in the previous steps, the current shutter speed (i.e., the coarse shutter speed obtained in step S20) is immediately fixed and no longer responds to brightness deviation feedback adjustment. The purpose of this is to ensure that the image brightness remains constant during the focus search process, avoiding image brightness fluctuations caused by shutter speed changes, which could interfere with the calculation results of the sharpness evaluation function, thus ensuring the accuracy and stability of the focus search.

[0070] For example, the preset second distance threshold refers to a critical distance value used to trigger the focus fine-tuning stage. Its function is to lock the shutter speed and initiate high-precision focus search when the drone enters this distance range, ensuring a clear image is obtained when the target is close enough, while avoiding excessively long search times or misjudgments due to depth of field effects caused by starting fine-tuning too early at a greater distance. The determination of the preset second distance threshold can comprehensively consider factors such as the type of inspected target, typical size, lens focal length, depth of field range, and drone flight speed.

[0071] For example, those skilled in the art can dynamically set a preset second distance threshold according to specific application scenarios: for smaller inspection targets, to ensure that the inspection target occupies enough pixels in the image for sharpness evaluation, the preset second distance threshold can be appropriately reduced; for larger inspection targets, the preset second distance threshold can be appropriately increased to allow more search time. Specifically, it can be estimated using a formula based on the principle of similar triangles: Preset second distance threshold = (inspection target size × lens focal length) / (number of pixels the inspection target is expected to occupy on the image sensor × pixel size). Wherein, the inspection target size is the actual physical length of the target in the direction perpendicular to the optical axis, the lens focal length is the camera optical parameter, the number of pixels expected to occupy is determined according to the image resolution requirements of the target recognition algorithm, and the pixel size is the physical side length of a single pixel on the image sensor.

[0072] For example, in the inspection of power poles, the target pole's crossarm width is approximately 10 meters. The camera lens on the drone has a focal length of 50mm, and the image sensor pixel size is 5μm. It is desired that the pole occupies at least 1000 pixels in the image height direction to ensure recognition accuracy. Substituting into the formula, we get: Preset second distance threshold = (10,000mm × 50mm) / (1000 pixels × 0.005mm / pixel) = 100000mm = 100 meters. Therefore, the preset second distance threshold can be set to 100 meters. When the drone flies to a distance of less than 100 meters from the power pole, the shutter speed is locked at a coarse adjustment shutter speed, and focus searching begins.

[0073] Secondly, the real-time distance and dynamic range of illumination between the current UAV and the inspection target are obtained, and the adaptive search step size is calculated based on the real-time distance and dynamic range of illumination. Specifically, after locking the shutter speed, it is necessary to determine the focus search step size. The traditional fixed step size has a trade-off between efficiency and accuracy: if the step size is too large, it is easy to skip the peak, and if the step size is too small, the search time will be too long. Therefore, this application dynamically calculates the adaptive search step size based on the real-time distance and dynamic range of illumination between the current UAV and the inspection target.

[0074] Next, starting from the initial focusing distance, the lens is driven to move along the focusing direction according to an adaptive search step size. A real-time image frame is acquired at each search position, and the sharpness evaluation value of each real-time image frame is calculated using an improved multi-directional Sobel operator, forming a sharpness evaluation curve. For example, the focus search can employ a "hill-climbing" strategy: starting from the initial focusing distance, the lens moves in one direction (e.g., closer) with an adaptive step size, acquiring a real-time image frame at each step and calculating its sharpness evaluation value using the improved multi-directional Sobel operator. During this process, if the sharpness evaluation value continues to rise, the movement continues in the same direction; conversely, when the sharpness evaluation value begins to fall, it indicates that the peak has been exceeded, at which point the movement reverses and a fine search is performed to finally determine the peak position.

[0075] Finally, the peak point of the sharpness evaluation value is searched in the sharpness evaluation curve, and the focus distance corresponding to the peak point is locked as the fine-tuning focus distance. Specifically, after obtaining the sharpness evaluation curve, the maximum value point of the sharpness evaluation curve is selected as the peak point.

[0076] Optionally, to further improve peak positioning accuracy, a parabolic fitting method can be used: Select the point with the highest sharpness evaluation value and its two adjacent points (a total of three points), and let their coordinates be (d1, F1), (d2, F2), and (d3, F3), where d is the focusing distance and F is the sharpness evaluation value. Assume that these three points approximately satisfy the quadratic function F = a·d 2 Substituting the coordinates of the three points into the equation +b·d+c, and solving the linear equations to obtain the coefficients a and b, the focusing distance corresponding to the vertex (peak point) of the parabola is d_vertex = -b / (2a). This is used as the fine-tuning focusing distance. This method achieves sub-step-level positioning accuracy, which is superior to coarse positioning that only selects the maximum value point. After determining the peak value, the lens position corresponding to the peak value is locked as the fine-tuning focusing distance for subsequent formal shooting.

[0077] For example, suppose that on the sharpness evaluation curve, the point with the highest sharpness evaluation value is located at a focus distance d2 = 15.2 mm, with a sharpness evaluation value F2 = 1560. The adjacent point to the left has a focus distance d1 = 15.1 mm and an F1 = 1520, while the adjacent point to the right has a focus distance d3 = 15.3 mm and an F3 = 1540. Substituting the coordinates of these three points into the quadratic function F = a·d mentioned above... 2 Solving for +b·d+c, we get the vertex position d_vertex = 15.18mm. Therefore, we lock 15.18mm as the fine-tuning focus distance.

[0078] Furthermore, a basic search step size is preset based on the stepping accuracy of the lens focusing motor and the lens focal length. The basic search step size refers to the minimum physical displacement driven by the lens, which is determined by the stepping accuracy of the lens focusing motor and the lens's mechanical structure.

[0079] For example, the minimum step size of the focusing motor of a certain lens model is 0.1mm, which means that the lens can be moved by 0.1mm per step. This value is used as the basic search step size.

[0080] Furthermore, the ratio of the real-time distance to the preset second distance threshold is calculated to obtain the distance adjustment coefficient. The distance adjustment coefficient = real-time distance / preset second distance threshold. This coefficient reflects the distance between the real-time distance and the preset second distance threshold: when the UAV is far from the target, the depth of field is large, and the focusing accuracy requirement is relatively low, allowing for a larger step size to speed up the search; conversely, when the UAV is close to the target, the depth of field becomes shallower, the focusing accuracy requirement increases, and the step size needs to be reduced.

[0081] For example, if the real-time distance is 8 meters and the preset second distance threshold is 10 meters, then the distance adjustment coefficient = 8 / 10 = 0.8.

[0082] Furthermore, the ratio of the illumination dynamic range to the maximum dynamic range of the UAV camera image sensor is calculated to obtain the illumination adjustment coefficient. The illumination adjustment coefficient = illumination dynamic range / maximum dynamic range of the UAV camera image sensor. This coefficient reflects the impact of the current scene's illumination richness on the search step size: a larger illumination dynamic range indicates richer brightness levels in the scene, more image details, and a sharper sharpness evaluation curve near the peak, requiring a smaller search step size to accurately capture the peak position; conversely, a smaller illumination dynamic range results in less image detail and a flatter sharpness curve, allowing for a larger step size to improve search efficiency.

[0083] The maximum dynamic range of an image sensor for a drone camera refers to the range of brightness ratios between the brightest and darkest areas that the sensor can simultaneously record. It is an inherent parameter of the sensor and can be obtained from the drone camera's technical manual or through laboratory calibration. For example, the maximum dynamic range of the CMOS sensor of a certain type of industrial camera is 12EV.

[0084] For example, if the maximum dynamic range of the drone camera image sensor is 12 EV and the currently predicted dynamic range of illumination is 10 EV, then the illumination adjustment coefficient = 10 / 12 ≈ 0.833, indicating that the current illumination details are relatively rich and the search step size needs to be appropriately reduced.

[0085] Furthermore, the adaptive search step size is obtained by multiplying the base search step size by the distance adjustment coefficient and the illumination adjustment coefficient. The adaptive search step size is calculated as: Adaptive Search Step Size = Base Search Step Size × Distance Adjustment Coefficient × Illumination Adjustment Coefficient. This formula combines the combined effects of distance and illumination on the search step size: the distance adjustment coefficient reflects the distance relative to a preset second distance threshold; the closer the distance, the smaller the step size to meet higher accuracy requirements. The illumination adjustment coefficient reflects the richness of current illumination details; the larger the dynamic range of illumination, the smaller the step size to capture sharper peaks. Through the product adjustment of these two factors, the adaptive search step size can dynamically balance search efficiency and positioning accuracy under different distance and illumination conditions, avoiding the problems of excessively coarse step sizes leading to peak loss or excessively fine step sizes resulting in excessive time consumption.

[0086] For example, if the basic search step size is 0.1 mm, the distance adjustment coefficient is 0.8, and the illumination adjustment coefficient is 0.833, then the adaptive search step size = 0.1 × 0.8 × 0.833 ≈ 0.0666 mm. The adaptive search step size takes into account both the need for improved accuracy when the UAV is close to the inspection target (close distance) and the requirement for fine-grained searching when the current illumination details are rich, thus achieving a dynamic balance between search efficiency and accuracy.

[0087] Specifically, the improved multi-directional Sobel operator is used to calculate the sharpness evaluation value of each real-time image frame, forming a sharpness evaluation curve, including: Configure an improved multi-directional Sobel operator, wherein the improved multi-directional Sobel operator includes at least a horizontal direction template, a vertical direction template, a 45-degree diagonal direction template, and a 135-degree diagonal direction template; The real-time image is convolved using the horizontal template, the vertical template, the 45-degree diagonal template, and the 135-degree diagonal template, respectively, to generate corresponding horizontal gradient magnitude images, vertical gradient magnitude images, 45-degree diagonal gradient magnitude images, and 135-degree diagonal gradient magnitude images. The gradient magnitudes at corresponding pixel positions in the horizontal gradient magnitude image, the vertical gradient magnitude image, the 45-degree diagonal gradient magnitude image, and the 135-degree diagonal gradient magnitude image are squared and squared to obtain the comprehensive gradient magnitude of each pixel position, thus generating a comprehensive gradient magnitude image. Calculate the global standard deviation of the real-time image, and multiply the global standard deviation by a preset sensitivity coefficient to obtain the noise suppression threshold; Traverse each pixel position in the comprehensive gradient magnitude image, set the comprehensive gradient magnitude less than the noise suppression threshold to zero, and retain the comprehensive gradient magnitude greater than or equal to the noise suppression threshold to obtain the noise-suppressed gradient magnitude image; The gradient magnitudes of all pixel locations in the noise-suppressed gradient magnitude image are summed to obtain the sharpness evaluation value of the current focus position. The sharpness evaluation value corresponding to each search position is repeatedly calculated to form a sharpness evaluation curve with the focus distance as the horizontal axis and the sharpness evaluation value as the vertical axis.

[0088] In this embodiment, an improved multi-directional Sobel operator is first configured. The improved multi-directional Sobel operator includes at least a horizontal template, a vertical template, a 45-degree diagonal template, and a 135-degree diagonal template. Specifically, the improved multi-directional Sobel operator contains four 3×3 templates: a horizontal template: [[-1,0,1],[-2,0,2],[-1,0,1]], used for detecting horizontal edges; a vertical template: [[-1,-2,-1],[0,0,0],[1,2,1]], used for detecting vertical edges; a 45-degree diagonal template: [[-2,-1,0],[-1,0,1],[0,1,2]], used for detecting 45-degree edges; and a 135-degree diagonal template: [[0,-1,-2],[1,0,-1],[2,1,0]], used for detecting 135-degree edges.

[0089] Thus, compared to the traditional Sobel operator which only considers the horizontal and vertical directions, the improved multi-directional Sobel operator adds two diagonal directions, which can more comprehensively capture edge details in any direction in the image, and is especially suitable for scenarios where the inspected target may have diagonal textures.

[0090] Secondly, convolution operations are performed on the real-time image using horizontal, vertical, 45-degree diagonal, and 135-degree diagonal templates, respectively, to generate corresponding horizontal, vertical, 45-degree diagonal, and 135-degree gradient magnitude images. Specifically, each 3×3 template is used as a convolution kernel, sliding pixel by pixel on the real-time image. For each pixel, the template coefficients are multiplied and accumulated with the corresponding pixel value in the image to obtain the convolution response value of the template at that pixel. The absolute value of this convolution response value reflects the edge strength in that direction; therefore, the absolute value is taken as the gradient magnitude in that direction. By traversing the entire real-time image, gradient magnitude images with the same size as the original image can be obtained.

[0091] For example, for the horizontal template: [[-1,0,1],[-2,0,2],[-1,0,1]], the horizontal gradient magnitude G at the image position (x,y) is... x The formula for calculating (x,y) is: G x(x,y)=|(-1)×I(x-1,y-1)+0×I(x,y-1)+1×I(x+1,y-1)+(-2)×I(x-1,y)+0×I(x,y)+2×I(x+1,y)+(-1)×I(x-1,y+1)+0×I(x,y+1)+1×I(x+1,y+1)|, where I(x,y) represents the pixel grayscale value of the real-time image at (x,y).

[0092] Similarly, the convolution operations for the vertical template, the 45-degree diagonal template, and the 135-degree diagonal template are performed in the same way, and the corresponding gradient magnitude images G are calculated respectively. y G 45 G 135 Finally, four gradient magnitude images with the same size as the original image are generated, corresponding to the edge intensity information in the four directions.

[0093] Optionally, zero padding or mirror filling can be used at image boundaries.

[0094] Next, the gradient magnitudes at corresponding pixel positions in the horizontal, vertical, 45-degree diagonal, and 135-degree diagonal gradient magnitude images are squared and taken as the square root to obtain the comprehensive gradient magnitude for each pixel position, generating a comprehensive gradient magnitude image. The formula for calculating the comprehensive gradient magnitude is: G x G y G 45 G 135 By taking the gradient magnitudes in the four directions—horizontal, vertical, 45-degree diagonal, and 135-degree diagonal—and iterating through all pixel positions in the entire image, a composite gradient magnitude image with the same size as the original image can be generated.

[0095] The integrated gradient magnitude incorporates edge information from four directions, providing a more comprehensive reflection of the edge strength of each pixel in the image. Compared to the traditional Sobel operator, which only uses horizontal and vertical directions, the integrated gradient magnitude is more sensitive to diagonal edges and has a stronger ability to capture image details, thus exhibiting higher accuracy and robustness in sharpness evaluation.

[0096] For example, for insulator strings or transmission lines with diagonal textures in inspection images, the comprehensive gradient amplitude can more realistically reflect their clarity.

[0097] Furthermore, the global standard deviation of the real-time image is calculated, and then multiplied by a preset sensitivity coefficient to obtain the noise suppression threshold. Specifically, the global standard deviation is calculated for the composite gradient magnitude image, rather than the original grayscale image. This is because the composite gradient magnitude image has already fused gradient information from four directions, and its pixel value distribution can more intuitively reflect the statistical characteristics of edge intensity in the image.

[0098] The global standard deviation reflects the dispersion of the overall pixel values ​​of a real-time image, i.e., image contrast. In sharpness evaluation, it is necessary to suppress false gradients caused by noise and avoid misjudging them as edges. Therefore, it is necessary to set a noise suppression threshold and regard gradient magnitudes below the noise suppression threshold as noise or flat areas and not participate in sharpness calculation.

[0099] The formula for calculating the global standard deviation is as follows: ,in, Let μ be the gradient magnitude of the i-th pixel in the composite gradient magnitude image, μ be the average gradient magnitude of the composite gradient magnitude image, and N be the total number of pixels.

[0100] Specifically, the calculated global standard deviation is multiplied by a preset sensitivity coefficient to obtain the noise suppression threshold. The preset sensitivity coefficient is a parameter used to control the stringency of noise suppression: a larger preset sensitivity coefficient results in a higher noise suppression threshold and stronger noise suppression, but may also filter out some weak edges; a smaller preset sensitivity coefficient results in a lower noise suppression threshold and retains more details, but also increases noise residue. Those skilled in the art can set the sensitivity coefficient based on experience, considering the noise level of the image sensor, the texture features of the inspected target, and the accuracy requirements for sharpness evaluation in practical applications, or determine its value through experimental calibration. Preferably, the preset sensitivity coefficient can be set between 0.05 and 0.2.

[0101] The noise suppression threshold serves as a filtering condition: pixels with a comprehensive gradient magnitude lower than the noise suppression threshold are considered noise or flat regions and are not included in the subsequent sharpness evaluation summation calculation, thereby improving the robustness of the sharpness evaluation.

[0102] For example, if the global standard deviation of the overall gradient magnitude image is 50 and the preset sensitivity coefficient is 0.1, then the noise suppression threshold = 50 × 0.1 = 5. In subsequent processing, only pixels with an overall gradient magnitude greater than or equal to 5 are included in the sharpness evaluation value, thereby effectively filtering out noise interference.

[0103] Furthermore, by iterating through each pixel in the composite gradient magnitude image, composite gradient magnitudes less than the noise suppression threshold are set to zero, while those greater than or equal to the noise suppression threshold are retained, resulting in a noise-suppressed gradient magnitude image. Specifically, a hard thresholding process is applied to the composite gradient magnitude image: for each pixel, if its composite gradient magnitude is less than the noise suppression threshold, the gradient at that point is set to 0; otherwise, the original value is retained. This effectively filters out weak gradients caused by noise, allowing the sharpness evaluation to focus more on the true edges.

[0104] Furthermore, the gradient magnitudes of all pixel locations in the noise-suppressed gradient magnitude image are summed to obtain the sharpness evaluation value for the current focus position. Here, the sharpness evaluation value = Σ gradient magnitude, which is the total intensity of all effective edges in the image. A larger sharpness evaluation value indicates sharper image edges, richer details, and more accurate focus.

[0105] Finally, the sharpness evaluation value for each search position is repeatedly calculated to form a sharpness evaluation curve with focus distance as the x-axis and sharpness evaluation value as the y-axis. Specifically, as the lens moves, images are acquired and sharpness evaluation values ​​are calculated at each position, ultimately resulting in a curve reflecting the change in sharpness with focus distance, i.e., the sharpness evaluation curve.

[0106] In summary, compared to existing technologies, this application locks the coarse shutter speed when the distance between the UAV and the inspection target is less than a preset second distance threshold. Using the initial focusing distance as the search starting point, an improved multi-directional Sobel operator is employed as the sharpness evaluation function to perform focus search. The fine-tuning focusing distance is then locked at the peak point of the sharpness evaluation value. Thus, by locking the shutter speed to eliminate brightness interference, balancing search efficiency and accuracy through adaptive step size, and improving the accuracy of sharpness evaluation through multi-directional gradient fusion and noise suppression, fast and accurate focusing is ultimately achieved under complex lighting and distance conditions.

[0107] S40: Unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed according to the analysis results to obtain the fine shutter speed.

[0108] After fine-tuning the focus, the target area of ​​the image is clear. However, the shutter speed at this point (coarse shutter speed) is based on global brightness adjustment of the image, which may not guarantee that the brightness of the target area itself is optimal. Since the lighting conditions of the target area and the background may differ, for example, the target may be in shadow while the background is under direct sunlight. An image with appropriate global brightness may make the target area too dark or too bright, affecting the feature extraction of subsequent recognition algorithms.

[0109] To address the aforementioned issues, this application unlocks the coarse shutter speed adjustment, extracts the region of interest corresponding to the inspected target type, performs local histogram analysis, and fine-tunes the coarse shutter speed based on the analysis results to obtain a fine-tuned shutter speed. This optimizes the exposure of the inspected target area, thereby further improving the recognition success rate.

[0110] Specifically, step S40 in the method includes: After locking the fine-tuning focus distance, the coarse-tuning shutter speed is unlocked, restoring the coarse-tuning shutter speed to an adjustable state; Based on the type of the inspection target, a polygonal region containing the inspection target is segmented from the real-time image of the current frame acquired after the focus search is completed, and is taken as the region of interest. Centered on the coarse shutter speed, multiple candidate shutter speeds are generated within a preset search neighborhood to form a candidate solution set; For each iteration, a real-time image is acquired sequentially according to each candidate shutter speed in the candidate solution set, and the region of interest corresponding to the inspection target type is extracted from the real-time image. The average brightness value of the region of interest is calculated based on the local histogram features of the region of interest, and the absolute value of the difference between the average brightness value and the preset target brightness is used as the fitness value of the candidate shutter speed. Based on the fitness values ​​of all candidate shutter speeds in the current iteration round, record the minimum fitness value of each candidate shutter speed in the historical iteration process, and use it as the historical best fitness value of the candidate shutter speed. Then, take the candidate shutter speed corresponding to the historical best fitness value as the historical best position of the candidate shutter speed. Record the global minimum fitness value of all candidate shutter speeds in the historical iteration process, and use it as the global historical optimal fitness value. Then, use the candidate shutter speed corresponding to the global historical optimal fitness value as the global historical optimal position. Based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain the updated candidate solution set. Repeat the iterative process until the preset convergence condition is met, and use the candidate shutter speed corresponding to the global historical best position obtained from the final iteration as the fine-tuning shutter speed.

[0111] In this embodiment, after locking the fine-tuning focus distance, the coarse-tuning shutter speed is unlocked, restoring it to an adjustable state. Specifically, after step S30 completes the fine-tuning focus and locks the fine-tuning focus distance, the shutter speed is unlocked, restoring it to a dynamically adjustable state. This creates conditions for subsequent fine-tuning of local brightness based on the region of interest.

[0112] Secondly, based on the type of the inspected target, a polygonal region containing the inspected target is segmented from the real-time image of the current frame acquired after the focus search is completed, serving as the region of interest. Specifically, after the focus search is completed, the UAV acquires a real-time image frame, which already has a relatively clear focus. Based on the pre-defined type of inspected target (e.g., insulators, transmission lines, towers, etc.), a target detection algorithm segments the polygonal region containing the inspected target from this image, serving as the region of interest. The region of interest refers to the pixel area occupied by the inspected target in the image; the brightness of this area directly affects the subsequent target recognition results.

[0113] For example, polygonal regions containing inspection targets can be segmented from the real-time image of the current frame using object detection algorithms. For instance, an object detection model based on the YOLOv8 architecture can be used. This model is pre-tuned to fit the inspection scenario: 1. Collect a large number of image samples containing typical inspection targets such as insulators, transmission lines, and towers, and use annotation tools such as LabelImg to annotate the bounding boxes of targets in each image sample, forming a training dataset; 2. Starting with the pre-trained weights of YOLOv8 on the COCO dataset, supervised fine-tuning is performed using this training dataset. For example, the initial learning rate can be set to 0.001, the batch size to 16, and 50 training iterations can be performed. Backpropagation is used to optimize the model parameters so that it can accurately identify specific inspection targets, thus obtaining a finely tuned object detection model adapted to the scenario.

[0114] Next, using the coarse shutter speed as the center, multiple candidate shutter speeds are generated within a preset search neighborhood to form a candidate solution set. Specifically, the coarse shutter speed obtained in step S20 is used as the center value, and a certain range is extended above and below it (e.g., 1 EV above and below). N candidate shutter speeds are generated by uniformly sampling according to the shutter speed levels supported by the camera, forming a candidate solution set for subsequent iterative optimization.

[0115] The preset search neighborhood refers to the range of optical parameters used to limit the fine-tuning range. Its size determines the search range of candidate shutter speeds and can be dynamically set by those skilled in the art based on the camera's exposure adjustment accuracy, the magnitude of changes in scene lighting, and the sensitivity of the target area to exposure errors. Furthermore, a preset search neighborhood that is too small may not cover the optimal solution, while a preset search neighborhood that is too large will increase the search time.

[0116] For example, if the coarse shutter speed is set to 1 / 400 second and the preset search neighborhood is set to 1 EV above and below, then candidate shutter speeds such as 1 / 800, 1 / 640, 1 / 500, 1 / 400, 1 / 320, 1 / 250, 1 / 200, 1 / 160, 1 / 125, and 1 / 100 second (the closest speed supported by the camera) can be generated to form a candidate solution set.

[0117] Furthermore, for each iteration, a real-time image is acquired sequentially based on each candidate shutter speed in the candidate solution set. The region of interest (ROI) corresponding to the type of the inspected target is extracted from the real-time image. The average brightness value of the ROI is calculated based on its local histogram features. The absolute value of the difference between the average brightness value and the preset target brightness is used as the fitness value of that candidate shutter speed. Specifically, in each iteration, the camera is controlled to acquire a real-time image sequentially using each candidate shutter speed in the candidate solution set. For each acquired image, the ROI (i.e., the polygonal region where the inspected target is located) is first extracted using the same method as described above. Then, the grayscale values ​​of all pixels within the ROI are statistically analyzed to generate a grayscale histogram: each pixel within the region is traversed, and its grayscale value (0-255) is used as an index. The frequency of occurrence of this grayscale value is accumulated to obtain the pixel distribution of each grayscale level, which is the local histogram feature. Based on the local histogram features, the average brightness value of the region of interest is calculated using the arithmetic mean formula. The calculated average brightness value is then subtracted from the preset target brightness determined in step S20, and the absolute value is taken to obtain the fitness value of the candidate shutter speed. The smaller the fitness value, the closer the brightness of the inspected target area is to the ideal value.

[0118] For example, if the candidate shutter speed is 1 / 320 second, and the average brightness value of the region of interest is 118 after acquiring a real-time image, and the preset target brightness is 125, then the fitness value = |118-125| = 7.

[0119] Furthermore, based on the fitness values ​​of all candidate shutter speeds in the current iteration, the minimum fitness value of each candidate shutter speed during the historical iteration process is recorded as the historical best fitness value of the candidate shutter speed, and the candidate shutter speed corresponding to the historical best fitness value is taken as the historical best position of the candidate shutter speed. Specifically, for each candidate shutter speed in the candidate solution set, its historical minimum fitness value, i.e., its individual optimum, is maintained. The shutter speed value corresponding to this minimum fitness value is also recorded, called the "historical best fitness value" of that candidate shutter speed. In each iteration, if the current fitness value is less than this historical best fitness value, its historical best value and optimal position are updated.

[0120] Furthermore, the global minimum fitness value of all candidate shutter speeds during the historical iteration process is recorded as the global historical optimal fitness value, and the candidate shutter speed corresponding to the global historical optimal fitness value is recorded as the global historical optimal position. Specifically, throughout the entire iteration process, the global minimum fitness value that has appeared among all candidate shutter speeds is recorded, called the "global historical optimal fitness value," and the shutter speed corresponding to this global historical optimal fitness value is recorded, called the "global historical optimal position." The global historical optimal position is the best solution found by all candidate shutter speeds in history.

[0121] Furthermore, based on the current value of each candidate shutter speed, its historical best position, and the global historical best position, each candidate shutter speed in the candidate solution set is updated, resulting in an updated candidate solution set. Specifically, the update of each candidate shutter speed is jointly determined by its own historical best position and the group's historical best position. Each candidate shutter speed can utilize its own search experience and also draw on the group's optimal experience, thus quickly converging to the global optimal solution.

[0122] Finally, the iterative process is repeated until the preset convergence condition is met. The candidate shutter speed corresponding to the global historical best position obtained from the final iteration is used as the fine-tuned shutter speed. The preset convergence condition refers to the criterion for terminating the iteration, used to ensure that the iteration stops promptly after reaching a satisfactory solution, avoiding unnecessary computation. The preset convergence condition can be set according to actual application needs. For example, a maximum number of iterations can be preset (e.g., 30 times) to ensure that the algorithm completes within a finite time, or the global optimal fitness value can be set to no longer decrease for multiple consecutive iterations (e.g., 5 rounds), indicating that a stable solution has been converged.

[0123] Specifically, the above steps are repeated until the preset maximum number of iterations is reached, or the global optimal fitness value no longer decreases after several consecutive rounds, at which point the iteration stops. The shutter speed corresponding to the final globally optimal historical position is then the fine-tuned shutter speed. Fine-tuning the shutter speed ensures that the brightness of the inspected target area is closest to the preset target brightness, thus providing the best exposure parameters for subsequent inspection photography.

[0124] Specifically, based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain an updated candidate solution set, including: Based on the current iteration round and the preset total iteration round, calculate the first weight coefficient and the second weight coefficient, wherein the first weight coefficient decreases as the iteration round increases, and the second weight coefficient increases as the iteration round increases; For each candidate shutter speed, calculate the first difference between the current value of the candidate shutter speed and the historical best position, and multiply the first difference by the first weighting coefficient to obtain the first update component; Calculate the second difference between the current value of the candidate shutter speed and the global historical best position, and multiply the second difference by the second weighting coefficient to obtain the second update component; Add the first update component to the second update component to obtain the total update amount; Based on the magnitude and direction of the total update amount, the current value of the candidate shutter speed is adjusted to obtain the updated candidate shutter speed; The updated candidate shutter speeds corresponding to all candidate shutter speeds are used to form the updated candidate solution set.

[0125] In this embodiment, a first weighting coefficient and a second weighting coefficient are first calculated based on the current iteration round and the preset total iteration rounds. The first weighting coefficient decreases with increasing iteration rounds, while the second weighting coefficient increases. Specifically, in the early stages of iteration, the first weighting coefficient is set to a larger value, causing candidate shutter speeds to tend to move towards their own historical best position, thus fully exploring individual experience. As the iteration rounds increase, the first weighting coefficient gradually decreases, while the second weighting coefficient gradually increases, causing candidate shutter speeds to tend to move towards the global historical best position, thus utilizing collective experience to accelerate convergence. The sum of the first and second weighting coefficients is 1 to ensure the stability of the update step size.

[0126] For example, let the current iteration round be t, and the preset total iteration rounds be T, then the first weight coefficient can be expressed as: The second weighting coefficient can be expressed as: ,in, and These are the initial maximum and final minimum values ​​of the first weighting coefficient, respectively. and These are the initial minimum and final maximum values ​​of the second weighting coefficient, respectively.

[0127] A better location is available. =0.9, =0.4, =0.6, =0.1, and satisfies + =1、 + =1. This design results in a larger first weight coefficient in the early stages of iteration, causing candidate shutter speeds to tend to move towards their own historical best positions, fully exploring individual experience. As the number of iterations increases, the first weight coefficient gradually decreases, while the second weight coefficient gradually increases, causing candidate shutter speeds to tend to move towards the global historical best positions, utilizing collective experience to accelerate convergence. This achieves global exploration in the early stages of iteration and local fine-tuning search in the later stages, balancing global exploration and local development. Those skilled in the art can also dynamically adjust this design according to the actual application scenario. , , , The value of .

[0128] Secondly, for each candidate shutter speed, the first difference between the current value of the candidate shutter speed and the historical best position is calculated, and the first difference is multiplied by the first weight coefficient to obtain the first update component. The first update component = (current value of candidate shutter speed - historical best position) × first weight coefficient, and the first update component reflects the adjustment direction and magnitude of the candidate shutter speed based on its own search experience.

[0129] Next, calculate the second difference between the current value of the candidate shutter speed and the global historical best position, and multiply the second difference by the second weighting coefficient to obtain the second update component. The second update component = (current value of candidate shutter speed - global historical best position) × second weighting coefficient. The second update component reflects the adjustment direction and magnitude of the candidate shutter speed based on the group's optimal experience.

[0130] It should be noted that the global historical best position is defined in the aforementioned steps as "taking the candidate shutter speed corresponding to the global historical best fitness value as the global historical best position", that is, the essence of the global historical best position is shutter speed.

[0131] Furthermore, the first update component is added to the second update component to obtain the total update amount. The total update amount combines the combined influence of individual experience and group experience on candidate shutter speeds, preserving its own historical search advantages while also drawing on the optimization directions of other candidates in the group.

[0132] Finally, based on the magnitude and direction of the total update, the current values ​​of the candidate shutter speeds are adjusted to obtain updated candidate shutter speeds. All updated candidate shutter speeds corresponding to the current values ​​of the candidate shutter speeds constitute the updated candidate solution set. Specifically, the current values ​​of the candidate shutter speeds are added to the total update, resulting in updated candidate shutter speeds. All updated candidate shutter speeds then constitute the updated candidate solution set.

[0133] It should be noted that if the calculated updated candidate shutter speed exceeds the range of shutter speeds supported by the camera, it will be limited to the boundary value and rounded to the nearest available shutter speed.

[0134] In summary, compared to existing technologies, this application unlocks the coarse shutter speed adjustment, extracts the region of interest corresponding to the inspection target type, performs local histogram analysis, and fine-tunes the coarse shutter speed based on the analysis results to obtain a fine shutter speed. Thus, by focusing brightness adjustment from the global perspective to the inspection target area and iteratively searching for the optimal shutter speed within this local range, the problem of adequate global brightness but poor exposure in the target area is effectively solved, improving the image quality of the inspection target area.

[0135] S50: Control the drone to perform inspection shooting according to the fine-tuned focus distance and the fine-tuned shutter speed.

[0136] In this embodiment, the fine-tuned focus distance and fine-tuned shutter speed are used as the final camera parameters and sent to the UAV camera control system. The camera is controlled to perform inspection shooting according to these parameters, and clear, appropriately bright inspection images are collected. This provides high-quality data support for subsequent image anomaly detection based on generative adversarial networks and transfer learning, and adapts to the needs of UAV power line inspection in complex environments at high altitudes.

[0137] In summary, the embodiments of this application have at least the following technical effects: Compared to existing technologies, this application first predicts scene illumination parameters by acquiring environmental data of the area where the UAV inspection mission is located, and then obtains the initial shutter speed and initial focus distance by matching the inspection target type. This achieves pre-coarse adjustment of camera parameters, providing a reasonable starting point for subsequent dynamic adjustments and avoiding blind searching from default parameters, effectively improving the overall efficiency of parameter adjustment. Secondly, in the first stage of the UAV approaching the target, the initial shutter speed is dynamically adjusted based on the brightness deviation between the global image brightness and the preset target brightness. Through closed-loop feedback, the image brightness is quickly converged to the sensor's linear response range, obtaining a coarsely adjusted shutter speed. This provides a stable image foundation for subsequent focus search and overcomes the impact of illumination changes during flight on image quality. Secondly, in the second stage of further approaching the target, after locking the coarse shutter speed, the initial focusing distance is used as the search starting point. An improved multi-directional Sobel operator is used as the sharpness evaluation function to perform the focus search. Multi-directional gradient information fusion is used to improve the accuracy and noise resistance of the sharpness evaluation. Combined with an adaptive search step size, the fine-tuned focusing distance is locked at the peak point of the sharpness evaluation value to ensure the target image is sharp, solving the problem that traditional single-directional gradient evaluation is susceptible to noise interference. Finally, after unlocking the coarse shutter speed, the region of interest corresponding to the inspected target type is extracted and local histogram analysis is performed. The fine-tuned shutter speed is obtained through particle swarm optimization iteration, which optimizes the brightness of the target area, further improving the target recognition success rate and overcoming the defect of appropriate global brightness but poor exposure of the target area.

[0138] Through the above technical solutions, this application has constructed a dynamic adjustment mechanism for the entire process, from environmental prediction, coarse brightness adjustment, fine focus adjustment to local brightness fine adjustment. It effectively solves the technical problems of improper image exposure and inaccurate focus caused by large dynamic range of illumination, uneven distribution, and dynamic changes in target distance during flight. It significantly improves the brightness uniformity, clarity and target recognition accuracy of inspection images, and provides reliable technical support for intelligent inspection of UAVs.

[0139] Example 2, as Figure 2 As shown, based on the same inventive concept as the camera parameter dynamic adjustment method for UAV intelligent recognition provided in Embodiment 1, this embodiment of the invention also provides a camera parameter dynamic adjustment system for UAV intelligent recognition, including: The parameter initialization module 11 is used to predict scene illumination parameters based on environmental data of the area where the UAV inspection mission is located, and to obtain the initial shutter speed and initial focus distance according to the scene illumination parameters and the inspection target type. The shutter speed coarse adjustment module 12 is used to start real-time image acquisition when the distance between the UAV and the inspection target is less than a preset first distance threshold, and adjust the initial shutter speed based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain the coarse shutter speed. The focus distance search module 13 is used to lock the coarse shutter speed when the distance between the UAV and the inspection target is less than the preset second distance threshold, take the initial focus distance as the search starting point, use the improved multi-directional Sobel operator as the sharpness evaluation function and perform focus search, and lock the fine focus distance at the peak point of the sharpness evaluation value. The shutter speed fine-tuning module 14 is used to unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed according to the analysis results to obtain the fine shutter speed. The parameter output module 15 is used to control the UAV to perform inspection shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

[0140] Specifically, the parameter initialization module 11 is used for: The system acquires environmental data for the area where the drone inspection mission is located, including weather forecast data, satellite cloud image data, geographic coordinate information, and current time information. The environmental data is input into a pre-trained illumination prediction model, and scene illumination parameters are output. The scene illumination parameters include illumination dynamic range and illumination principal direction intensity ratio. The illumination dynamic range is used to characterize the brightness difference between the brightest and darkest areas in the area where the UAV inspection task is located. The illumination principal direction intensity ratio is used to characterize the ratio between the illumination intensity in the light source direction and the global average illumination intensity. The training process for the illumination prediction model includes: Collect environmental data samples from historical inspection areas to form an environmental data training sample set; For each environmental data sample in the environmental data training sample set, obtain the corresponding dynamic range of illumination and the intensity ratio of the main direction of illumination to form an illumination parameter annotation set. An initial illumination prediction model is constructed based on a convolutional neural network; Using the environmental data training sample set as input and the illumination parameter annotation set as supervision labels, the initialized illumination prediction model is trained in a supervised manner until the verification convergence is achieved, thus obtaining the pre-trained illumination prediction model.

[0141] Specifically, the initial shutter speed and initial focus distance are obtained by matching the scene lighting parameters with the inspection target type, including: Pre-store the recommended shutter speed and recommended focus distance corresponding to the combination of dynamic range of illumination, intensity ratio of main direction of illumination and type of inspection target, and construct a camera parameter mapping table; Using the dynamic range of illumination, the ratio of illumination intensity in the main direction, and the type of the inspection target in the scene illumination parameters as joint query conditions, the corresponding recommended shutter speed and recommended focus distance are obtained from the camera parameter mapping table and used as the initial shutter speed and initial focus distance, respectively.

[0142] Specifically, the shutter speed coarse adjustment module 12 is used for: When the distance between the UAV and the inspection target is less than a preset first distance threshold, periodic real-time image acquisition is initiated. Global histogram features are extracted for each frame of real-time image acquired, and the average brightness value of the current image is calculated based on the global histogram features. Obtain the linear response range of the camera image sensor on the drone, calculate the median brightness of the linear response range, and use the median brightness as the preset target brightness; The difference between the average brightness value of the current real-time image and the preset target brightness is calculated to obtain the brightness deviation; Based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed. The next real-time image is re-acquired at the adjusted shutter speed, and the average brightness value and new brightness deviation of the next real-time image are calculated. The step of dynamically adjusting the shutter speed according to the brightness deviation is repeated until the average brightness value converges to the preset target brightness. The shutter speed at which convergence occurs is used as the coarse adjustment shutter speed.

[0143] Furthermore, based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed, including: The brightness deviation is used as a feedback quantity, and the current adjustment step size is dynamically determined based on the absolute value of the brightness deviation, wherein the current adjustment step size is positively correlated with the absolute value of the brightness deviation. According to the positive or negative direction of the brightness deviation, the initial shutter speed is adjusted with the current adjustment step size to obtain the adjusted shutter speed: if the brightness deviation is positive, the initial shutter speed is decreased according to the current adjustment step size; if the brightness deviation is negative, the initial shutter speed is increased according to the current adjustment step size.

[0144] Specifically, the focus distance search module 13 is used for: When the distance between the drone and the inspection target is less than the preset second distance threshold, the coarse shutter speed is locked to a fixed value, and the adjustment of the shutter speed is stopped. The real-time distance and dynamic range of illumination between the current UAV and the inspection target are obtained, and the adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination. Using the initial focusing distance as the search starting point, the lens is driven to move along the focusing direction according to the adaptive search step size. A real-time image is acquired at each search position, and the sharpness evaluation value of each real-time image is calculated using the improved multi-directional Sobel operator to form a sharpness evaluation curve. Search for the peak point of the sharpness evaluation value in the sharpness evaluation curve, and lock the focus distance corresponding to the peak point as the fine-tuning focus distance; The adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination, including: The basic search step size is preset based on the stepping accuracy of the lens focusing motor and the focal length of the lens. Calculate the ratio of the real-time distance to the preset second distance threshold to obtain the distance adjustment coefficient; The ratio of the illumination dynamic range to the maximum dynamic range of the UAV camera image sensor is calculated to obtain the illumination adjustment coefficient; The adaptive search step size is obtained by multiplying the basic search step size by the distance adjustment coefficient and the illumination adjustment coefficient.

[0145] Furthermore, an improved multi-directional Sobel operator is used to calculate the sharpness evaluation value of each real-time image frame, forming a sharpness evaluation curve, including: Configure an improved multi-directional Sobel operator, wherein the improved multi-directional Sobel operator includes at least a horizontal direction template, a vertical direction template, a 45-degree diagonal direction template, and a 135-degree diagonal direction template; The real-time image is convolved using the horizontal template, the vertical template, the 45-degree diagonal template, and the 135-degree diagonal template, respectively, to generate corresponding horizontal gradient magnitude images, vertical gradient magnitude images, 45-degree diagonal gradient magnitude images, and 135-degree diagonal gradient magnitude images. The gradient magnitudes at corresponding pixel positions in the horizontal gradient magnitude image, the vertical gradient magnitude image, the 45-degree diagonal gradient magnitude image, and the 135-degree diagonal gradient magnitude image are squared and squared to obtain the comprehensive gradient magnitude of each pixel position, thus generating a comprehensive gradient magnitude image. Calculate the global standard deviation of the real-time image, and multiply the global standard deviation by a preset sensitivity coefficient to obtain the noise suppression threshold; Traverse each pixel position in the comprehensive gradient magnitude image, set the comprehensive gradient magnitude less than the noise suppression threshold to zero, and retain the comprehensive gradient magnitude greater than or equal to the noise suppression threshold to obtain the noise-suppressed gradient magnitude image; The gradient magnitudes of all pixel locations in the noise-suppressed gradient magnitude image are summed to obtain the sharpness evaluation value of the current focus position. The sharpness evaluation value corresponding to each search position is repeatedly calculated to form a sharpness evaluation curve with the focus distance as the horizontal axis and the sharpness evaluation value as the vertical axis.

[0146] Specifically, the shutter speed fine-tuning module 14 is used for: After locking the fine-tuning focus distance, the coarse-tuning shutter speed is unlocked, restoring the coarse-tuning shutter speed to an adjustable state; Based on the type of the inspection target, a polygonal region containing the inspection target is segmented from the real-time image of the current frame acquired after the focus search is completed, and is taken as the region of interest. Centered on the coarse shutter speed, multiple candidate shutter speeds are generated within a preset search neighborhood to form a candidate solution set; For each iteration, a real-time image is acquired sequentially according to each candidate shutter speed in the candidate solution set, and the region of interest corresponding to the inspection target type is extracted from the real-time image. The average brightness value of the region of interest is calculated based on the local histogram features of the region of interest, and the absolute value of the difference between the average brightness value and the preset target brightness is used as the fitness value of the candidate shutter speed. Based on the fitness values ​​of all candidate shutter speeds in the current iteration round, record the minimum fitness value of each candidate shutter speed in the historical iteration process, and use it as the historical best fitness value of the candidate shutter speed. Then, take the candidate shutter speed corresponding to the historical best fitness value as the historical best position of the candidate shutter speed. Record the global minimum fitness value of all candidate shutter speeds in the historical iteration process, and use it as the global historical optimal fitness value. Then, use the candidate shutter speed corresponding to the global historical optimal fitness value as the global historical optimal position. Based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain the updated candidate solution set. Repeat the iterative process until the preset convergence condition is met, and use the candidate shutter speed corresponding to the global historical best position obtained from the final iteration as the fine-tuning shutter speed.

[0147] Specifically, based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain an updated candidate solution set, including: Based on the current iteration round and the preset total iteration round, calculate the first weight coefficient and the second weight coefficient, wherein the first weight coefficient decreases as the iteration round increases, and the second weight coefficient increases as the iteration round increases; For each candidate shutter speed, calculate the first difference between the current value of the candidate shutter speed and the historical best position, and multiply the first difference by the first weighting coefficient to obtain the first update component; Calculate the second difference between the current value of the candidate shutter speed and the global historical best position, and multiply the second difference by the second weighting coefficient to obtain the second update component; Add the first update component to the second update component to obtain the total update amount; Based on the magnitude and direction of the total update amount, the current value of the candidate shutter speed is adjusted to obtain the updated candidate shutter speed; The updated candidate shutter speeds corresponding to all candidate shutter speeds are used to form the updated candidate solution set.

[0148] Specifically, the parameter output module 15 is used for: The drone is controlled to perform inspection and shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

[0149] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0150] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0152] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0153] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0154] Although preferred embodiments of the invention have been described, those skilled in the art, once they have learned the basic inventive concept, can make other changes and modifications to these embodiments.

[0155] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of this invention and its equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for dynamically adjusting camera parameters for intelligent recognition in unmanned aerial vehicles (UAVs), characterized in that, The method includes: Scene illumination parameters are predicted based on environmental data of the area where the UAV inspection mission is located, and the initial shutter speed and initial focus distance are obtained by matching the scene illumination parameters with the inspection target type. When the distance between the drone and the inspection target is less than a preset first distance threshold, real-time image acquisition is initiated, and the initial shutter speed is adjusted based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain a coarse shutter speed. When the distance between the drone and the inspection target is less than the preset second distance threshold, the coarse shutter speed is locked, the initial focus distance is used as the search starting point, the improved multi-directional Sobel operator is used as the sharpness evaluation function and the focus search is performed, and the fine focus distance is locked at the peak point of the sharpness evaluation value. Unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed based on the analysis results to obtain the fine shutter speed; The drone is controlled to perform inspection and shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

2. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 1, characterized in that, Scene illumination parameters are predicted based on environmental data of the area where the drone inspection mission is located, including: The system acquires environmental data for the area where the drone inspection mission is located, including weather forecast data, satellite cloud image data, geographic coordinate information, and current time information. The environmental data is input into a pre-trained illumination prediction model, and scene illumination parameters are output. The scene illumination parameters include illumination dynamic range and illumination principal direction intensity ratio. The illumination dynamic range is used to characterize the brightness difference between the brightest and darkest areas in the area where the UAV inspection task is located. The illumination principal direction intensity ratio is used to characterize the ratio between the illumination intensity in the light source direction and the global average illumination intensity. The training process for the illumination prediction model includes: Collect environmental data samples from historical inspection areas to form an environmental data training sample set; For each environmental data sample in the environmental data training sample set, obtain the corresponding dynamic range of illumination and the intensity ratio of the main direction of illumination to form an illumination parameter annotation set. An initial illumination prediction model is constructed based on a convolutional neural network; Using the environmental data training sample set as input and the illumination parameter annotation set as supervision labels, the initialized illumination prediction model is trained in a supervised manner until the verification convergence is achieved, thus obtaining the pre-trained illumination prediction model.

3. The method for dynamic adjustment of camera parameters for intelligent recognition of unmanned aerial vehicles according to claim 1, characterized in that, The initial shutter speed and initial focus distance are obtained by matching the scene lighting parameters with the inspection target type, including: Pre-store the recommended shutter speed and recommended focus distance corresponding to the combination of dynamic range of illumination, intensity ratio of main direction of illumination and type of inspection target, and construct a camera parameter mapping table; Using the dynamic range of illumination, the ratio of illumination intensity in the main direction, and the type of the target being inspected as joint query conditions, the corresponding recommended shutter speed and recommended focus distance are obtained from the camera parameter mapping table and used as the initial shutter speed and initial focus distance, respectively.

4. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 1, characterized in that, When the drone flies to a distance less than a preset first distance threshold from the inspection target, real-time image acquisition is initiated, and the initial shutter speed is adjusted based on the brightness deviation between the global image brightness and the preset target brightness to obtain a coarse shutter speed, including: When the distance between the UAV and the inspection target is less than a preset first distance threshold, periodic real-time image acquisition is initiated. Global histogram features are extracted for each frame of real-time image acquired, and the average brightness value of the current image is calculated based on the global histogram features. Obtain the linear response range of the camera image sensor on the drone, calculate the median brightness of the linear response range, and use the median brightness as the preset target brightness; The difference between the average brightness value of the current real-time image and the preset target brightness is calculated to obtain the brightness deviation; Based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed. The next real-time image is re-acquired at the adjusted shutter speed, and the average brightness value and new brightness deviation of the next real-time image are calculated. The step of dynamically adjusting the shutter speed according to the brightness deviation is repeated until the average brightness value converges to the preset target brightness. The shutter speed at which convergence occurs is used as the coarse adjustment shutter speed.

5. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 4, characterized in that, Based on the sign and magnitude of the brightness deviation, the initial shutter speed is dynamically adjusted to obtain the adjusted shutter speed, including: The brightness deviation is used as a feedback quantity, and the current adjustment step size is dynamically determined based on the absolute value of the brightness deviation, wherein the current adjustment step size is positively correlated with the absolute value of the brightness deviation. According to the positive or negative direction of the brightness deviation, the initial shutter speed is adjusted with the current adjustment step size to obtain the adjusted shutter speed: if the brightness deviation is positive, the initial shutter speed is decreased according to the current adjustment step size; if the brightness deviation is negative, the initial shutter speed is increased according to the current adjustment step size.

6. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 1, characterized in that, When the drone flies to a distance less than a preset second distance threshold from the inspection target, the coarse shutter speed is locked. Using the initial focus distance as the search starting point, an improved multi-directional Sobel operator is employed as the sharpness evaluation function to perform a focus search. At the peak point of the sharpness evaluation value, the fine focus distance is locked, including: When the distance between the drone and the inspection target is less than the preset second distance threshold, the coarse shutter speed is locked to a fixed value, and the adjustment of the shutter speed is stopped. The real-time distance and dynamic range of illumination between the current UAV and the inspection target are obtained, and the adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination. Using the initial focusing distance as the search starting point, the lens is driven to move along the focusing direction according to the adaptive search step size. At each search position, a real-time image is acquired, and the sharpness evaluation value of each real-time image is calculated using the improved multi-directional Sobel operator to form a sharpness evaluation curve. Search for the peak point of the sharpness evaluation value in the sharpness evaluation curve, and lock the focus distance corresponding to the peak point as the fine-tuning focus distance; The adaptive search step size is calculated based on the real-time distance and the dynamic range of illumination, including: The basic search step size is preset based on the stepping accuracy of the lens focusing motor and the focal length of the lens. Calculate the ratio of the real-time distance to the preset second distance threshold to obtain the distance adjustment coefficient; The ratio of the illumination dynamic range to the maximum dynamic range of the UAV camera image sensor is calculated to obtain the illumination adjustment coefficient; The adaptive search step size is obtained by multiplying the basic search step size by the distance adjustment coefficient and the illumination adjustment coefficient.

7. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 6, characterized in that, An improved multi-directional Sobel operator is used to calculate the sharpness evaluation value of each real-time image frame, forming a sharpness evaluation curve, including: Configure an improved multi-directional Sobel operator, wherein the improved multi-directional Sobel operator includes at least a horizontal direction template, a vertical direction template, a 45-degree diagonal direction template, and a 135-degree diagonal direction template; The real-time image is convolved using the horizontal template, the vertical template, the 45-degree diagonal template, and the 135-degree diagonal template, respectively, to generate corresponding horizontal gradient magnitude images, vertical gradient magnitude images, 45-degree diagonal gradient magnitude images, and 135-degree diagonal gradient magnitude images. The gradient magnitudes at corresponding pixel positions in the horizontal gradient magnitude image, the vertical gradient magnitude image, the 45-degree diagonal gradient magnitude image, and the 135-degree diagonal gradient magnitude image are squared and squared to obtain the comprehensive gradient magnitude of each pixel position, thus generating a comprehensive gradient magnitude image. Calculate the global standard deviation of the real-time image, and multiply the global standard deviation by a preset sensitivity coefficient to obtain the noise suppression threshold; Traverse each pixel position in the comprehensive gradient magnitude image, set the comprehensive gradient magnitude less than the noise suppression threshold to zero, and retain the comprehensive gradient magnitude greater than or equal to the noise suppression threshold to obtain the noise-suppressed gradient magnitude image; The gradient magnitudes of all pixel locations in the noise-suppressed gradient magnitude image are summed to obtain the sharpness evaluation value of the current focus position. The sharpness evaluation value corresponding to each search position is repeatedly calculated to form a sharpness evaluation curve with the focus distance as the horizontal axis and the sharpness evaluation value as the vertical axis.

8. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 1, characterized in that, Unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed based on the analysis results to obtain the fine shutter speed, including: After locking the fine-tuning focus distance, the coarse-tuning shutter speed is unlocked, restoring the coarse-tuning shutter speed to an adjustable state; Based on the type of the inspection target, a polygonal region containing the inspection target is segmented from the real-time image of the current frame acquired after the focus search is completed, and is taken as the region of interest. Centered on the coarse shutter speed, multiple candidate shutter speeds are generated within a preset search neighborhood to form a candidate solution set; For each iteration, a real-time image is acquired sequentially according to each candidate shutter speed in the candidate solution set, and the region of interest corresponding to the inspection target type is extracted from the real-time image. The average brightness value of the region of interest is calculated based on the local histogram features of the region of interest, and the absolute value of the difference between the average brightness value and the preset target brightness is used as the fitness value of the candidate shutter speed. Based on the fitness values ​​of all candidate shutter speeds in the current iteration round, record the minimum fitness value of each candidate shutter speed in the historical iteration process, and use it as the historical best fitness value of the candidate shutter speed. Then, take the candidate shutter speed corresponding to the historical best fitness value as the historical best position of the candidate shutter speed. Record the global minimum fitness value of all candidate shutter speeds in the historical iteration process, and use it as the global historical optimal fitness value. Then, use the candidate shutter speed corresponding to the global historical optimal fitness value as the global historical optimal position. Based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain the updated candidate solution set. Repeat the iterative process until the preset convergence condition is met, and use the candidate shutter speed corresponding to the global historical best position obtained from the final iteration as the fine-tuning shutter speed.

9. The method for dynamic adjustment of camera parameters for intelligent identification of unmanned aerial vehicles according to claim 8, characterized in that, Based on the current value of each candidate shutter speed, the historical best position of the candidate shutter speed, and the global historical best position, each candidate shutter speed in the candidate solution set is updated to obtain the updated candidate solution set, including: Based on the current iteration round and the preset total iteration round, calculate the first weight coefficient and the second weight coefficient, wherein the first weight coefficient decreases as the iteration round increases, and the second weight coefficient increases as the iteration round increases; For each candidate shutter speed, calculate the first difference between the current value of the candidate shutter speed and the historical best position, and multiply the first difference by the first weighting coefficient to obtain the first update component; Calculate the second difference between the current value of the candidate shutter speed and the global historical best position, and multiply the second difference by the second weighting coefficient to obtain the second update component; Add the first update component to the second update component to obtain the total update amount; Based on the magnitude and direction of the total update amount, the current value of the candidate shutter speed is adjusted to obtain the updated candidate shutter speed; The updated candidate shutter speeds corresponding to all candidate shutter speeds are used to form the updated candidate solution set.

10. A camera parameter dynamic adjustment system for intelligent recognition of unmanned aerial vehicles (UAVs), characterized in that, The method for dynamically adjusting camera parameters for intelligent recognition of unmanned aerial vehicles as described in any one of claims 1-9 includes: The parameter initialization module is used to predict scene illumination parameters based on environmental data of the area where the UAV inspection mission is located, and to obtain the initial shutter speed and initial focus distance by matching the scene illumination parameters with the inspection target type. The shutter speed coarse adjustment module is used to start real-time image acquisition when the distance between the UAV and the inspection target is less than a preset first distance threshold, and adjust the initial shutter speed based on the brightness deviation between the global brightness of the image and the preset target brightness to obtain the coarse shutter speed. The focus distance search module is used to lock the coarse shutter speed when the distance between the UAV and the inspection target is less than a preset second distance threshold. It uses the initial focus distance as the search starting point, adopts an improved multi-directional Sobel operator as the sharpness evaluation function and performs focus search, and locks the fine focus distance at the peak point of the sharpness evaluation value. The shutter speed fine-tuning module is used to unlock the coarse shutter speed, extract the region of interest corresponding to the inspection target type, perform local histogram analysis, and fine-tune the coarse shutter speed according to the analysis results to obtain the fine shutter speed. The parameter output module is used to control the UAV to perform inspection shooting based on the fine-tuned focus distance and the fine-tuned shutter speed.

Images