A multi-environmental UAV inspection image evaluation method

Abstract

The present invention provides a multi-environment-based UAV inspection image evaluation method, which relates to the technical field of power line UAV inspection. The method uses UAVs to collect power line inspection images in different environments and weather conditions, constructs a multi-dimensional standard database of inspection images and a lightweight dual-classification model, and then trains and adapts the lightweight dual-classification model through the collected power line inspection images. The input inspection images are efficiently and quickly evaluated according to an adaptive image evaluation mechanism, and the lightweight dual-classification model is fine-tuned and iteratively optimized using a closed-loop optimization mechanism to achieve dynamic update and evolution of thresholds. The inspection images collected by the UAV can be quickly, intelligently and automatically screened, providing high-quality image data for subsequent power line defect detection and fault diagnosis, thereby solving the problems of poor environmental adaptability, strong manual dependence and insufficient robustness in complex scenarios in the quality evaluation of UAV inspection images.

Description

Technical Field

[0001] The present invention relates to the technical field of power line unmanned aerial vehicle inspection, and in particular to a multi-environment-based unmanned aerial vehicle inspection image evaluation method. Background Art

[0002] With the development of intelligent power grids, a large number of unmanned equipment are now being deployed for power line inspections. Digital images captured by drones contain a wealth of valuable information, providing a key source for subsequent identification and defect detection of power components. Power lines are often located in complex terrain, such as mountainous and forested areas. During power line inspections, drone-generated images are intensity images of visible or infrared light, and their clarity is significantly affected by the external environment. For example, in rainy or foggy weather, images can become blurred by raindrops or haze particles. Furthermore, since inspections are mostly conducted outdoors, high-intensity daytime lighting can cause overexposure to images, while low-light conditions such as dusk or nighttime can result in dark images. Furthermore, the complex and random nature of drones' relative motion can lead to blurring and jitter in captured images. Currently, power line drones typically conduct high-frequency inspections around the clock, resulting in a massive amount of collected image data.

[0003] Therefore, it is essential to evaluate the quality of the large number of inspection images with varying quality, and to filter out poor-quality images. This can improve the accuracy of target recognition and defect detection in subsequent processes, reduce model false positives, and reduce computing power usage, which is of great significance to intelligent power grid inspection.

[0004] The existing drone inspection image screening methods mainly include manual inspection-based methods and automatic screening methods based on algorithm models.

[0005] The problems of the prior art mainly involve two points:

[0006] First, the manual periodic inspection and screening method requires professionals to visually screen degraded images (such as blur, overexposure, etc.). This method is time-consuming and inefficient, and there is a problem of missed detection and false detection due to visual fatigue.

[0007] Secondly, the automatic screening method based on the algorithm model has poor environmental adaptability and cannot adapt to the differences in image characteristics in different terrains such as mountains and plains, and in complex weather conditions such as sunny, rainy and snowy weather, resulting in a high misjudgment rate.

[0008] Therefore, it is urgent to develop a solution to solve the above problems. Summary of the Invention

[0009] The purpose of the present invention is to provide a multi-environmental drone inspection image evaluation method, which solves the problems of poor environmental adaptability, strong manual dependence, and insufficient robustness in complex scenes in the current drone inspection image quality evaluation.

[0010] The present invention provides a multi-environmental drone inspection image evaluation method, which adopts the following technical solutions:

[0011] A multi-environmental UAV inspection image evaluation method includes the following steps:

[0012] Use drones to collect line inspection image data in different environments and weather conditions, and build a multi-dimensional standard database of inspection images;

[0013] Based on the environment classification and the weather classification, a lightweight dual-classification model is constructed, and the lightweight dual-classification model is trained and adapted;

[0014] Inputting drone inspection image data, and using an adaptive image evaluation mechanism to filter and fine-tune the drone inspection image;

[0015] The inspection images that fail the screening are saved and manually reviewed, and the inspection images that pass the review are input into the lightweight dual-classification model, and the lightweight dual-classification model is iteratively optimized using a closed-loop optimization mechanism.

[0016] A multi-environment-based drone inspection image evaluation method uses drones to collect power line inspection images in different environments and weather conditions, builds a multi-dimensional standard database of inspection images, and constructs a lightweight dual-classification model based on different classifications of environments and weather conditions. The lightweight dual-classification model is trained and adapted using the collected power line inspection images, and then the input inspection images are efficiently and quickly evaluated based on an adaptive image evaluation mechanism. The lightweight dual-classification model is fine-tuned and iteratively optimized using a closed-loop optimization mechanism to achieve dynamic update and evolution of thresholds. The method can quickly, intelligently, and automatically screen inspection images collected by power drones, improve the accuracy of target recognition and defect detection, reduce computing power usage, and provide high-quality image data for subsequent power line defect detection and fault diagnosis.

[0017] This invention uses an adaptive image quality assessment threshold system coupled with environmental and weather conditions, enabling the establishment of quality assessment criteria under the combined influence of terrain and weather through statistical methods. Furthermore, it employs a hybrid discrimination method based on physical and deep learning models, organically combining image processing features with deep learning classification. This method utilizes a dynamic threshold generation strategy, dynamically adjusting the judgment criteria using normal distribution parameters to achieve dynamic threshold update and evolution. Furthermore, it utilizes an edge-to-cloud processing mechanism based on drones, filtering 80% of invalid data on the drone side, reducing bandwidth consumption by 80% and significantly reducing response latency.

[0018] Optionally, constructing a multi-dimensional standard database of inspection images includes:

[0019] The collected images are labeled with the environment and weather information, and classified and combined according to different environments and weather conditions to obtain the original image set of the multi-dimensional standard database;

[0020] Calculate the Laplace variance and brightness mean of each image in each combination to form a data set;

[0021] Calculate normal distribution parameters and generate dynamic thresholds.

[0022] Optionally, calculating the normal distribution parameters includes calculating the mean and standard deviation of the Laplace variance normal distribution of each set of data sets, and calculating the mean and standard deviation of the normal distribution of the brightness of each set of data sets.

[0023] Optionally, the Laplace variance of each image is calculated as:

[0024]

[0025] in, is the Laplace operator, operator template , is a grayscale image.

[0026] Optionally, the brightness mean of each image is calculated as:

[0027]

[0028] in, is the coordinate of the pixel point, is the image height, is the image width, for The HSV space brightness value corresponding to the pixel coordinates.

[0029] Optionally, constructing a lightweight dual-classification model and training and adapting the lightweight dual-classification model includes using the lightweight model YOLOv8 as the backbone model, and setting an environmental classification output branch and a weather classification output branch in parallel at the neck of the lightweight model YOLOv8; sampling 80% of the collected inspection image data for training and adaptation of the lightweight dual-classification model, and verifying the remaining 20% ​​of the inspection image data.

[0030] Optionally, the adaptive image evaluation mechanism includes a two-level judgment mechanism, which performs primary filtering and fine judgment on the input inspection image data; the primary filtering is based on the brightness histogram of the inspection image to exclude extremely dark or overexposed inspection images; the fine judgment is to classify the inspection image after primary filtering into scenes, obtain thresholds and multi-dimensional judgments.

[0031] Optionally, the adaptive image evaluation mechanism also includes a real-time feedback mechanism, which judges the input inspection images. When several unqualified inspection images appear continuously in the input inspection images, the hovering quality is sent to the drone and the inspection is suspended; the real-time feedback mechanism is used to handle shooting abnormalities in different scenarios.

[0032] Optionally, the closed-loop optimization mechanism includes an online learning mechanism, which regularly feeds back the results of manual review of the inspection images to the lightweight dual-classification model to fine-tune the lightweight dual-classification model.

[0033] Optionally, the closed-loop optimization mechanism also includes a threshold dynamic update mechanism, which maintains a latest image sample buffer with a fixed window size through a sliding window algorithm, and recalculates the distribution parameters of the standard database when the newly added data reaches 10% of the window size.

[0034] The beneficial effects of the present invention are: a multi-environment-based drone inspection image evaluation method, which uses drones to collect power line inspection images in different environments and weather conditions, builds a multi-dimensional standard database of inspection images, and builds a lightweight dual-classification model based on different classifications of environment and weather. The lightweight dual-classification model is trained and adapted through the collected power line inspection images, and then the input inspection images are efficiently and quickly evaluated for quality based on the adaptive image evaluation mechanism. The lightweight dual-classification model is fine-tuned and iteratively optimized using a closed-loop optimization mechanism to achieve dynamic update and evolution of the threshold. It can quickly, intelligently, and automatically screen the inspection images collected by power drones, improve the accuracy of target recognition and defect detection, reduce computing power usage, significantly reduce response delays, and provide high-quality image data for subsequent power line defect detection and fault diagnosis. The present invention solves the problems of poor environmental adaptability, strong manual dependence, and insufficient robustness in complex scenarios in drone inspection image quality assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the flow chart of the multi-environment-based UAV inspection image evaluation method provided by the present invention;

[0036] Figure 2 Schematic diagram of the lightweight dual-classification model architecture of the present invention;

[0037] Figure 3 This is a schematic diagram of the primary filtration process of the present invention;

[0038] Figure 4 This is a schematic diagram of the detailed determination process of the present invention;

[0039] Figure 5 Schematic diagram of the process of iterative optimization of the model of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Unless otherwise defined, the technical terms or scientific terms used herein should be the common meanings understood by people with ordinary skills in the field to which the invention belongs. The words "including" and similar words used in this article mean that the elements or objects appearing before the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0041] Example 1

[0042] like Figure 1 As shown, an embodiment of the present invention provides a multi-environmental drone inspection image evaluation method, which specifically includes the following steps:

[0043] S1. Use drones to collect line inspection image data in different environments and weather conditions, and build a multi-dimensional standard database of inspection images;

[0044] S2. Building a lightweight dual-classification model based on the environment classification and the weather classification, and training and adapting the lightweight dual-classification model;

[0045] S3. Inputting drone inspection image data, and using an adaptive image evaluation mechanism to filter and fine-tune the drone inspection image;

[0046] S4. The inspection images that fail the screening are saved and manually reviewed. The inspection images that pass the review are input into the lightweight dual-classification model, and the lightweight dual-classification model is iteratively optimized using a closed-loop optimization mechanism.

[0047] A multi-environment-based drone inspection image evaluation method uses drones to collect power line inspection images in different environments and weather conditions, builds a multi-dimensional standard database of inspection images, and constructs a lightweight dual-classification model based on different classifications of environments and weather conditions. The lightweight dual-classification model is trained and adapted using the collected power line inspection images, and then the input inspection images are efficiently and quickly evaluated based on an adaptive image evaluation mechanism. The lightweight dual-classification model is fine-tuned and iteratively optimized using a closed-loop optimization mechanism to achieve dynamic update and evolution of thresholds. The method can quickly, intelligently, and automatically screen inspection images collected by power drones, improve the accuracy of target recognition and defect detection, reduce computing power usage, and provide high-quality image data for subsequent power line defect detection and fault diagnosis.

[0048] This invention uses an adaptive image quality assessment threshold system coupled with environmental and weather conditions, enabling the establishment of quality assessment criteria under the combined influence of terrain and weather through statistical methods. Furthermore, it employs a hybrid discrimination method based on physical and deep learning models, organically combining image processing features with deep learning classification. This method utilizes a dynamic threshold generation strategy, dynamically adjusting the judgment criteria using normal distribution parameters to achieve dynamic threshold update and evolution. Furthermore, it utilizes an edge-to-cloud processing mechanism based on drones, filtering 80% of invalid data on the drone side, reducing bandwidth consumption by 80% and significantly reducing response latency.

[0049] In some embodiments, constructing a multi-dimensional standard database of inspection images specifically includes:

[0050] The collected images are labeled with the environment and weather information, and classified and combined according to different environments and weather conditions to obtain the original image set of the multi-dimensional standard database;

[0051] Calculate the Laplace variance and brightness mean of each image in each combination to form a data set;

[0052] Calculate normal distribution parameters and generate dynamic thresholds.

[0053] Furthermore, the standard database includes three dimensions: environment-weather matrix sampling, physical feature quantification, and dynamic threshold generation. The environment-weather matrix sampling dimension covers a complete combination of four terrain types (mountainous / hilly / plain / urban) and five weather types (sunny / overcast / rainy / snowy / foggy), collecting ≥500 properly captured, clear original images for each combination. The physical feature quantification dimension includes grayscale Laplace variance calculation (reflecting image clarity) and HSV spatial brightness mean (detecting low light and overexposure). The dynamic threshold generation dimension calculates the normal distribution parameters of the characteristic values ​​for each environment-weather combination, using the 3σ principle (μ±3σ) of the normal distribution as the quality criterion.

[0054] The calculating of the normal distribution parameters includes calculating the mean and standard deviation of the Laplace variance normal distribution of each set of data sets, and calculating the mean and standard deviation of the normal distribution of the brightness of each set of data sets.

[0055] In some embodiments, the specific steps of constructing a multi-dimensional standard database of inspection images can be expressed as follows:

[0056] The collected images are annotated with weather (sunny, cloudy, rainy, snowy, foggy) and environment (mountainous, hilly, plain, urban) to obtain the original image set in the standard database. ,in , .

[0057] For each image Calculate the grayscale Laplace variance and brightness mean.

[0058] The Laplace variance of each image is calculated as:

[0059]

[0060] in, is the Laplace operator, operator template , is a grayscale image.

[0061] The formula for calculating the mean brightness of each image is:

[0062]

[0063] in, is the coordinate of the pixel point, is the image height, is the image width, for The HSV space brightness value corresponding to the pixel coordinates. The characteristic values ​​are stored by environment-weather combination group to obtain the data set .

[0064] For each data set Calculate the mean and standard deviation of the Laplace variance normal distribution. The calculation formulas are:

[0065]

[0066] in, is the mean of the Laplace variance normal distribution, is the standard deviation of the Laplace variance normal distribution, is the number of images in each dataset, For the corresponding Laplace variance of the image.

[0067] For each data set Calculate the mean and standard deviation of the normal distribution of brightness. The calculation formulas are:

[0068]

[0069] in, is the mean of the normal distribution of brightness, is the standard deviation of the normal distribution of brightness, is the number of images in each dataset, For the corresponding The mean brightness of the image.

[0070] Generate dynamic threshold in real time. When the input image is larger than 5 consecutive , the threshold is recalculated. The formula for recalculating the threshold is:

[0071]

[0072] in, Take 0.9, is the threshold used last time, is the current threshold.

[0073] In some embodiments, as Figure 2As shown, the construction of the lightweight dual-classification model and the training and adaptation of the lightweight dual-classification model include taking the lightweight model YOLOv8 as the backbone model, setting the environment classification output branch and the weather classification output branch in parallel at the neck of the lightweight model YOLOv8; sampling 80% of the collected inspection image data for training and adaptation of the lightweight dual-classification model, and verifying the remaining 20% ​​of the inspection image data.

[0074] Further, such as Figure 2 As shown, the lightweight YOLOv8 model is used as the backbone model. Two classification branches are added to the original model structure, with parallel outputs for environmental classification (4 categories) and weather classification (5 categories). 80% of the collected sample images are used for model training, and the remaining 20% ​​are used for validation. Targeted data augmentation strategies are used during model training. To account for lighting disturbances, training images are subjected to a ±30% brightness variation. For specific scenarios such as rain and snow, rain and snow noise are simulated and Poisson noise is added to augment the training dataset.

[0075] To meet low computational requirements and fast inference speed, and to adapt to diverse device deployment requirements, the model weights are quantized to INT8 precision, ensuring a memory footprint of no more than 100MB. The lightweight dual-classification model architecture is based on improvements to YOLOv8, adding environmental and weather classification branches after the Neck Feature Pyramid Network (PAN-FPN) of YOLOv8. The feature maps output by the dual-classification branches are processed through global average pooling (GAP) and a fully connected layer (FC) before outputting the classification result.

[0076] Specifically, the model feature map size is calculated as:

[0077] (1) Model input size: 640 × 640 × 3 (RGB image);

[0078] (2) Output multi-scale features through the backbone network:

[0079] 80×80×256 (shallow features: edges / textures);

[0080] 40×40×512 (middle-level features: component level);

[0081] 20×20×1024 (deep features: semantic level);

[0082] (3) Output size after feature pyramid network (PAN-FPN): 20×20×256;

[0083] (4) After global average pooling (GAP), the output size is 1×1×256;

[0084] (5) After the fully connected layer (FC), the classification result is obtained: 1×1×4 or 1×1×5.

[0085] Furthermore, in some embodiments, the adaptive image evaluation mechanism includes a two-level judgment mechanism, which performs primary filtering and fine judgment on the input inspection image data; the primary filtering is based on the brightness histogram of the inspection image to exclude extremely dark or overexposed inspection images; the fine judgment is to classify the inspection image after primary filtering into scenes, obtain thresholds and multi-dimensional judgments.

[0086] Specifically, primary filtering uses the image brightness histogram to eliminate extremely dark or overexposed images. First, the image is converted to the HSV (hue, saturation, value) color space, where the V channel directly reflects image brightness information. The V channel's pixel brightness histogram is then calculated to determine the average brightness, the percentage of low-light levels, and the percentage of high-light levels.

[0087] The average brightness reflects the overall brightness of the image, and the threshold range If the value exceeds the threshold, it is judged as unqualified. The calculation formula is as follows:

[0088]

[0089] Where N is the total number of image pixels, is the number of pixels with brightness value i.

[0090] The low brightness ratio reflects the proportion of pixels in the dark area of ​​the image. If it is greater than 40%, it is judged as unqualified. The calculation formula is as follows:

[0091]

[0092] Where N is the total number of image pixels, The low brightness ratio reflects the ratio of pixels in the dark area of ​​the image. is the number of pixels with brightness value i.

[0093] The high brightness ratio reflects the ratio of pixels in the bright area of ​​the image. If it is greater than 30%, it is considered unqualified. The calculation formula is as follows:

[0094]

[0095] Where N is the total number of image pixels, The high brightness ratio reflects the ratio of pixels in the dark area of ​​the image. is the number of pixels with brightness value i.

[0096] Specifically, such as Figure 3As shown in the figure, the primary filtering process can be described as follows: first, the image is input, and then the image's brightness histogram is calculated. Based on the brightness histogram, three calculation steps are performed: calculating the average brightness, calculating the low brightness ratio, and calculating the high brightness ratio. For the average brightness, if its value is between 30 and 220, the image is considered qualified; otherwise, it is unqualified. For the low brightness ratio, if its value is less than 40%, the image is considered qualified; otherwise, it is unqualified. For the high brightness ratio, if its value is less than 30%, the image is considered qualified; otherwise, it is unqualified. Through these calculation and judgment steps, the entire process comprehensively evaluates the brightness characteristics of the image to determine whether it meets the specific brightness standard.

[0097] Specifically, such as Figure 4 As shown in the figure, fine-grained judgment mainly includes scene classification, threshold acquisition, and multi-dimensional judgment. Scene classification obtains the image's environment and weather category by inputting the image into a dual-classification model. Then, based on the environment and weather category, the corresponding Laplace variance threshold and brightness threshold range are obtained from the standard database. The threshold range is established based on the 3σ principle of normal distribution (μ±3σ). The Laplace variance (i.e., clarity) qualification criterion formula is:

[0098]

[0099] The brightness qualification criterion formula is:

[0100]

[0101] in, is the mean of the Laplace variance normal distribution, is the standard deviation of the Laplace variance normal distribution, is the mean of the normal distribution of brightness, is the standard deviation of the normal distribution of brightness.

[0102] Finally, the Laplace variance and brightness value of the input image are calculated, and the image is determined to be qualified by comparing it with the standard database threshold.

[0103] Specifically, such as Figure 4As shown, the refined assessment process can be described as follows: First, the image to be evaluated is input. A dual-classification model is then used to classify and identify the image. This step aims to initially distinguish the image's categories or features, providing a basis for subsequent quality assessment. Subsequently, the image is further analyzed, taking into account environmental and weather factors. During this analysis, a standard Laplace variance threshold is queried and used to perform a Laplace variance assessment on the image. Laplace variance is often used to measure image clarity. By comparing it with the standard threshold, it can be determined whether the image meets the required clarity. Furthermore, a standard brightness threshold is queried and used to perform a brightness assessment based on this threshold to determine whether the image's brightness meets the required brightness. Finally, the results of the Laplace variance and brightness assessments are combined. If both meet the standards, the image is deemed qualified; if either fails, the image is deemed unqualified. This multi-step analysis and assessment process comprehensively assesses image quality.

[0104] Furthermore, the adaptive image evaluation mechanism also includes a real-time feedback mechanism, which judges the input inspection images. When several unqualified inspection images appear continuously in the input inspection images, the hovering quality is sent to the drone and the inspection is suspended; the real-time feedback mechanism is used to handle shooting abnormalities in different scenarios.

[0105] The real-time feedback mechanism primarily addresses possible anomalies during drone photography in different scenarios. If more than five consecutive inspection images are deemed unqualified, a hover command is sent to the drone, pausing the inspection and awaiting manual intervention to prevent continued ineffective photography.

[0106] Specifically, the closed-loop optimization mechanism includes an online learning mechanism that regularly feeds back the results of manual review of the inspection images to the lightweight dual-classification model for fine-tuning. The online learning mechanism is responsible for regularly feeding back the manual review results (approximately 10% of the eliminated image annotation set) to the model fine-tuning process.

[0107] Furthermore, the closed-loop optimization mechanism also includes a threshold dynamic update mechanism. The threshold dynamic update mechanism maintains a fixed window size of the latest image sample buffer through a sliding window algorithm. When the new data reaches 10% of the window size, the distribution parameters of the standard database are recalculated. The threshold dynamic update mechanism is implemented through two types of strategies. One is to use a sliding window algorithm to maintain a fixed window size of the latest image sample buffer. When the amount of new data reaches 10% of the window size, the distribution parameters of the standard database are recalculated. The other is to record the image screening process. When the input image continuously exceeds the threshold, it triggers the recalculation of the threshold and dynamically adjusts the threshold.

[0108] Example 2

[0109] Specifically, taking a 10kV line inspection as an example, combined with Figure 1 As shown, the specific implementation steps of the present invention are as follows:

[0110] S1, collects line inspection image samples taken by power drones in different weather conditions and environments across the province from the provincial drone platform.

[0111] S2, selects images that are taken normally and have clear pictures, and carries out annotation of the environment (mountains, hills, plains, urban areas) and weather (sunny, cloudy, rainy, snowy, foggy).

[0112] S3 builds a multi-dimensional standard database covering a complete combination of 4 types of terrain (mountains, hills, plains, and urban areas) and 5 types of weather (sunny, cloudy, rainy, snowy, and foggy), and collects no less than 500 original images for each combination.

[0113] S31, calculating the Laplace variance and brightness mean of each image in each combination, grouping and storing the feature values ​​according to the environment-weather combination to form a data set.

[0114] S32, calculating the distribution parameters of each combined eigenvalue.

[0115] S33, based on the 3σ principle of normal distribution (covering 99.7% normal data), set the feature threshold range.

[0116] S34, recording the feature threshold range of each combined image, saving it to a file, and forming a standard feature database file standard_db.json.

[0117] Specifically, taking the combination of (mountain, snowy) and (plain, sunny) as an example, the calculated threshold range is shown in Table 1. Since a larger Laplace variance value indicates better image clarity, only a minimum value is set for this feature value. lap_var_min is the minimum Laplace variance value, brightness_min is the minimum brightness value, and brightness_max is the maximum brightness value.

[0118] environment weather lap_var_min brightness_min brightness_max Mountain Snowy Day 28.5 60 180 plains sunny 65.0 120 210

[0119] S4: Input the dataset annotated in step S2 into the lightweight dual-classification model for iterative training to obtain a classification model that can identify environmental categories and weather categories.

[0120] S5, input the drone inspection image and calculate the three indicators of image average brightness, low brightness ratio, and high brightness ratio.

[0121] S6, perform primary screening based on the calculation results of S5. If the average brightness is within the range of [30,220], the low brightness ratio is less than 40%, and the high brightness ratio is less than 30%, then the primary screening is passed.

[0122] Specifically, for example, for night-time images, the calculated brightness mean (v_mean=18) results in a failure. For highly reflective overexposed images, the calculated high brightness ratio is 60%, resulting in a failure.

[0123] S7, performing a detailed judgment on the image that has passed the primary screening, inputting the image into the classification model trained in step S4, and identifying the environment category and weather category of the image.

[0124] S8, querying the standard database for the characteristic value threshold corresponding to the environment-weather category, including the minimum Laplace variance value and the brightness range threshold.

[0125] Specifically, taking a snowy mountain scene as an example, the input image has lap_var = 25.3 and brightness = 82. Based on the table above, the comparison thresholds are: lap_var_min = 28.5, brightness_range = (60, 180). The result is: Fail (the clarity does not meet the standard).

[0126] In S9, unqualified images are stored in the local file system. 10% of them are regularly (monthly) selected for manual review. The reviewed images are input into the lightweight dual-classification model for fine-tuning training. The learning rate is adjusted to 1 / 2 of the previous one, and the model is updated iteratively.

[0127] Specifically, the model iterative optimization process is as follows Figure 5 As shown in the figure, first, inspection image samples are collected. These samples serve as the basic data source for subsequent model training. Next, the collected samples are screened and labeled. The purpose of screening is to remove samples that do not meet the requirements or are of poor quality. Labeling is to add corresponding labels to each sample so that the model can learn the characteristics and categories of different images.

[0128] Afterwards, data preprocessing and data augmentation are performed on the filtered and labeled samples. Data preprocessing may include image normalization and resizing to ensure data consistency and standardization. Data augmentation, through methods such as brightness changes and noise addition, increases the diversity and quantity of samples, improving the model's generalization capabilities.

[0129] After data preprocessing and enhancement, a lightweight two-branch model is trained using this processed data, along with a portion of the original annotated data (10% data iterative training). This 10% data iterative training process means that during training, a portion of the data is continuously recycled to update model parameters, gradually optimizing model performance. The entire process, from data collection and processing to model training, forms a complete image recognition model training system, aiming to build a model that can accurately identify inspection images.

[0130] In step S10, qualified images are stored by category into the dataset from step S2. A fixed-capacity (N=2000) dataset buffer window is maintained. When the newly stored data reaches 10% of the window size, step S3 is repeated to recalculate the feature distribution parameters, dynamically adjusting the standard thresholds for image features.

[0131] S11, record the judgment result of each input image. When 5 consecutive images are judged as unqualified, the threshold is triggered to be recalculated. At the same time, a hover command is sent to suspend the inspection, realizing a real-time feedback mechanism.

[0132] A multi-environment-based drone inspection image evaluation method uses drones to collect power line inspection images in different environments and weather conditions, builds a multi-dimensional standard database of inspection images, and constructs a lightweight dual-classification model based on different classifications of environments and weather conditions. The lightweight dual-classification model is trained and adapted using the collected power line inspection images, and then the input inspection images are efficiently and quickly evaluated based on an adaptive image evaluation mechanism. The lightweight dual-classification model is fine-tuned and iteratively optimized using a closed-loop optimization mechanism to achieve dynamic update and evolution of thresholds. The method can quickly, intelligently, and automatically screen inspection images collected by power drones, improve the accuracy of target recognition and defect detection, reduce computing power usage, and provide high-quality image data for subsequent power line defect detection and fault diagnosis.

[0133] In an embodiment of the present invention, an adaptive image quality assessment threshold system coupled with environmental weather can be used to establish quality assessment standards under the combined influence of terrain and weather using statistical methods. Furthermore, the present invention employs a hybrid discrimination method based on physical and deep learning models, organically combining image processing features with deep learning classification. This method utilizes a dynamic threshold generation strategy, dynamically adjusting the judgment criteria using normal distribution parameters to achieve dynamic updating and evolution of the threshold. Furthermore, an edge-to-cloud processing mechanism can be implemented based on drones, completing 80% of invalid data filtering on the drone side, reducing bandwidth consumption by 80%, and significantly reducing response latency.

[0134] While the embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations of these embodiments are possible. However, it should be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the claims. Furthermore, the invention described herein is susceptible to other embodiments and may be practiced or implemented in a variety of ways.

Claims

1. A multi-environmental UAV inspection image evaluation method, characterized in that: include: Use drones to collect line inspection image data in different environments and weather conditions, and build a multi-dimensional standard database of inspection images; Based on environmental classification and weather classification, a lightweight dual-classification model is constructed. When training and adapting the lightweight dual-classification model, the lightweight model YOLOv8 is used as the backbone model, and an environmental classification output branch and a weather classification output branch are set in parallel at the neck of the lightweight model YOLOv8; 80% of the collected inspection image data is sampled for training and adaptation of the lightweight dual-classification model, and the remaining 20% ​​of the inspection image data is verified; Inputting drone inspection image data, adopting an adaptive image evaluation mechanism, performing primary filtering and fine judgment on the drone inspection image, the fine judgment is to classify the inspection image after primary filtering into scenes, obtain thresholds and make multi-dimensional judgments; the fine judgment includes inputting the inspection image after primary filtering into the lightweight dual classification model for classification and identification to obtain the environment category and weather category of the inspection image, and then obtaining the corresponding Laplace variance threshold range and brightness threshold range in the standard database according to the environment category and weather category. The Laplace variance threshold range and brightness threshold range are established according to the 3σ principle of normal distribution, and the Laplace variance and brightness value of the inspection image after primary filtering are calculated, and the Laplace variance threshold range and brightness threshold range are compared to determine whether the image is qualified; The inspection images that fail the screening are saved and manually reviewed, and the inspection images that pass the review are input into the lightweight dual-classification model, and the lightweight dual-classification model is iteratively optimized using a closed-loop optimization mechanism.

2. The multi-environmental drone inspection image evaluation method according to claim 1 is characterized in that: The multi-dimensional standard database for inspection images is constructed as follows: The collected images are labeled with the environment and weather information, and classified and combined according to different environments and weather conditions to obtain the original image set of the multi-dimensional standard database; Calculate the Laplace variance and brightness mean of each image in each combination to form a data set; Calculate normal distribution parameters and generate dynamic thresholds.

3. The multi-environment based UAV inspection image evaluation method according to claim 2 is characterized in that: The calculating of the normal distribution parameters includes calculating the mean and standard deviation of the Laplace variance normal distribution of each set of data sets, and calculating the mean and standard deviation of the normal distribution of the brightness of each set of data sets.

4. The multi-environment based UAV inspection image evaluation method according to claim 2 is characterized in that: The Laplace variance of each image is calculated as: ； in, is the Laplace operator, operator template , is a grayscale image.

5. The multi-environment based UAV inspection image evaluation method according to claim 2 is characterized in that: The formula for calculating the mean brightness of each image is: ； in, is the coordinate of the pixel point, is the image height, is the image width, for The HSV space brightness value corresponding to the pixel coordinates.

6. The multi-environment based UAV inspection image evaluation method according to claim 1 is characterized in that: The adaptive image evaluation mechanism includes a two-level judgment mechanism, which performs primary filtering and fine judgment on the input inspection image data; the primary filtering is based on the brightness histogram of the inspection image to exclude extremely dark or overexposed inspection images.

7. The multi-environment based UAV inspection image evaluation method according to claim 1 is characterized in that: The adaptive image evaluation mechanism also includes a real-time feedback mechanism, which judges the input inspection images. When several unqualified inspection images appear in succession, the hovering quality is sent to the drone and the inspection is suspended. The real-time feedback mechanism is used to handle abnormal shooting situations in different scenarios.

8. The multi-environment based UAV inspection image evaluation method according to claim 1 is characterized in that: The closed-loop optimization mechanism includes an online learning mechanism, which regularly feeds back the results of manual review of the inspection images to the lightweight dual-classification model to fine-tune the lightweight dual-classification model.

9. The multi-environment based UAV inspection image evaluation method according to claim 1, characterized in that: The closed-loop optimization mechanism also includes a threshold dynamic update mechanism, which maintains a latest image sample buffer with a fixed window size through a sliding window algorithm. When the newly added data reaches 10% of the window size, the distribution parameters of the standard database are recalculated.

Images