Unmanned aerial vehicle inspection image defogging method and system based on diffusion model

Abstract

The application discloses a kind of unmanned aerial vehicle inspection image defogging method and system based on diffusion model.The method comprises the following steps: constructing fog-containing-no-fog image dataset, and dividing into training set and test set;Three diffusion defogging models are used to learn the mapping rule from fog-containing image to no-fog image in the training set, then the different fog condition type images in the test set are defogged, and the optimal diffusion defogging model corresponding to each fog condition type is selected;Train the meta-learner for fog condition type division;Collect the unmanned aerial vehicle inspection image to be defogged, identify the fog condition type of the unmanned aerial vehicle inspection image to be defogged by the meta-learner, call the optimal diffusion defogging model corresponding to the fog condition type for processing, and output the no-fog clear image.The application not only has a fog condition self-adaptive matching mechanism, realizes the purpose of strong detail recovery and good real-time performance, but also meets the deployment requirements of unmanned aerial vehicle embedded devices.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, specifically to a method and system for dehazing UAV inspection images based on a diffusion model. Background Technology

[0002] With its advantages of high flexibility, wide coverage, and strong security, drone inspection has become a core tool for power system operation and maintenance. However, the outdoor inspection environment is complex, and foggy weather can lead to reduced image contrast, loss of detail, and color distortion, seriously affecting the accuracy of personnel's judgment of equipment defects.

[0003] Existing dehazing techniques include physical model dehazing, traditional deep learning dehazing, and diffusion model dehazing.

[0004] Among them, the physical model defogging method relies on manual experience, has poor adaptability to non-uniform fog, is prone to artifacts, and lacks real-time performance. Traditional deep learning dehazing methods use a single network architecture, which is difficult to adapt to various fog conditions such as dense fog, medium fog, light fog, and non-uniform fog. The image detail recovery is insufficient, which cannot meet the key detail recognition requirements such as insulator texture and conductor outline in power line inspection. In addition, the model has a large number of parameters, which cannot be deployed on embedded devices of drones. The diffusion model defogging method lacks an adaptive matching mechanism for fog conditions, has weak generalization ability, and requires many sampling steps, resulting in low processing efficiency. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method and system for dehazing UAV inspection images based on a diffusion model. This method not only features an adaptive matching mechanism for fog conditions, achieving strong detail recovery and excellent real-time performance, but also meets the deployment requirements of embedded UAV devices, thus possessing significant engineering application value.

[0006] To achieve the above objectives, the present invention provides a method for dehazing UAV inspection images based on a diffusion model, which is characterized by including the following steps: S1) Collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images to construct a fog-fog-free image dataset. Perform two-dimensional data augmentation, and then divide the fog-fog-free image dataset into a training set and a test set. S2) Three diffusion dehazing models are designed, including the adaptive sensing diffusion model, the wavelet transform residual denoising diffusion model, and the basic residual denoising diffusion model. The above three diffusion dehazing models are used to learn the mapping rules from foggy images to fog-free images in the training set. Then, dehazing tests are performed on images with different fog conditions in the test set, and the optimal diffusion dehazing model corresponding to each fog condition is selected. S3) Based on the fog-fog-free image dataset, extract multiple fog condition meta-features, use the multiple fog condition meta-features as input, and use predefined different fog condition types as labels to train a meta-learner for fog condition type classification; S4) Collect inspection images of the drone to be defogged. Identify the fog type of the drone inspection images through the meta-learner constructed in step S3). Call the optimal diffusion defogging model corresponding to the fog type for processing and output a clear, fog-free image.

[0007] Furthermore, in S1), fog conditions are classified into dense fog, medium fog, light fog, and non-uniform fog types according to the atmospheric scattering coefficient β and spatial distribution variance.

[0008] Furthermore, in S1), the steps for constructing the fog-fog-free image dataset are as follows: First, select multiple clear, fog-free drone inspection images covering typical power line inspection scenarios as reference data sources and unify their resolution; then, calculate the pixel values ​​of the reference data sources to obtain darkened fog-free images, and then generate fog images of different fog conditions by adjusting the transmittance and atmospheric light values; finally, pair each fog-free image with the corresponding fog images of different fog conditions to construct a fog-fog-free image dataset.

[0009] Furthermore, in S1), the fog images of different fog conditions are generated by the following formula. hazy(p)=inf(p)t(p)+A(1-1.02×t(p)) In the formula, hazy(p) represents the hazy image with different haze types at pixel value p. inf(p) represents the darkened, hazy-free image at pixel value p. t(p) represents the transmittance at pixel value p. A represents the atmospheric light value at pixel value p. 1.02 represents the fog concentration correction factor.

[0010] Furthermore, in S1), the dual-dimensional data enhancement includes complexity enhancement and diversity enhancement. The complexity enhancement is to expand the range of atmospheric scattering coefficient and the range of atmospheric light value to generate complex samples of dense fog with low signal-to-noise ratio. The diversity enhancement is to generate heterogeneous fog samples by fusing the low-frequency amplitude of foggy images with high-frequency phase of fog-free images through Fourier transform and smoothing the boundaries through Gaussian filtering of standard deviation.

[0011] Furthermore, in S2), the adaptive sensing diffusion model includes a haze data enhancement module, an improved MITNet physical feature extraction module, a distribution correction module, and a dynamic feature fusion module; The dynamic feature fusion module adaptively fuses the physical modeling results and diffusion-generated features using the following formula. x t-1 =t(x)·J t-1 +(1-t(x))·x t-1 In the formula, x t-1 This represents the noise prediction image from the previous time step of the diffusion model. t(x) represents the transmittance map of the image. J t-1 This represents a partially dehazed, clear image generated by the physical model; The wavelet transform residual denoising diffusion model adds a high-frequency enhancement module, a dual-branch residual denoising diffusion network, and a cross-frequency adjustment module to the adaptive sensing diffusion model.

[0012] Furthermore, in S2), the method for selecting the optimal diffusion defogging model for each fog condition type is as follows: by comparing the peak signal-to-noise ratio and structural similarity of different diffusion defogging models under the corresponding fog condition, if the peak signal-to-noise ratio of a certain diffusion defogging model under the corresponding fog condition is larger than that of the other two diffusion defogging models, and the structural similarity value is also larger than that of the other two diffusion defogging models, then the diffusion defogging model is the optimal diffusion defogging model under the corresponding fog condition.

[0013] Furthermore, in S3), the fog condition element features include five fog condition element features: average fog concentration, image contrast, proportion of high-frequency components, transmittance variance, and atmospheric light value.

[0014] Further, in S4), the specific steps for the meta-learner to identify the fog condition type of the drone inspection image to be defogged include: first, cropping the input image to a resolution of 512×512 and normalizing the RGB3 channels to [0,1]; then extracting the fog condition meta-features and inputting them into the trained meta-learner to output the fog condition type.

[0015] The present invention also designs a drone inspection image dehazing system based on a diffusion model, which is characterized by including a dataset construction and enhancement module, a model design and matching module, a meta-learner training module, and a dehazing processing module. The dataset construction and enhancement module is used to collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images to construct a fog-fog-free image dataset. It also performs two-dimensional data enhancement and then divides the fog-fog-free image dataset into a training set and a test set. The model design and matching module is used to design three diffusion dehazing models, including an adaptive perceptual diffusion model, a wavelet transform residual denoising diffusion model, and a basic residual denoising diffusion model. The module then uses these three diffusion dehazing models to learn the mapping rules from foggy images to fog-free images in the training set. Finally, it performs dehazing tests on images with different fog conditions in the test set and selects the optimal diffusion dehazing model for each fog condition type. The meta-learner training module is used to extract multiple fog condition meta-features based on a fog-fog-free image dataset, and to train a meta-learner for fog condition type classification using the multiple fog condition meta-features as input and predefined different fog condition types as labels. The defogging processing module is used to acquire inspection images of the UAV to be defogged, identify the fog condition type of the inspection images of the UAV to be defogged through the constructed meta-learner, call the optimal diffusion defogging model corresponding to the fog condition type for processing, and output a clear image without fog.

[0016] The advantages of this invention are: 1. Existing technologies only focus on the algorithm level and do not consider the actual engineering process of drone inspection (image acquisition, preprocessing, dehazing, postprocessing, and embedded deployment), which makes it impossible to implement the technology. This invention constructs a complete engineering closed loop from dataset construction → model design → fog condition recognition → adaptive defogging → embedded deployment, with each step designed around the actual needs of UAV power line inspection: The process includes: preprocessing: cropping the image to be dehazed to 512×512 to match the computing power of the drone's embedded device; postprocessing: restoring the original resolution and mapping the pixel values ​​back to [0,255] to meet the subsequent viewing / analysis requirements of the inspection image; model deployment: through lightweight design and parameter control, the model can be directly deployed on the drone's embedded device to achieve real-time dehazing on the edge, rather than only being processed in the cloud. This is an engineering breakthrough that has not been achieved by existing dehazing technologies. 2. Existing defogging technologies mostly use the common PSNR / SSIM evaluation, but have not been specifically optimized for the detailed recovery needs of power inspection. Although some models have high PSNR, they are insufficient in recovering key details of power equipment (such as insulator cracks and broken conductor strands), and cannot support subsequent fault detection. This invention strongly binds the PSNR / SSIM filtering rules to the fog conditions of power inspection, and uses "the maximum PSNR and SSIM under each fog condition" as the optimal model standard, rather than a general global optimal one, to ensure that the model under each fog condition can accurately restore the details of power equipment. 3. This invention enhances the frequency domain of the wavelet transform residual denoising diffusion model (wavelet transform + logarithmic transform), cross-frequency attention interaction, and the fusion of physical and diffusion features to ensure that the dehazed image is free of artifacts and oversaturation, and fully preserves key details such as insulator texture and conductor outline. This is a detailed optimization not done by existing patents for power inspection fault detection, enabling the dehazed image to directly support subsequent equipment defect identification, achieving end-to-end empowerment of "dehazed inspection". The wavelet transform residual denoising diffusion model (WRD-DM) and the basic residual denoising diffusion model (RDDM) designed in this invention optimize the number of sampling steps and network structure, with a minimum processing time of 0.2 seconds per image and ≤60M parameters, meeting the requirements of embedded deployment of UAVs. 4. This invention is the first to define a quantifiable fog condition classification standard for UAV inspection scenarios (classifying dense fog / medium fog / light fog / non-uniform fog according to atmospheric scattering coefficient β and spatial distribution variance), transforming the ambiguous fog condition types into calculable numerical features, and solving the problem of the lack of a unified judgment standard for fog conditions; 5. The dual-dimensional data augmentation designed in this invention generates heterogeneous fog samples by combining extreme fog condition samples with frequency domain fusion, making the dataset coverage more comprehensive. The model achieves a defogging accuracy of ≥90% in real foggy scenarios. 6. This invention employs multiple diffusion defogging models with specially designed structures for different fog conditions, combined with a meta-learner for adaptive fog condition recognition, to solve the problem of weak generalization ability of a single model. The PSNR and SSIM scores are improved by 5% to 15% compared with traditional diffusion models. The present invention provides a method and system for dehazing UAV inspection images based on a diffusion model. This method not only has an adaptive matching mechanism for fog conditions, achieving strong detail recovery and excellent real-time performance, but also meets the deployment requirements of embedded UAV devices, thus having significant engineering application value. Attached Figure Description

[0017] Figure 1 This is a flowchart of the UAV inspection image dehazing method based on the diffusion model in this invention. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] In the description of this invention, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention.

[0020] Example 1 like Figure 1 As shown, the present invention provides a method for dehazing UAV inspection images based on a diffusion model, comprising the following steps: S1) Collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images. Then perform two-dimensional data augmentation to construct a fog-fog-free image dataset, and then divide the fog-fog-free image dataset into a training set and a test set.

[0021] Specifically, fog conditions are classified into dense fog, moderate fog, light fog, and non-uniform fog types according to the atmospheric scattering coefficient β and spatial distribution variance. In this embodiment, the atmospheric scattering coefficient β of dense fog is [0.08, 0.10], the atmospheric scattering coefficient β of moderate fog is [0.04, 0.07], the atmospheric scattering coefficient β of light fog is [0.01, 0.03], the atmospheric scattering coefficient β of non-uniform fog is [0.02, 0.09], and the atmospheric scattering coefficient β of fogless fog is infinitely close to 0, with a spatial distribution variance ≥ 0.005.

[0022] The steps for constructing the fog-free image dataset are as follows: First, select multiple clear drone inspection images covering typical power line inspection scenarios as reference data sources and unify their resolution; then, calculate the pixel values ​​of the reference data sources to obtain darkened fog-free images, and then generate fog images of different fog conditions by adjusting the transmittance and atmospheric light values; finally, pair each fog-free image with the corresponding fog images of different fog conditions to construct a fog-free image dataset, and divide the fog-free image dataset into a training set and a test set.

[0023] In this embodiment, 1200 clear, fog-free drone inspection images covering typical power inspection scenarios such as high-voltage towers, transmission lines, and insulators are selected as reference data sources. The unified resolution is 1600×1200, which conforms to the actual acquisition specifications of drones and is different from general image datasets.

[0024] Specifically, the pixel values ​​of the reference data source are calculated using the following formula to obtain the darkened, fog-free image. inf = clear(p) 0.8 In the formula, inf represents the adjusted, darkened, haze-free image. "Clear" indicates a fog-free, clear drone inspection image. p represents a pixel value in the range [0, 255].

[0025] Specifically, the transmittance is adjusted by the following formula: t(p)=e-βd(p) In the formula, t(p) represents the adjusted transmittance. β represents the atmospheric scattering coefficient. d(p) represents the scene depth, which is obtained through a monocular image depth estimation algorithm with an error ≤5%.

[0026] Specifically, the atmospheric light value is adjusted by the following formula. A = cos(5t(p)) × γ In the formula, A represents the adjusted atmospheric light value. t(p) represents the adjusted transmittance. γ represents the range of values ​​[0.5, 0.95].

[0027] Specifically, the fog images of different fog conditions are generated using the following formula. hazy(p)=inf(p)t(p)+A(1-1.02×t(p)) In the formula, hazy(p) represents the hazy image with different haze types at pixel value p. inf(p) represents the darkened, hazy-free image at pixel value p. t(p) represents the transmittance at pixel value p. A represents the atmospheric light value at pixel value p. 1.02 represents the fog concentration correction factor.

[0028] Based on the atmospheric scattering characteristics of UAV aerial images, this invention derives specific fog image formulas for different fog conditions, introduces a fog concentration correction coefficient of 1.02, and combines a monocular depth estimation algorithm to control the transmittance error to ≤5%. The simulated fog conditions are more in line with the real atmospheric environment of UAV outdoor inspections, rather than directly reusing the general atmospheric scattering model.

[0029] In this embodiment, each fog-free image is paired with the corresponding foggy images of the four fog conditions, resulting in a total of 4800 foggy-fog-free image datasets.

[0030] This invention is the first to define a quantifiable fog condition classification standard for drone inspection scenarios (classifying dense fog / medium fog / thin fog / non-uniform fog according to atmospheric scattering coefficient β and spatial distribution variance), transforming the ambiguous fog condition types into calculable numerical features, and solving the problem of the lack of a unified judgment standard for fog conditions.

[0031] Specifically, the dual-dimensional data augmentation includes complexity enhancement and diversity enhancement. The impact of different data augmentation methods on the dehazing effect of UAV inspection images is compared to determine the optimal data augmentation method.

[0032] If a certain complexity enhancement method generates a low signal-to-noise ratio dense fog complex sample that performs better than other complexity enhancement methods when simulating extreme foggy scenarios, then this complexity enhancement method is the corresponding optimal complexity enhancement method.

[0033] Preferably, the complexity enhancement involves expanding the range of atmospheric scattering coefficient β to [0.8, 2.8] and the range of atmospheric light value A to [0.5, 1.8], thereby generating a complex sample of dense fog with low signal-to-noise ratio.

[0034] If a certain diversity enhancement method generates heterogeneous fog samples that are more effective at improving data diversity than other diversity enhancement methods, then that diversity enhancement method is the corresponding optimal diversity enhancement method.

[0035] Preferably, the diversity enhancement is achieved by using Fourier transform to fuse the low-frequency amplitude of the foggy image and the high-frequency phase of the fog-free image, and then smoothing the boundaries using Gaussian filtering with standard deviation to generate heterogeneous fog samples.

[0036] The Fourier transform is performed using the following formula. In the formula, I aug (p) represents the output image after frequency domain enhancement. F Indicates Fourier transform, M δ Represents the low-frequency mask, where , F A This indicates amplitude extraction from a foggy image. I a This refers to a fog-free image for reference. F P This represents phase extraction from a fog-free image. J b This indicates an image that needs to be enhanced or processed.

[0037] In this embodiment, a heterogeneous fog sample is generated by smoothing the boundary through Fourier transform and Gaussian filtering with a standard deviation of 1.2.

[0038] This invention is the first to propose a dual-dimensional enhancement approach combining complexity and diversity. It expands the range of values ​​for β and A for extreme foggy scenarios in power line inspection, generating low signal-to-noise ratio dense fog samples. By fusing the low-frequency amplitude of foggy images with the high-frequency phase of fog-free images through Fourier transform, heterogeneous fog samples are generated. This solves the problem of existing technologies having single data enhancement methods and being unable to cover complex fog conditions in power line inspection, significantly improving the coverage and adaptability of the dataset to real-world scenarios.

[0039] In this embodiment, the 4800 sets of fog-free and fog-containing image datasets are divided into 3840 training sets and 960 test sets in an 8:2 ratio.

[0040] Preferably, the training set is further augmented by random flipping and 512×512 cropping.

[0041] S2) Three diffusion dehazing models are designed, including the Adaptive Perception Diffusion Model (AFP-DM), the Wavelet Transform Residual Denoising Diffusion Model (WRD-DM), and the Basic Residual Denoising Diffusion Model (RDDM).

[0042] The above three diffusion dehazing models are used to learn the mapping rules from foggy images to fog-free images in the training set. Then, dehazing tests are performed on images with different fog conditions in the test set, and the optimal diffusion dehazing model corresponding to each fog condition is selected.

[0043] The Adaptive Perceptual Diffusion Model (AFP-DM) includes a haze data augmentation module, an improved MITNet physical feature extraction module, a distribution correction module, and a dynamic feature fusion module. The improved MITNet physical feature extraction module outputs a semi-dehazed image, a transmission map t(x), and atmospheric light value A; the dynamic feature fusion module adaptively fuses the physical modeling results with the diffusion generation features.

[0044] Specifically, the dynamic feature fusion module adaptively fuses the physical modeling results and diffusion-generated features using the following formula: x t-1 =t(x)·J t-1 +(1-t(x))·x t-1 In the formula, x t-1 This represents the noise prediction image from the previous time step of the diffusion model. t(x) represents the transmittance map of the image. J t-1 This represents a partially dehazed, clear image generated by the physical model.

[0045] The above formula adaptively integrates the physical characteristics of fog, such as atmospheric light and transmittance, with the generation characteristics of the diffusion model, rather than simply "physical model preprocessing + diffusion model denoising". This solves the problem of the physical model and deep learning model being disconnected and prone to artifacts in the existing technology.

[0046] The Wavelet Transform Residual Denoising and Diffusion Model (WRD-DM) adds a high-frequency enhancement module, a two-branch residual denoising and diffusion network, and a cross-frequency adjustment module to the AFP-DM model. The high-frequency enhancement module decomposes the image into one low-frequency sub-band (LL) and three high-frequency sub-bands (LH, HL, HH) using discrete wavelet transform, and enhances the high-frequency sub-bands through normalization and logarithmic transformation. The low-frequency branch of the two-branch residual denoising and diffusion network uses a 7×7 convolutional kernel and a 3-layer encoder-decoder, while the high-frequency branch uses a 3×3 convolutional kernel and a 2-layer encoder-decoder. The cross-frequency adjustment module achieves information interaction through an attention mechanism.

[0047] Specifically, the high-frequency sub-band is standardized using the following formula. In the formula, W(x,y) represents the normalized high-frequency subband coefficients. S(x,y) represents the original high-frequency subband coefficients. μ represents the mean value of the high-frequency subband coefficients. σ represents the standard deviation of the high-frequency subband coefficient. ε represents the minimum value to prevent the denominator from being zero.

[0048] Specifically, the high-frequency subband is enhanced by logarithmic transformation using the following formula. In the formula, W´(x,y) represents the high-frequency subband coefficients after logarithmic transformation enhancement. W(x,y) represents the normalized high-frequency subband coefficients. δ represents the compensation coefficient that ensures the stability of logarithmic operations.

[0049] Specifically, the cross-frequency adjustment module achieves information interaction through an attention mechanism using the following formula: In the formula, Attention(Q,K) represents the cross-frequency attention output. Q represents the query vector, i.e., the high-frequency feature. K represents the key vector, i.e., the low-frequency feature. d k This represents the feature dimension of the key vector.

[0050] The basic residual denoising diffusion model (RDDM) includes retaining the residual-noise dual diffusion channels, simplified to a network structure with 2 layers each for the encoder and decoder and 3×3 convolutional kernels, with the sampling step set to 3 steps.

[0051] Step S2 also includes model optimization, which involves training the model using the L2 loss function for AFP-DM and the L1 loss function for WRD-DM and the basic RDDM, adjusting the learning rate using a cosine annealing strategy, and optimizing the model performance by controlling the number of parameters for AFP-DM to be ≤80M, WRD-DM to be ≤60M, and the basic RDDM to be ≤30M.

[0052] The above three diffusion dehazing models are used to learn the mapping rules from foggy images to fog-free images in the training set. Specifically, the dehazing modes and parameters under different fog conditions are learned, and the hyperparameters such as network weights and diffusion steps of the model are continuously adjusted through backpropagation and other methods to minimize the error between the dehazed output image and the real fog-free image, thereby improving the model's dehazing ability.

[0053] Specifically, the method for selecting the optimal diffusion defogging model for each fog condition is as follows: by comparing the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of different diffusion defogging models under the corresponding fog conditions, if a certain diffusion defogging model has a larger PSNR value and a larger SSIM value under the corresponding fog conditions than the other two diffusion defogging models, then that diffusion defogging model is the optimal diffusion defogging model under the corresponding fog conditions.

[0054] The peak signal-to-noise ratio (PSNR) is calculated using the following formula: In the formula, PSNR represents the peak signal-to-noise ratio. MAX is 255. I(i,j) represents the pixel value of the dehazed image. K(i,j) represents the pixel value of the fog-free image.

[0055] The structural similarity (SSIM) is calculated using the following formula. In the formula, SSIM(x,y) represents structural similarity. μ x This represents the average pixel value of the dehazed image. μ y This represents the average pixel value of a true haze-free image. σ x This represents the standard deviation of pixels in the dehazed image. σ y This represents the standard deviation of pixels in a true haze-free image. σ xy This represents the covariance between the dehazed image and the true haze-free image.

[0056] If the adaptive sensing diffusion model performs better than the other two models in non-uniform fog scenarios, then this model is the optimal diffusion defogging model for non-uniform fog scenarios. If the wavelet transform residual denoising diffusion model performs better than the other two models in dense fog scenarios, then this model is the optimal diffusion denoising model for dense fog scenarios. If the basic residual denoising diffusion model performs better than the other two models in medium fog and light fog scenarios, then this model is the optimal diffusion denoising model for medium fog and light fog scenarios.

[0057] This invention, for the first time, designs differentiated diffusion model structures for three core fog conditions—dense fog, medium / light fog, and non-uniform fog—during UAV inspections, rather than simply modifying diffusion model parameters. The optimal diffusion defogging model for non-uniform fog scenarios is the Adaptive Perception Diffusion Model (AFP-DM), which integrates improved MITNet physical feature extraction and dynamic feature fusion to address the spatial distribution differences in non-uniform fog. The optimal diffusion defogging model for dense fog scenarios is the Wavelet Transform Residual Denoising Diffusion Model (WRD-DM), which adds frequency domain enhancement and cross-frequency attention interaction to strengthen the recovery of details of power equipment (insulator texture, conductor outline) under dense fog. The optimal diffusion defogging model for medium and light fog scenarios is the Basic Residual Denoising Diffusion Model (RDDM), which simplifies the network structure and reduces the sampling steps to 3, balancing defogging effectiveness and processing speed.

[0058] S3) Based on the fog-fog-free image dataset, extract multiple fog condition meta-features, use the multiple fog condition meta-features as input, and use predefined different fog condition types as labels to train a meta-learner for fog condition type classification.

[0059] The fog condition meta-features include five fog condition meta-features: mean fog concentration, image contrast, proportion of high-frequency components, transmittance variance, and atmospheric light value.

[0060] The average fog concentration is calculated using the following formula. In the formula, t mean This represents the average fog density across the entire image. H represents the image height. W represents the image width. t(x,y) represents the transmittance value at coordinate (x,y).

[0061] The image contrast is calculated using the following formula. In the formula, contrast represents the image contrast value. max(gray) represents the maximum value in the image's grayscale matrix. min(gray) represents the minimum value in the image grayscale matrix. mean(gray) represents the mean of the image grayscale matrix. gray represents the image grayscale value matrix.

[0062] The proportion of high-frequency components is the ratio of the sum of high-frequency subband coefficients after wavelet transform to the sum of all subband coefficients.

[0063] The transmittance variance is calculated using the following formula. In the formula, t var This represents the transmittance variance. H represents the image height. W represents the image width. t(x,y) represents the transmittance value at coordinates (x,y). t mean This represents the average fog density across the entire image.

[0064] The atmospheric light value is A, which is output by the physical feature extraction module.

[0065] This invention selects five types of meta-features that are strongly correlated with fog conditions in power line inspection images: mean fog concentration, transmittance variance, image contrast, proportion of high-frequency components, and atmospheric light value. These features are all derived based on the physical / frequency domain features of UAV inspection images, which differs from the general image features of existing technologies and makes the identification more accurate.

[0066] The meta-learner is constructed using the KNN algorithm. Specifically, it calculates the Euclidean distance between the test data and the training data, selects the k samples with the smallest distances, and uses the fog condition type with the highest frequency among them as the prediction result. Finally, the dataset is re-divided into training and test sets in a 7:3 ratio, and the Adam optimizer is used. =0.9, =0.999), initial learning rate 0.001, decaying to 50% of the current rate every 50 epochs, training 300 epochs, batch size=16 to complete training.

[0067] Specifically, the Euclidean distance between the test data and the training data is calculated using the following formula. In the formula, dist represents the Euclidean distance between the test sample and the training sample. f test,i This represents the i-th fog condition feature of the test sample. f train,i This represents the i-th fog condition feature of the training sample. k represents the number of samples.

[0068] This invention combines the KNN algorithm with a diffusion model to train a meta-learner for fog condition recognition, achieving an end-to-end process of automatically calling the optimal model for fog condition recognition without manual intervention. This "meta-learning + multi-diffusion model" architecture solves the core pain points of existing diffusion defogging technologies, namely "lack of adaptive mechanism and weak generalization ability," improving the PSNR / SSIM of the model under complex fog conditions by 5%-15% compared to traditional diffusion models.

[0069] S4) Collect inspection images of the drone to be defogged. Identify the fog type of the drone inspection images through the meta-learner constructed in step S3). Call the optimal diffusion defogging model corresponding to the fog type for processing and output a clear, fog-free image.

[0070] Preferably, the specific steps for the meta-learner to identify the fog condition type of the drone inspection image to be defogged include: first, cropping the input image to a resolution of 512×512 and normalizing the RGB3 channels to [0,1]; then extracting the fog condition meta-features and inputting them into the trained meta-learner to output the fog condition type.

[0071] Specifically, non-uniform fog uses the Adaptive Perception Diffusion Model (AFP-DM), dense fog uses the Wavelet Transform Residual Denoising Diffusion Model (WRD-DM), and medium / thin fog uses the Basic Residual Denoising Diffusion Model (RDDM). The AFP-DM process sequentially performs physical feature extraction, diffusion denoising, dynamic feature fusion, and output via a Sigmoid activation function. The WRD-DM process, based on the AFP-DM workflow, performs additional frequency domain enhancement using discrete wavelet transform, normalization, and logarithmic transform after physical feature extraction, and performs cross-frequency adjustment using an attention mechanism after dynamic feature fusion. The RDDM process directly outputs the image after diffusion denoising. Finally, the output image resolution is restored to its original size, and pixel values ​​are mapped back to [0, 255].

[0072] The present invention also designs a drone inspection image dehazing system based on a diffusion model, including a dataset construction and enhancement module 1, a model design and matching module 2, a meta-learner training module 3, and a dehazing processing module 4.

[0073] The dataset construction and enhancement module 1 is used to collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images to construct a fog-fog-free image dataset. It then performs two-dimensional data enhancement and divides the fog-fog-free image dataset into a training set and a test set.

[0074] The model design and matching module 2 is used to design three diffusion dehazing models, including the Adaptive Perceptual Diffusion Model (AFP-DM), the Wavelet Transform Residual Denoising Diffusion Model (WRD-DM), and the Basic Residual Denoising Diffusion Model (RDDM). The above three diffusion dehazing models are used to learn the mapping rules from foggy images to fog-free images in the training set. Then, dehazing tests are performed on images with different fog conditions in the test set, and the optimal diffusion dehazing model corresponding to each fog condition type is selected.

[0075] The meta-learner training module 3 is used to extract multiple fog condition meta-features based on a fog-free image dataset, and to train a meta-learner for fog condition type classification using the multiple fog condition meta-features as input and predefined different fog condition types as labels.

[0076] The defogging processing module 4 is used to acquire inspection images of the drone to be defogged, identify the fog condition type of the drone inspection image through the constructed meta-learner, call the optimal diffusion defogging model corresponding to the fog condition type for processing, and output a clear image without fog.

[0077] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for dehazing UAV inspection images based on a diffusion model, characterized in that, Includes the following steps: S1) Collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images. Then perform two-dimensional data augmentation to construct a fog-fog-free image dataset, and then divide the fog-fog-free image dataset into a training set and a test set. S2) Three diffusion dehazing models are designed, including the adaptive sensing diffusion model, the wavelet transform residual denoising diffusion model, and the basic residual denoising diffusion model. The above three diffusion dehazing models are used to learn the mapping rules from foggy images to fog-free images in the training set. Then, dehazing tests are performed on images with different fog conditions in the test set, and the optimal diffusion dehazing model corresponding to each fog condition is selected. S3) Based on the fog-fog-free image dataset, extract multiple fog condition meta-features, use the multiple fog condition meta-features as input, and use predefined different fog condition types as labels to train a meta-learner for fog condition type classification; S4) Collect inspection images of the drone to be defogged. Identify the fog type of the drone inspection images through the meta-learner constructed in step S3). Call the optimal diffusion defogging model corresponding to the fog type for processing and output a clear, fog-free image.

2. The method for dehazing UAV inspection images based on a diffusion model according to claim 1, characterized in that: In S1), fog conditions are classified into dense fog, medium fog, light fog, and non-uniform fog types according to the atmospheric scattering coefficient β and spatial distribution variance.

3. The UAV inspection image dehazing method based on diffusion model according to claim 2, characterized in that: In S1), the construction steps of the fog-fog-free image dataset are as follows: First, select multiple clear drone inspection images covering typical power inspection scenarios as reference data sources and unify the resolution; then, calculate the pixel values ​​of the reference data sources to obtain darkened fog-free images, and then generate fog images of different fog conditions by adjusting the transmittance and atmospheric light values; finally, form ordered pairs of each fog-free image with corresponding fog images of different fog conditions to construct the fog-fog-free image dataset.

4. The UAV inspection image dehazing method based on diffusion model according to claim 3, characterized in that: In S1), the fog images of different fog conditions are generated by the following formula. hazy(p)=inf(p)t(p)+A(1-1.02×t(p)) In the formula, hazy(p) represents the hazy image with different haze types at pixel value p. inf(p) represents the darkened, hazy-free image at pixel value p. t(p) represents the transmittance at pixel value p. A represents the atmospheric light value at pixel value p. 1.02 represents the fog concentration correction factor.

5. The UAV inspection image dehazing method based on diffusion model according to claim 4, characterized in that: In S1), the dual-dimensional data enhancement includes complexity enhancement and diversity enhancement. The complexity enhancement is to expand the range of atmospheric scattering coefficient and the range of atmospheric light value to generate complex samples of dense fog with low signal-to-noise ratio. The diversity enhancement is achieved by using Fourier transform to fuse the low-frequency amplitude of the foggy image with the high-frequency phase of the fog-free image, and then smoothing the boundaries using Gaussian filtering of the standard deviation to generate heterogeneous fog samples.

6. The UAV inspection image dehazing method based on diffusion model according to claim 1, characterized in that: In S2), the adaptive sensing diffusion model includes a haze data enhancement module, an improved MITNet physical feature extraction module, a distribution correction module, and a dynamic feature fusion module; The dynamic feature fusion module adaptively fuses the physical modeling results and diffusion-generated features using the following formula. x t-1 =t(x)·J t-1 +(1-t(x))·x t-1 In the formula, x t-1 This represents the noise prediction image from the previous time step of the diffusion model. t(x) represents the transmittance map of the image. J t-1 This represents a partially dehazed, clear image generated by the physical model; The wavelet transform residual denoising diffusion model adds a high-frequency enhancement module, a dual-branch residual denoising diffusion network, and a cross-frequency adjustment module to the adaptive sensing diffusion model.

7. The UAV inspection image dehazing method based on diffusion model according to claim 6, characterized in that: In S2), the method for selecting the optimal diffusion defogging model for each fog condition type is as follows: by comparing the peak signal-to-noise ratio and structural similarity of different diffusion defogging models under the corresponding fog condition, if the peak signal-to-noise ratio of a certain diffusion defogging model under the corresponding fog condition is larger than that of the other two diffusion defogging models, and the structural similarity value is also larger than that of the other two diffusion defogging models, then the diffusion defogging model is the optimal diffusion defogging model under the corresponding fog condition.

8. The method for dehazing UAV inspection images based on a diffusion model according to claim 1, characterized in that: In S3), the fog condition element features include five fog condition element features: average fog concentration, image contrast, proportion of high-frequency components, transmittance variance, and atmospheric light value.

9. The method for dehazing UAV inspection images based on a diffusion model according to claim 1, characterized in that: In S4), the specific steps for the meta-learner to identify the fog condition type of the drone inspection image to be defogged include: first, cropping the input image to a resolution of 512×512 and normalizing the RGB3 channels to [0,1]; then extracting the fog condition meta-features and inputting them into the trained meta-learner to output the fog condition type.

10. A dehazing system for UAV inspection images based on a diffusion model, characterized in that, It includes a dataset construction and enhancement module (1), a model design and matching module (2), a meta-learner training module (3), and a dehazing module (4). The dataset construction and enhancement module (1) is used to collect fog-free drone inspection images covering typical power inspection scenarios, and generate corresponding drone inspection images of different fog conditions based on the fog-free drone inspection images, construct a fog-fog-free image dataset, perform dual-dimensional data enhancement, and then divide the fog-fog-free image dataset into a training set and a test set. The model design and matching module (2) is used to design three diffusion dehazing models, including an adaptive perceptual diffusion model, a wavelet transform residual denoising diffusion model, and a basic residual denoising diffusion model; and to learn the mapping rules from foggy images to fog-free images in the training set using the above three diffusion dehazing models, and then to perform dehazing tests on images of different fog conditions in the test set, and select the optimal diffusion dehazing model corresponding to each fog condition type. The meta-learner training module (3) is used to extract multiple fog condition meta-features based on the fog-fog-free image dataset, and to train a meta-learner for fog condition type classification by taking the multiple fog condition meta-features as input and predefined different fog condition types as labels. The defogging processing module (4) is used to collect inspection images of the UAV to be defogging, identify the fog condition type of the UAV inspection image through the constructed meta-learner, call the optimal diffusion defogging model corresponding to the fog condition type for processing, and output a clear image without fog.

Images