Static image defogging method and system

Abstract

The invention discloses a static image defogging method and system. The method comprises the following steps: extracting shallow layer features of an image to be processed; performing high-level semantic feature extraction on the shallow-layer features to realize semantic segmentation, and completing region division; further carrying out feature extraction on the shallow features and carrying out illumination intensity estimation; respectively performing weighting processing on the feature map of the region division and the feature map of the illumination intensity estimation by utilizing an attention mechanism, splicing the two weighted feature maps, returning a result to the head ends of a plurality of organizational structures which are connected in sequence, and connecting each organizational structure to the attention mechanism in a jumping manner, feature fusion is carried out after the feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple organizational structures which are connected in sequence are weighted through an attention mechanism, a fusion result is input into a first feature extraction layer, and a final defogged image is output; the method has the advantages that detail missing is avoided, non-uniformly distributed fog-containing images can be processed, and the scene adaptability is high.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a static image defogging method and system. Background Art

[0002] In the power industry, an increasing number of automated inspection and recognition solutions are emerging. However, due to the vast reach of transmission facilities and significant regional variations in climate, improving image processing and achieving more accurate anomaly detection and analysis is crucial. Dehazing visible light images is one such approach. While many machine learning methods currently exist for image dehazing, the following challenges remain:

[0003] 1. The dehazing method based on dark channel priors is less effective in processing areas such as the sky. This dehazing algorithm poorly estimates atmospheric brightness and easily results in loss of details.

[0004] 2. Multi-image restoration based on light polarization properties: This method uses multiple images with different polarization angles to estimate atmospheric light and transmittance. It has difficulty handling dynamic scenes and makes strong assumptions about atmospheric light, which makes it difficult to handle images with non-uniform fog distribution.

[0005] 3. Wavelet transform image enhancement. This defogging method relies too much on manually adjusted dynamic parameters, is difficult to process dense fog images, and is not adaptable to different scenes.

[0006] 4. Image dehazing methods based on the Retinex algorithm defog the image by decomposing it into incident light components and reflected light components. However, in areas with sudden changes in illumination, the Gaussian filter's inaccurate estimation of illumination components can easily lead to bright / dark bands appearing at the edges. Furthermore, since the Gaussian kernel scale and filter template size require manual re-adjustment for scenes with different fog concentrations, the method is not very adaptable to changing scenes. For example, Chinese Patent Publication No. CN116012253A discloses an image dehazing method based on a convolutional neural network fused with a Transformer. This method lacks brightness estimation and is difficult to adapt to scenes with varying brightness changes. Summary of the Invention

[0007] The technical problem to be solved by the present invention is that the existing image defogging methods are prone to cause loss of details, are difficult to process non-uniformly distributed foggy images, and have poor scene adaptability.

[0008] The present invention solves the above technical problems through the following technical means: a static image defogging method, comprising:

[0009] S1, extract shallow features of the image to be processed;

[0010] S2, extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division;

[0011] S3, further extracting features from shallow features and estimating light intensity;

[0012] S4. The feature map of region division and the feature map of illumination intensity estimation are weighted by the attention mechanism respectively, the two weighted feature maps are spliced, and the results are returned to the head end of the multiple sequentially connected organizational structures. Each organizational structure is jump-connected to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure is jump-connected to the second feature extraction layer. The processing process of each basic structure is first convolution operation, then normalization operation, and the normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer.

[0013] The present invention first performs shallow feature extraction, and then performs region division and light intensity estimation respectively, and then performs attention mechanism weighting and returns it to the organizational structure. The feature map of region division, the feature map of light intensity estimation and the feature maps output by multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fusion is performed, so that the whole process performs multi-layer feature extraction to avoid loss of details, and semantic segmentation is achieved by performing high-level semantic feature extraction on shallow features, and region division is achieved, which ensures that the key areas and non-key areas in the image are assigned different weights by the attention mechanism. At the same time, through light intensity estimation, it is ensured that parts of the image with different light intensities can be assigned different weights by the attention mechanism, thereby avoiding the problem of non-uniform defogging, and through brightness estimation and region division, it can adapt to different brightness change scenes and interference situations where the inspection image contains a large amount of useless information, and has strong scene adaptability.

[0014] Furthermore, S1 includes:

[0015] The image to be processed is subjected to a convolution operation with a convolution kernel size of 3×3, and then processed with the ReLU activation function to extract shallow features.

[0016] Furthermore, S2 includes:

[0017] The shallow features are first input into the AOD-Net model, and the output results of the AOD-Net model are input into the Mobile-Net V2 model. The Mobile-Net V2 model implements semantic segmentation and distinguishes different areas of the image to be processed.

[0018] Furthermore, the expansion factor of the Mobile-Net V2 model was adjusted from 1 to 0.75, the last three bottleneck layers of the Mobile-Net V2 model were removed, the 3*3 depthwise convolution of the Mobile-Net V2 model was replaced with a 5*1 decomposed convolution plus a 1*5 decomposed convolution, and the FReLU activation function was used instead of the ReLU6 activation function of the Mobile-Net V2 model.

[0019] Furthermore, S3 includes:

[0020] After convolution and pooling operations on the shallow features, they are input into the GeoCalib model to estimate the vertical direction and internal parameters of the image. Then, the vertical direction and internal parameters of the image are input into the StableDiffusion diffusion model to estimate the light intensity.

[0021] Furthermore, the attention mechanism is a sequentially connected spatial attention module and a pixel attention module.

[0022] Furthermore, the activation function sampled by the activation function layer is a ReLU activation function.

[0023] Furthermore, the static image defogging method further includes:

[0024] S5. Construct a loss function and perform model training. When the loss function is minimized, stop training to obtain a trained model. Input the image to be processed collected in real time into the trained model to obtain a defogging image.

[0025] Furthermore, the loss function is constructed as follows:

[0026] The mean square error or absolute error between the model prediction value and the true value is used as the basic loss. The attention weight map output after feature fusion is multiplied by the basic loss to obtain the weighted loss of each pixel position. The weighted losses of all pixel positions are averaged to obtain the final loss.

[0027] The present invention also provides a static image defogging system, comprising:

[0028] A shallow feature extraction module, used to extract shallow features from the image to be processed;

[0029] The region division module is used to extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division;

[0030] Light estimation module, used to further extract features from shallow features and estimate light intensity;

[0031] The attention mechanism module is used to perform weighted processing on the feature map of region division and the feature map of illumination intensity estimation respectively using the attention mechanism, splice the two weighted feature maps, and return the results to the head end of multiple sequentially connected organizational structures. Each organizational structure is jump-connected to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure is jump-connected to the second feature extraction layer. The processing process of each basic structure is first a convolution operation and then a normalization operation. The normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer.

[0032] Furthermore, the shallow feature extraction module is also used to:

[0033] The image to be processed is subjected to a convolution operation with a convolution kernel size of 3×3, and then processed with the ReLU activation function to extract shallow features.

[0034] Furthermore, the region division module is also used to:

[0035] The shallow features are first input into the AOD-Net model, and the output results of the AOD-Net model are input into the Mobile-Net V2 model. The Mobile-Net V2 model implements semantic segmentation and distinguishes different areas of the image to be processed.

[0036] Furthermore, the expansion factor of the Mobile-Net V2 model was adjusted from 1 to 0.75, the last three bottleneck layers of the Mobile-Net V2 model were removed, the 3*3 depthwise convolution of the Mobile-Net V2 model was replaced with a 5*1 decomposed convolution plus a 1*5 decomposed convolution, and the FReLU activation function was used instead of the ReLU6 activation function of the Mobile-Net V2 model.

[0037] Furthermore, the light estimation module is further configured to:

[0038] After convolution and pooling operations on the shallow features, they are input into the GeoCalib model to estimate the vertical direction and internal parameters of the image. Then, the vertical direction and internal parameters of the image are input into the StableDiffusion diffusion model to estimate the light intensity.

[0039] Furthermore, the attention mechanism is a sequentially connected spatial attention module and a pixel attention module.

[0040] Furthermore, the activation function sampled by the activation function layer is a ReLU activation function.

[0041] Furthermore, the static image defogging system also includes a training module, which is used to construct a loss function and perform model training. When the loss function is minimized, the training is stopped to obtain a trained model. The image to be processed collected in real time is input into the trained model to obtain a defogged image.

[0042] Furthermore, the loss function is constructed as follows:

[0043] The mean square error or absolute error between the model prediction value and the true value is used as the basic loss. The attention weight map output after feature fusion is multiplied by the basic loss to obtain the weighted loss of each pixel position. The weighted losses of all pixel positions are averaged to obtain the final loss.

[0044] The advantages of the present invention are:

[0045] (1) The present invention first performs shallow feature extraction, and then performs region division and illumination intensity estimation respectively, and then performs attention mechanism weighting and returns it to the organizational structure, and the feature map of region division, the feature map of illumination intensity estimation and the feature map output by multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fusion is performed, so that the whole process performs multi-layer feature extraction to avoid loss of details, and semantic segmentation is achieved by extracting high-level semantic features from shallow features, and region division is achieved, ensuring that the key areas and non-key areas in the image are assigned different weights by the attention mechanism, and at the same time, through illumination intensity estimation, ensuring that the parts of the image with different illumination intensities can be assigned different weights by the attention mechanism, thereby avoiding the problem of non-uniform defogging, and through brightness estimation and region division, it can adapt to different brightness change scenes and the interference situation of the inspection image containing a large amount of useless information, and has strong scene adaptability.

[0046] (2) The weighted processing of the feature maps after region division and light intensity estimation in the present invention returns the results to the organizational structure feature extraction part of the network. Through the region division module and the area downsampling of the light estimation module, the computational cost is saved and the real-time performance of the calculation results is guaranteed. The design of the key features (i.e., the processing before the attention mechanism) + attention mechanism improves the efficiency of other feature extraction and guides the dynamic adjustment of the loss function to ensure the image output effect.

[0047] (3) The present invention adopts a spatial attention mechanism and a pixel attention mechanism. The spatial attention mechanism focuses on the effective area of ​​the image, which is in line with the application scenario. The pixel attention mechanism performs weighted operations pixel by pixel, which can effectively retain the details of the image. The spatial attention and pixel attention are sequentially connected and the weights are superimposed to achieve the highlighting of the details of the key content. Spatial attention focuses on regional information and is suitable for the first layer of weighted processing, while pixel attention performs weighted processing on the weighted key areas pixel by pixel on the basis of the first layer of weighted processing, which is suitable as the second layer of the attention mechanism. In addition, since the pixel attention mechanism often assigns weights to each pixel, it consumes a lot of computing power. By first performing weighted processing through the spatial attention mechanism, the subsequent pixel attention mechanism processing can be adjusted through dynamic sparse computing and other methods to save computing costs.

[0048] (4) The present invention outputs a simple dehazed image through the AOD-Net model. The model is very lightweight and has a very fast output speed, which can meet the requirements of subsequent semantic segmentation. It is then paired with the independently improved Mobile-Net V2 network to extract high-level semantic features, thereby achieving rapid region segmentation.

[0049] (5) The present invention adopts the GeoCalib model to estimate the vertical direction and internal parameters from a single image, and uses the StableDiffusion diffusion model to estimate the light intensity based on the diffusion prior. Using this model as the main model of the light estimation module has good complex lighting robustness and is more suitable for industrial or field scenes with variable noise.

[0050] (6) The organizational architecture and basic architecture of the present invention set up multiple layers of feature extraction parts and use residual learning to directly connect them with the subsequent attention mechanism module. On the one hand, the organizational architecture receives the feature map transmitted by the attention mechanism and reduces the number of convolutions in low-weight areas through skip connections to remove details. On the other hand, the organizational architecture and basic architecture allow some features to skip redundant extraction operations, which not only avoids the occurrence of problems such as gradient vanishing, but also enhances the network's learning ability.

[0051] (7) The first feature extraction layer of the present invention is the last part before output. It receives the result of feature fusion output, recalibrates the channel and spatial dimensions of the result, implements consistency constraints, and suppresses fusion artifacts. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A system architecture diagram of a static image defogging method disclosed in an embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the weighted superposition principle in a static image defogging method disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0055] Example 1

[0056] like Figure 1 As shown, embodiment 1 of the present invention provides a static image defogging method, comprising:

[0057] S1. Extract shallow features from the image to be processed. Shallow feature extraction is a preprocessing module for the entire neural network. Its main purpose is to extract primary features of the image and provide high-quality initial features for subsequent modules. The fog image is passed through a convolutional layer and activated with the ReLU function to generate a shallow feature map. The number of channels is expanded to a higher dimension. The specific process is as follows:

[0058] First, the fog image, or the image to be processed, is input into a convolutional layer. This convolutional layer uses a 3×3 kernel, a stride of 1, and padding (i.e., padding at the edges to ensure the output feature map has the same size as the input). The convolution operation can be expressed as: O = f(I, K), where O is the output feature map, I is the input fog image, K is the convolution kernel, and f represents the convolution operation. After the convolution operation, the ReLU activation function f(x) = max(0, x) is used to expand the number of channels in the output feature map to a higher dimension. This introduces nonlinearity, enabling the network to learn complex features, and offers the advantages of simple computation and fast convergence. The shallow feature extraction module, through convolution operations and activation functions, extracts primary features such as edges and textures from the image, providing foundational features for subsequent region segmentation, light estimation, and attention mechanisms. These primary features are crucial for subsequent operations such as semantic segmentation, light estimation, and feature weighting.

[0059] S2. Extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division. The main task of the region division module is to achieve defogging preprocessing and semantic segmentation. The specific process is as follows:

[0060] First, we selected the AOD-Net model to output a simple dehazed image. AOD-Net is an end-to-end dehazing model whose fundamental principle is to directly recover a haze-free image from a haze image by learning the inverse of the atmospheric scattering model. Although AOD-Net's dehazing performance is relatively poor due to its overreliance on strong assumptions about atmospheric light, its lightweight model and fast output speed meet the requirements of subsequent semantic segmentation.

[0061] Subsequently, the improved Mobile-Net V2 network is used to extract high-level semantic features. Mobile-Net V2 utilizes an inverted residual structure and a linear bottleneck design, enabling efficient image semantic feature extraction. In this module, the present invention makes multiple improvements to the original Mobile-Net V2 network, making the model more lightweight and significantly increasing processing speed. Specific improvements are as follows:

[0062] A hyperparameter in the network channel, namely the expansion factor, was optimized: considering that the subsequent modules have lower requirements for semantic segmentation accuracy, the expansion factor was reduced from 1 to 0.75. At the same time, the last three (deep) bottleneck layers in the Mobile-Net V2 network were removed. Here, the deep bottleneck is responsible for more advanced features, but it also takes up a lot of computation and parameters. By cutting redundant parts, unnecessary computation is reduced while keeping the basic functions unaffected. Secondly, some of the 3*3 depth convolutions used in the original network are replaced with mixed 5*1+1*5 decomposition convolutions to improve computational efficiency. Finally, the FReLU activation function is used instead of the original ReLU6 activation function to improve feature richness.

[0063] In the region segmentation module, the low-level, mid-level, and high-level semantic features of the image are gradually extracted through multiple feature extraction layers in the improved Mobile-Net V2 network to achieve semantic segmentation.

[0064] The region segmentation module performs preliminary dehazing and semantic segmentation on the fog image, extracting key areas within the image (such as power transmission and transformation facilities). This module then generates new feature maps that are fed into the attention mechanism module for weighted processing. Accurately extracting these key areas is crucial for subsequent feature weighting and anomaly detection.

[0065] S3. Further extract features from shallow features and estimate light intensity. The main task of the light estimation module is to extract features and estimate light intensity. The specific process is as follows:

[0066] First, the feature map output by the shallow feature extraction module is received and further convolution and pooling operations are performed on it. The convolution operation uses a 3×3 convolution kernel with a stride of 1 and the same padding. The pooling operation uses max pooling with a pooling window size of 2×2 and a stride of 2. Through convolution and pooling operations, features are gradually extracted and the size of the feature map is reduced, while the semantic information of the features is enhanced.

[0067] Next, the GeoCalib model is used to estimate the vertical direction and intrinsic parameters from a single image. By analyzing the image's geometric and lighting information, the GeoCalib model estimates the vertical direction and intrinsic parameters, providing a foundation for light intensity estimation. Next, the StableDiffusion diffusion model is employed. This model uses a diffusion prior to estimate light intensity. This prior assumes that light in an image has certain diffusion characteristics. By analyzing and modeling the image, it is possible to estimate light intensity in different regions. Specifically, a light diffusion model is constructed based on image grayscale values ​​and gradient information, and an optimization algorithm is used to solve for the light intensity distribution. The implementation process is as follows: First, the GeoCalib model is used to analyze the image and estimate the vertical direction and intrinsic parameters. The GeoCalib output is then encoded into a vector, concatenated with the noisy input, and fed into the StableDiffusion diffusion model. The diffusion model generates a light distribution based on information such as scene geometry and outputs the result. Using this model as the primary model in the light estimation module demonstrates excellent robustness to complex lighting conditions and is particularly suitable for industrial or outdoor scenes with highly variable noise.

[0068] Through the above steps, the light estimation module estimates the illumination intensity of different regions in the image and generates corresponding feature maps, which are then passed to the attention mechanism module. This illumination intensity information plays an important role in subsequent feature weighting and dehazing, enabling differentiated processing of regions with different illumination intensities. It should be noted that the GeoCalib model and the StableDiffusion diffusion model are both mature existing technologies and will not be described in detail here.

[0069] This invention utilizes a separately designed region segmentation module and light estimation module to extract features from the original image. The output feature map is then directly fed into the subsequent attention mechanism for processing. This design ensures effective defogging of key image regions (i.e., areas related to power transmission and transformation facilities). Furthermore, it ensures that different light intensities within the image are assigned different weights, thereby avoiding the uneven defogging problem that can occur with traditional defogging methods.

[0070] S4. The feature map of region division and the feature map of illumination intensity estimation are weighted by the attention mechanism respectively, the two weighted feature maps are spliced, and the results are returned to the head end of the multiple sequentially connected organizational structures. Each organizational structure jumps to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure jumps to the second feature extraction layer. The processing process of each basic structure is to perform a convolution operation first, and then a normalization operation. The normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer. The specific processing process is as follows:

[0071] The attention mechanism module is the core of the entire neural network. Its main task is to perform weighted processing on the extracted features and connect other modules in series to achieve efficient collaboration. The attention mechanism uses a sequentially connected spatial attention mechanism and pixel attention mechanism. First, the processing process of the spatial attention mechanism (SA) is as follows:

[0072] Perform weighted processing on the input feature map. For an input feature map f of size H×W×C, perform global maximum pooling and global average pooling in the channel dimension to obtain two feature maps of size H×W×1. The global maximum pooling and global average pooling operations can be expressed as:

[0073]

[0074] Among them, P max and P avg They are the feature maps after global maximum pooling and global average pooling, h and w are the height and width coordinates of the feature map, respectively, and c is the channel coordinate.

[0075] The pooling results are concatenated by channel to obtain a feature map of size H × W × 2. The result is then convolved with a kernel size of 1 × 1 and an output channel of 1. The convolution operation can be expressed as: O = f(P, K), where O is the output feature map after the convolution operation, P is the concatenated feature map, and K is the convolution kernel.

[0076] Finally, the spatial attention weight matrix MS is obtained through the Sigmoid activation function. The Sigmoid activation function is defined as: σ(x) = 1 / (1+e -x )This function compresses the input value to the interval [0,1], indicating the attention weight of the corresponding position.

[0077] In the above process, an input feature map f of size H × W × C is first subjected to global max pooling and global average pooling in the channel dimension, resulting in two feature maps of size H × W × 1. The pooling results are then concatenated by channel to obtain feature maps of size H × W × 2. Finally, a convolution operation is performed on the result. A sigmoid activation function is then used to obtain the spatial attention weight matrix MS. This simple weighting process allows the feature maps input from the region partitioning module, light estimation module, and organizational structure to be weighted for the first time, thus completing the channel attention mechanism.

[0078] The attention mechanism module uses a sequential connection method, first passing through the spatial attention mechanism (SA) and then passing it to the pixel attention mechanism (PA). This is because PA often has to assign weights to each pixel, which consumes a lot of computing power. By first performing weighted processing through SA, the subsequent pixel attention mechanism processing can be adjusted through dynamic sparse computing and other methods to save computing costs. The spatial attention mechanism focuses on the valid area of ​​the image and performs the first weighted processing on the input feature map; the pixel attention mechanism, based on the spatial attention mechanism, performs weighted operations pixel by pixel to further highlight the detailed features of the key areas. Specifically, the pixel attention mechanism can be expressed as:

[0079] OPA(h,w)=α(h,w)×OSA(h,w)

[0080] Among them, OPA is the feature map output by the pixel attention mechanism, α(h,w) is the pixel attention weight, and OSA is the feature map output by the spatial attention mechanism.

[0081] To reduce computational overhead, a dynamic sparse computing strategy is introduced. First, SA generates an attention map, then sets a threshold to generate a binary mask Mmask. Finally, PA performs weighted processing on regions where the binary mask is 1, while assigning a uniform value to the rest. This allows for coarse-to-fine weighted processing through dynamic sparse computing, reducing the number of pixels requiring weight assignment and computational complexity. The dynamic sparse computing strategy can be expressed as:

[0082] M mask (h,w)={1,if MS(h,w)>threshold;0,otherwise}

[0083] OPA(h,w)={α(h,w)×OSA(h,w),ifM mask (h,w)=1;default_value,otherwise}

[0084] Among them, M MaskIt is a binary mask, threshold is the set threshold, and default_value is the default value.

[0085] A comprehensive attention mechanism has been designed, tailored to the application scenario and offering a smooth interactive experience. This not only ensures that details in key areas are not lost, saving computational overhead, but also cascades modules such as the loss function and feature extraction. First, it employs both spatial and pixel-level attention mechanisms. The spatial attention mechanism focuses on valid image regions, making it ideal for the application scenario. The pixel-level attention mechanism performs weighted operations pixel by pixel, effectively preserving image details. Second, it employs sequential connections and weighted stacking to highlight key details. Spatial attention focuses on regional information, making it suitable for the first layer of weighted processing. Pixel-level attention, building on the first layer of weighted processing, applies pixel-by-pixel weighting to the weighted key areas, making it suitable for the second layer of the attention mechanism. Through the interaction between modules in the network, computational overhead is reduced in the application scenario, leveraging the inherent simplification capabilities of the neural network structure. With the attention mechanism at its core, the simplified computational capabilities of the neural network are exploited through dynamic sparse computation strategies and skip connections. This ensures real-time computational results while freeing up excess computational power for detailed processing, enabling each module to achieve more refined operations.

[0086] The feature maps output by the region division module and the light estimation module are processed by the attention mechanism, which is equivalent to weight superposition, such as Figure 2 As shown in the figure, the image features are extracted by the region division module and the light estimation module respectively, and the weights are assigned respectively using the attention mechanism. Then, the two feature maps are spliced ​​and returned to the organizational structure. After splicing, the same region of the feature map may have four weight ratios. Figure 2 This clearly displays four possible weightings within the image and assigns them a priority order. This weighting influences the feature extraction of the preceding structure. The weighting ratios set for the region segmentation module and the light estimation module can be adjusted. Key areas of the image, regardless of lighting intensity, require more refined processing, while less important background areas can be blurred and uniformly assigned. Consequently, the region segmentation results are given greater weight. This structure can be considered a simple channel attention mechanism.

[0087] The organizational architecture connects and coordinates the various modules in a neural network. The organizational and basic architectures incorporate multiple layers of feature extraction and utilize residual learning to directly connect them to the subsequent attention mechanism modules. On one hand, the organizational architecture receives feature maps from the attention mechanism and uses skip connections to reduce the number of convolutions in low-weight regions, removing details. On the other hand, the organizational and basic architectures allow some features to skip redundant extraction operations, preventing problems like vanishing gradients while enhancing the network's learning capabilities. The organizational architecture works as follows:

[0088] The deep residual network (ResNet) is used as the basic model of the organizational architecture. The organizational architecture receives the feature map from the attention mechanism module and directly connects it to the subsequent feature extraction module through skip connections. The skip connection can be expressed as:

[0089] O=f(X)+X

[0090] Among them, O is the output feature map, X is the input feature map, and f represents the operation of the feature extraction module.

[0091] The organizational architecture module consists of multiple sequentially connected organizational architectures. Within the organizational architecture, the base architecture first performs a convolution operation on the input feature map to extract preliminary features. This operation uses a 3*3 convolution kernel with a stride of 1 and padding of the same size. Max pooling with a stride of 2 is used for pooling. Next, a ReLU activation function is used to introduce nonlinearity. Finally, the multiple layers extracted from the base architecture are fused to produce a composite feature map. Each organizational architecture contains multiple base architectures, each used for multi-layer feature extraction. The specific working principle of the base architecture is as follows: First, the convolution layer passes through the normalization layer, where the convolved feature map is normalized to accelerate network training. Next, the activation layer passes through the ReLU activation function, introducing nonlinearity. Finally, the pooling layer downsamples the feature map, reducing its size and enhancing its semantic information. Each layer in the base architecture also applies residual learning to reduce unnecessary convolution operations. Based on this architectural design, a single organizational architecture relies on multiple base architectures to progressively extract features, and the entire organizational architecture module, in turn, relies on multiple organizational architectures to achieve multi-scale feature extraction.

[0092] This design allows some features to skip redundant extraction operations and be passed directly to subsequent modules, avoiding problems like vanishing gradients while enhancing the network's ability to learn complex features. Furthermore, the organizational architecture processes and extracts many high-level features, providing multi-scale feature information for the final feature fusion process.

[0093] The multi-scale feature fusion part is responsible for integrating features at different levels and scales to achieve a more comprehensive and accurate feature representation. The multi-scale feature fusion part first receives the weighted results output by the previous attention mechanism and the feature map of the organizational structure. The feature maps output by multiple group structures are then spliced ​​along the channel dimension. Finally, the spliced ​​result is element-by-element multiplied with the corresponding channel pixel weight to achieve adaptive weighted fusion of features. Such a multi-scale feature fusion part complements the attention-driven dynamic weight allocation with cross-level features, combines shallow details with deep semantics, and solves the problem of insufficient feature utilization. The previous output results can be well integrated, which is conducive to subsequent testing and output. Its working principle is as follows:

[0094] First, the weighted results of the attention mechanism module are received, which mainly include the feature maps processed by the light estimation module, region segmentation module, and organization architecture. These feature maps contain semantic information and detailed information at different levels.

[0095] Then, the feature maps output after attention mechanism processing are concatenated along the channel dimension. The concatenation operation can be expressed as:

[0096] O = concat(F1, F2, ..., Fn)

[0097] Among them, O is the concatenated feature map, and F1, F2, ..., Fn are different feature maps.

[0098] Finally, the concatenated result is multiplied element-by-element with the corresponding channel pixel weight to achieve adaptive weighted fusion of features. This process can be expressed as:

[0099] Ofuse(h,w,c)=O(h,w,c)×W(c)

[0100] Among them, Ofuse is the fused feature map, W(c) is the channel pixel weight, h and w are the height and width coordinates of the feature map respectively, and c is the channel coordinate.

[0101] Through such multi-scale feature fusion, we can fully utilize the advantages of features at different levels, combine shallow detail features with deep semantic features, solve the problem of insufficient feature utilization, and provide richer feature information for subsequent anomaly detection.

[0102] S5. Construct a loss function and train the model. When the loss function is minimized, training is stopped, resulting in a trained model. The real-time image to be processed is then fed into the trained model to produce a dehazed image. The loss function is constructed by using the mean squared error (MSE) or absolute error (ABSE) between the model's predicted value and the true value as the base loss. The attention weight map output after feature fusion is multiplied by the base loss to obtain a weighted loss for each pixel. The weighted losses for all pixel locations are averaged to obtain the final loss.

[0103] Although the network adopts an attention mechanism to retain the details of the key parts of the image and blur the background part of the image, if the loss function is not selected well in the last step of the training process, it may still cause the details of the key parts of the output image to be lost, thereby reducing the accuracy of subsequent abnormality recognition. An attention-guided weighted loss function can be used to dynamically adjust the loss contribution of different areas to achieve the training purpose. Directly use the previously generated attention weight map, then calculate the basic loss pixel by pixel, multiply the two and take the average. Using such a designed weighted loss function, the final output image can be focused on the high-weight key areas in the region segmentation module. Specifically, the loss function construction process is as follows:

[0104] First, we use the attention weight map generated previously to calculate the base loss pixel by pixel. The base loss can use common loss functions such as mean square error (MSE) or absolute error (MAE). For example, the mean square error can be expressed as:

[0105] MSE=(1 / N)×∑(Y_pred(i)-Y_true(i)) 2

[0106] Where N is the total number of pixels, Y_pred(i) is the predicted value of the i-th pixel position, and Y_true(i) is the true value of the i-th pixel position.

[0107] Then, the base loss is multiplied by the attention weight map to obtain the weighted loss. The weighted loss can be expressed as:

[0108] L_weighted(i)=L_base(i)×A(i)

[0109] Among them, L_weighted(i) is the weighted loss at the i-th pixel position, L_base(i) is the base loss at the i-th pixel position, and A(i) is the attention weight at the i-th pixel position.

[0110] Finally, take the average of the weighted losses to get the final loss value. The final loss can be expressed as:

[0111] L_final=(1 / N)×∑L_weighted(i)

[0112] This attention-guided weighted loss function can dynamically adjust the loss contribution of different regions, so that the network pays more attention to the feature learning of key areas during training, thereby improving the quality of the output image and the accuracy of anomaly detection.

[0113] The model training process of the present invention also includes system testing, which conducts comprehensive system testing and training, including functional testing of each module, module performance testing, and system integration testing. After the functional testing is completed, the model is trained to ensure that each module is working as expected. After the training results are verified to be normal, the performance test is then carried out. The performance indicators of each module and system are monitored, including response speed, output speed, peak signal-to-noise ratio, etc. Finally, the system integration test is carried out to compare the system performance results and put them into use in the power inspection system.

[0114] Through the above technical solutions, the present invention achieves efficient defogging of power transmission system inspection images by designing a CNN-based image defogging network, combined with a comprehensive attention mechanism and feature extraction module. Separately designed region segmentation and light estimation modules accurately extract key image regions and light intensity information, which are then fed into the attention mechanism for weighted processing. This method highlights key image regions, avoids the uneven defogging and haloing that can occur with traditional defogging methods, and improves defogging effectiveness. By combining key feature extraction with the attention mechanism and connecting other neural network structures in series, multiple modules are organically integrated. This design not only enhances inter-module collaboration but also reduces unnecessary computation and improves network efficiency. A comprehensive attention mechanism, including spatial and pixel attention, is designed, employing sequential connections and a dynamic sparse computation strategy. This attention mechanism effectively leverages its role within the overall neural network, highlighting key details while reducing computational overhead and enhancing network performance. The defogging network is optimized for the characteristics of power transmission system inspection images. By redesigning the attention mechanism, improving the optimization method, and adjusting the loss function, the network can better adapt to inspection scenarios and improve the detection of anomalies in power transmission and transformation facilities. Through the interaction between the modules in the network, the simplification capabilities of the neural network itself, such as dynamic sparse computing strategies and jump connections, are fully utilized. While ensuring the real-time calculation results, these methods channel excess computing power to detailed processing, thereby improving the network's computational efficiency. Therefore, the present invention can effectively solve the key problems existing in existing power inspection image defogging algorithms, improve image processing effects and anomaly detection accuracy, and promote the automation and intelligent development of the power inspection industry.

[0115] Example 2

[0116] Based on Example 1, Example 2 of the present invention further provides a static image defogging system, including:

[0117] A shallow feature extraction module, used to extract shallow features from the image to be processed;

[0118] The region division module is used to extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division;

[0119] Light estimation module, used to further extract features from shallow features and estimate light intensity;

[0120] The attention mechanism module is used to perform weighted processing on the feature map of region division and the feature map of illumination intensity estimation respectively using the attention mechanism, splice the two weighted feature maps, and return the results to the head end of multiple sequentially connected organizational structures. Each organizational structure is jump-connected to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure is jump-connected to the second feature extraction layer. The processing process of each basic structure is first a convolution operation and then a normalization operation. The normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer.

[0121] Specifically, the shallow feature extraction module is also used to:

[0122] The image to be processed is subjected to a convolution operation with a convolution kernel size of 3×3, and then processed with the ReLU activation function to extract shallow features.

[0123] Specifically, the region division module is also used to:

[0124] The shallow features are first input into the AOD-Net model, and the output results of the AOD-Net model are input into the Mobile-Net V2 model. The Mobile-Net V2 model implements semantic segmentation and distinguishes different areas of the image to be processed.

[0125] More specifically, the expansion factor of the Mobile-Net V2 model is adjusted from 1 to 0.75, the last three bottleneck layers of the Mobile-Net V2 model are removed, the 3*3 depthwise convolution of the Mobile-Net V2 model is replaced with a 5*1 decomposed convolution plus a 1*5 decomposed convolution, and the FReLU activation function is used instead of the ReLU6 activation function of the Mobile-Net V2 model.

[0126] Specifically, the light estimation module is also used to:

[0127] After convolution and pooling operations on the shallow features, they are input into the GeoCalib model to estimate the vertical direction and internal parameters of the image. Then, the vertical direction and internal parameters of the image are input into the StableDiffusion diffusion model to estimate the light intensity.

[0128] Specifically, the attention mechanism is a sequentially connected spatial attention module and a pixel attention module.

[0129] Specifically, the activation function sampled by the activation function layer is the ReLU activation function.

[0130] Specifically, the static image defogging system also includes a training module, which is used to construct a loss function and perform model training. When the loss function is minimized, the training is stopped to obtain a trained model. The image to be processed collected in real time is input into the trained model to obtain a defogged image.

[0131] More specifically, the process of constructing the loss function is:

[0132] The mean square error or absolute error between the model prediction value and the true value is used as the basic loss. The attention weight map output after feature fusion is multiplied by the basic loss to obtain the weighted loss of each pixel position. The weighted losses of all pixel positions are averaged to obtain the final loss.

[0133] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A static image defogging method, characterized in that: include: S1, extract shallow features of the image to be processed; S2, extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division; S3, further extracting features from shallow features and estimating light intensity; S4. The feature map of region division and the feature map of illumination intensity estimation are weighted by the attention mechanism respectively, the two weighted feature maps are spliced, and the results are returned to the head end of the multiple sequentially connected organizational structures. Each organizational structure is jump-connected to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure is jump-connected to the second feature extraction layer. The processing process of each basic structure is first convolution operation, then normalization operation, and the normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer.

2. The static image defogging method according to claim 1, characterized in that: S1 includes: The image to be processed is subjected to a convolution operation with a convolution kernel size of 3×3, and then processed with the ReLU activation function to extract shallow features.

3. The static image defogging method according to claim 1, wherein S2 include: The shallow features are first input into the AOD-Net model, and the output results of the AOD-Net model are input into the Mobile-Net V2 model. The Mobile-Net V2 model implements semantic segmentation and distinguishes different areas of the image to be processed.

4. The static image defogging method according to claim 3, characterized in that: The expansion factor of the Mobile-Net V2 model was adjusted from 1 to 0.75, the last three bottleneck layers of the Mobile-Net V2 model were removed, the 3*3 depthwise convolution of the Mobile-Net V2 model was replaced with a 5*1 decomposed convolution plus a 1*5 decomposed convolution, and the FReLU activation function was used instead of the ReLU6 activation function of the Mobile-Net V2 model.

5. The static image defogging method according to claim 1, characterized in that: S3 includes: After convolution and pooling operations on the shallow features, they are input into the GeoCalib model to estimate the vertical direction and internal parameters of the image. Then, the vertical direction and internal parameters of the image are input into the StableDiffusion diffusion model to estimate the light intensity.

6. The static image defogging method according to claim 1, characterized in that: The attention mechanism is a sequentially connected spatial attention module and a pixel attention module.

7. The static image defogging method according to claim 1, characterized in that: The activation function sampled by the activation function layer is the ReLU activation function.

8. The static image defogging method according to claim 1, characterized in that: Also includes: S5. Construct a loss function and perform model training. When the loss function is minimized, stop training to obtain a trained model. Input the image to be processed collected in real time into the trained model to obtain a defogging image.

9. The static image defogging method according to claim 8, characterized in that: The construction process of the loss function is: The mean square error or absolute error between the model prediction value and the true value is used as the basic loss. The attention weight map output after feature fusion is multiplied by the basic loss to obtain the weighted loss of each pixel position. The weighted losses of all pixel positions are averaged to obtain the final loss.

10. A static image defogging system, characterized in that: include: A shallow feature extraction module, used to extract shallow features from the image to be processed; The region division module is used to extract high-level semantic features from shallow features to achieve semantic segmentation and complete region division; Light estimation module, used to further extract features from shallow features and estimate light intensity; The attention mechanism module is used to perform weighted processing on the feature map of region division and the feature map of illumination intensity estimation respectively using the attention mechanism, splice the two weighted feature maps, and return the results to the head end of multiple sequentially connected organizational structures. Each organizational structure is jump-connected to the attention mechanism. The feature map of region division, the feature map of illumination intensity estimation and the feature maps output by the multiple sequentially connected organizational structures are weighted by the attention mechanism and then feature fused. The feature maps after feature fusion are input into the first feature extraction layer and the final dehazed image is output. Each organizational structure includes a second feature extraction layer and multiple sequentially connected basic structures. Each basic structure is jump-connected to the second feature extraction layer. The processing process of each basic structure is first a convolution operation and then a normalization operation. The normalization operation result is sent to the activation function layer, and then the processing result of the basic structure is output through the pooling layer.

Images