An image dehazing method, apparatus, electronic device, and storage medium.

Abstract

This invention provides an image dehazing method, apparatus, electronic device, and storage medium. The method includes: acquiring a first image to be processed; using a multi-scale feature extraction module of a trained dehazing model to extract features at different scales of the first image, obtaining multiple first feature maps; using a color calibration module to perform color calibration on the first feature maps, obtaining a second feature map; using an adaptive fusion module to fuse the second feature maps, obtaining a third feature map; and using a reconstruction module to reconstruct the dehazed image based on the third feature map, obtaining a dehazed second image. The multi-scale feature extraction module captures image information at different scales, effectively handling haze of varying densities; the color calibration module solves the color distortion problem during the dehazing process, making the image colors more realistic and natural; the adaptive fusion module fuses different features to improve the dehazing effect; and the residual learning strategy reduces the learning difficulty of the network, improving training efficiency and dehazing effect.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image dehazing method and apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of image processing technology, haze can severely affect the quality of images or videos (composed of consecutive image frames) captured by various cameras, thus affecting subsequent image processing.

[0003] Taking traffic monitoring and intelligent transportation systems as examples, smog can severely affect image quality, leading to a decline in the performance of traffic monitoring systems and impacting traffic safety and management efficiency. For instance, in autonomous driving systems, smog can blur sensor images, affecting the accuracy of target detection and path planning. Furthermore, images captured by traffic monitoring cameras under smog conditions often lose crucial details, such as license plate numbers or traffic signs, posing a challenge to violation detection and accident analysis.

[0004] Traditional image dehazing methods are mainly based on physical models, such as dark channel priors and atmospheric scattering models. However, these methods often perform poorly in complex scenes, especially in traffic scenes. Due to the diversity of targets such as vehicles, roads, and traffic signs, traditional methods are difficult to achieve ideal results.

[0005] In recent years, deep learning methods have made significant progress in the field of image dehazing. However, existing deep learning dehazing methods have the following problems: (1) lack of effective use of multi-scale features, resulting in inconsistent performance under different haze densities; (2) serious color distortion problem, with significant differences between the colors of the dehazed image and the real scene; (3) complex network structure, high computational resource consumption, and difficulty in meeting the needs of real-time traffic monitoring. Summary of the Invention

[0006] In view of the above-mentioned defects in the prior art, the present invention provides an image dehazing method and apparatus, electronic device and storage medium to solve the technical problems of unreliable image dehazing and color distortion in the prior art.

[0007] To achieve the above and other related objectives, the present invention provides an image dehazing method, comprising: acquiring a first image to be processed; extracting features of the first image at different scales using a multi-scale feature extraction module of a trained dehazing model to obtain multiple first feature maps; performing color calibration on the first feature maps using a color calibration module of the trained dehazing model to obtain a second feature map; fusing the second feature maps using an adaptive fusion module of the trained dehazing model to obtain a third feature map; and reconstructing the dehazed image based on the third feature map using a reconstruction module of the trained dehazing model to obtain a dehazed second image.

[0008] In one embodiment of the present invention, the multi-scale feature extraction module includes a backbone network and a multi-scale feature pyramid network; the multi-scale feature extraction module, using a trained dehazing model, extracts features of the first image at different scales to obtain a first feature map, including: extracting basic features of the first image using the backbone network; and generating multiple first feature maps from the basic features using multiple parallel convolutional branches of the multi-scale feature pyramid network.

[0009] In one embodiment of the present invention, the backbone network includes a first convolutional layer, a max pooling layer and a plurality of residual block groups cascaded in sequence, each residual block group being composed of a different number of residual blocks, and each residual block containing two second convolutional layers.

[0010] In one embodiment of the present invention, multiple first feature maps are generated from the basic features using multiple parallel convolutional branches of the multi-scale feature pyramid network, including: generating a first feature map at a first scale from the basic features using a convolutional layer with a kernel size of 3×3; generating a first feature map at a second scale from the basic features using a convolutional layer with a kernel size of 5×5; generating a first feature map at a third scale from the basic features using a convolutional layer with a kernel size of 5×5; and generating a first feature map at a fourth scale from the basic features using global average pooling and a convolutional layer with a kernel size of 1×1.

[0011] In one embodiment of the present invention, the color calibration module includes a global color statistics unit and a color mapping unit; the color calibration module of the trained dehazing model performs color calibration on the first feature map, including: calculating the color statistical features of the first image using the global color statistics unit; and performing color calibration on the first feature map using the color mapping unit according to the mapping relationship between the color statistical features of the first image and the color statistical features of a preset dehazing image dataset.

[0012] In one embodiment of the present invention, the color calibration module further includes a fine-tuning unit; after the step of color calibration of the first feature map using the color mapping unit, the method further includes: fine-tuning the color-calibrated feature map using the fine-tuning unit.

[0013] In one embodiment of the present invention, the adaptive fusion module includes a channel attention unit, a spatial attention unit, a scale attention unit, and a feature fusion unit. The adaptive fusion module of the trained dehazing model fuses the second feature map to obtain a third feature map, including: calculating the channel attention weights of the second feature map using the channel attention unit, and multiplying the channel attention weights by the second feature map to obtain a first fused feature; calculating the spatial attention weights of the first fused feature using the spatial attention unit, and multiplying the spatial attention weights by the first fused feature to obtain a second fused feature; calculating the weights of each first feature map using the scale attention unit to obtain scale weights, and performing a weighted summation of each first feature map according to the scale weights to obtain a third fused feature; aligning the scales of the third fused feature with the second fused feature, and then using the feature fusion unit to perform feature fusion to obtain the third feature map.

[0014] To achieve the above and other related objectives, the present invention also provides an image dehazing apparatus, comprising: a data acquisition unit for acquiring a first image to be processed; a first processing unit for extracting features of the first image at different scales using a multi-scale feature extraction module of a trained dehazing model to obtain multiple first feature maps; a second processing unit for performing color calibration on the first feature maps using a color calibration module of the trained dehazing model to obtain a second feature map; a third processing unit for fusing the second feature maps using an adaptive fusion module of the trained dehazing model to obtain a third feature map; and a fourth processing unit for reconstructing the dehazed image based on the third feature map using a reconstruction module of the trained dehazing model to obtain a dehazed second image.

[0015] To achieve the above and other related objectives, the present invention also provides an electronic device, including a processor, a memory, and a communication bus; the communication bus is used to connect the processor and the memory; the processor is used to execute a computer program stored in the memory to implement the method provided in any of the above embodiments.

[0016] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to perform the method provided in any of the above embodiments.

[0017] The beneficial effects of this invention are as follows: This invention proposes an image dehazing method, apparatus, electronic device, and storage medium. The method captures image information at different scales through a multi-scale feature extraction module, effectively processing haze of different densities; solves the color distortion problem in the dehazing process through a color calibration module, making the image colors more realistic and natural; improves the dehazing effect by adaptively fusing different features through a triple attention mechanism in the adaptive fusion module; and reduces the learning difficulty of the network through a residual learning strategy, improving training efficiency and dehazing effect. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. The accompanying drawings are incorporated in and constitute a part of this specification, illustrating embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0019] Figure 1 A flowchart of an image dehazing method provided in an embodiment of the present invention;

[0020] Figure 2 The flowchart of the multi-scale feature extraction module provided in an embodiment of the present invention is shown.

[0021] Figure 3 This is an architecture diagram of a multi-scale feature extraction module provided in an embodiment of the present invention;

[0022] Figure 4 This is a flowchart of the color calibration module provided in an embodiment of the present invention;

[0023] Figure 5 This is an architectural diagram of a color calibration module provided in an embodiment of the present invention;

[0024] Figure 6 This is a flowchart of the adaptive fusion module provided in an embodiment of the present invention;

[0025] Figure 7 This is an architecture diagram of an adaptive fusion module provided in an embodiment of the present invention;

[0026] Figure 8 This is an architecture diagram of a reconstruction module provided in an embodiment of the present invention;

[0027] Figure 9 This is a schematic diagram of an image dehazing device provided in an embodiment of the present invention;

[0028] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0029] Explanation of reference numerals in the attached drawings: 901, data acquisition unit; 902, first processing unit; 903, second processing unit; 904, third processing unit; 905, fourth processing unit; 1001, processor; 1002, memory. Detailed Implementation

[0030] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. It should be noted that, unless otherwise specified, the following embodiments and features can be combined with each other. In addition to the specific methods, equipment, and materials used in the embodiments, based on the knowledge of the prior art and the description of the present invention by those skilled in the art, any prior art methods, equipment, and materials similar to or equivalent to the methods, equipment, and materials in the embodiments of the present invention can be used to implement the present invention.

[0031] It should be understood that the terminology used in the embodiments of this invention is for describing specific implementations and not for limiting the scope of protection of this invention. Unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art.

[0032] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In some embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0033] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functions, and operations that may be implemented in the methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0034] Please see Figure 1 , Figure 1 An image dehazing method provided in an embodiment of the present invention includes steps S101 to S105.

[0035] Step S101: Obtain the first image to be processed. The first image is an unclear image containing haze; it can be a color image in RGB format, a grayscale image, or an image in other formats. Taking a traffic scene as an example, the original image may come from a traffic monitoring camera, a vehicle-mounted camera, or other traffic scene image acquisition equipment. After preprocessing (e.g., format conversion, size adjustment), the first image is obtained. Understandably, the size of the first image must meet the input size requirements of the subsequent dehazing model.

[0036] Step S102: Using the multi-scale feature extraction module of the trained dehazing model, extract features at different scales of the first image to obtain multiple first feature maps.

[0037] Please see Figure 2 In a specific embodiment of the present invention, the multi-scale feature extraction module includes a backbone network and a multi-scale feature pyramid network; step S102 includes steps S201 and S202, which can be combined. Figure 3 Use the architecture diagram in the image to understand the following steps.

[0038] Step S201: Extract the basic features of the first image using the backbone network.

[0039] In one specific embodiment of the present invention, the backbone network includes a first convolutional layer, a max-pooling layer, and several residual block groups cascaded sequentially. Each residual block group consists of a different number of residual blocks, and each residual block contains two second convolutional layers. The backbone network adopts an improved ResNet-50 architecture, which performs excellently in image classification and feature extraction tasks. ResNet-50 solves the gradient vanishing problem in deep network training through residual connections, enabling the network to learn deeper feature representations. In this invention, we fine-tuned ResNet-50 to adapt to the task of traffic image dehazing. Specifically, we retained the first four residual blocks of ResNet-50 for extracting basic image features. These features contain low-level information of the image, such as edges and textures, which are crucial for subsequent dehazing processing.

[0040] In a specific embodiment of the present invention, taking the first image in RGB format as an example, its size can be recorded as H×W×3. (1) The first convolutional layer uses a 7×7 convolutional kernel with a stride of 2. The size of the feature map after processing by the first convolutional layer is H / 2×W / 2×64. (2) The pooling kernel size of the max pooling layer is 3×3 with a stride of 2. The size of the feature map after processing by the max pooling layer is H / 4×W / 4×64. (3) The first residual block group consists of 3 residual blocks. The size of the feature map after processing by the first residual block group is H / 4×W / 4×64. (4) The second residual block group consists of 4 residual blocks. The size of the feature map after processing by the second residual block group is H / 8×W / 8×128. (5) The third residual block group consists of 6 residual blocks. The size of the feature map after processing by the third residual block group is H / 16×W / 16×256. (4) The third residual block group consists of six residual blocks. The size of the feature map after processing by the third residual block group is H / 16×W / 16×256. (5) The fourth residual block group consists of three residual blocks. The size of the feature map after processing by the fourth residual block group is H / 32×W / 32×512.

[0041] Step S202: Multiple first feature maps are generated from the basic features using multiple parallel convolutional branches of the multi-scale feature pyramid network (FPN). After the backbone network extracts the basic features, the multi-scale feature pyramid network (FPN) is used to generate feature maps at different scales. FPN is a top-down architecture that generates multi-scale feature representations by combining high-level semantic information with low-level detail information. In this invention, FPN includes multiple parallel convolutional branches, each using convolutional kernels of different sizes for feature extraction.

[0042] In a specific embodiment of the present invention, step S202 includes: (1) using a convolutional layer with a kernel size of 3×3 to generate a first feature map at a first scale from the basic features, which can be denoted as F1; (2) using a convolutional layer with a kernel size of 5×5 to generate a first feature map at a second scale from the basic features, which can be denoted as F2; ​​(3) using a convolutional layer with a kernel size of 7×7 to generate a first feature map at a third scale from the basic features, which can be denoted as F3; (4) using global average pooling and a convolutional layer with a kernel size of 1×1 to generate a first feature map at a fourth scale from the basic features, which can be denoted as F4. The dimensions of the above features F1~F4 are all H / 32×W / 32×128. In order to facilitate subsequent processing, these four features can be spliced ​​together in the channel dimension to form a comprehensive feature representation, which can be expressed by the formula: F multi =Concat(F1,F2,F3,F4), concatenates the features F1,F2,F3,F4, and F4. multiThe dimensions are H / 32×W / 32×512. The stitched composite feature map contains multi-scale information from fine-grained to coarse-grained, providing rich input for subsequent color calibration and feature fusion.

[0043] In the above embodiments, the convolutions in each branch can be ordinary convolutions. To further expand the receptive field, in a specific embodiment of the present invention, dilated convolutions can also be introduced in larger convolution kernel branches (e.g., the third branch and the second branch) to capture a wider range of contextual information without increasing the number of parameters.

[0044] Furthermore, to improve computational efficiency, techniques such as group convolution and depthwise separable convolution can be used in FPN to reduce the number of parameters and computational complexity.

[0045] Step S103: Use the color calibration module of the trained dehazing model to perform color calibration on the first feature map to obtain the second feature map.

[0046] Please see Figure 4 In a specific embodiment of the present invention, the color calibration module includes a global color statistics unit and a color mapping unit. The color calibration module, using a trained dehazing model, performs color calibration on the first feature map, including steps S401 and S402, which can be combined with... Figure 5 Use the architecture diagram in the image to understand the following steps.

[0047] Step S401: Calculate the color statistical features of the first image using the global color statistics unit. Color statistical features may include, for example, the mean, standard deviation, and histogram distribution of each channel. Taking an RGB image as the first image and the mean and standard deviation as the color statistical features, the mean and standard deviation of the three channels can be calculated, resulting in the following six color statistical features: μ fog-r μ fog-g μ fog-b σ fog-r σ fog-g σ fog-b Where μ represents the mean, σ represents the standard deviation, the subscript fog represents the first image, and the subscripts r, g, and b correspond to different channels. The calculation formula is as follows:

[0048] ,

[0049] ,

[0050] In the formula, H represents the pixel value of channel c (c∈{r,g,b}) at coordinate (i,j) in the first image, and H and W represent the height and width of the first image.

[0051] Step S402: Based on the mapping relationship between the color statistical features of the first image and the color statistical features of the preset haze-free image dataset, the color mapping unit is used to perform color calibration on the first feature map.

[0052] In this step, the color statistical features of the haze-free image dataset can be calculated in the same way as in step S301. It should be noted that the haze-free image dataset contains many haze-free images; therefore, the calculation uses the pixel mean and standard deviation of all these images. After calculation, six color statistical features can be obtained: μ ref-r μ ref-g μ ref-b σ ref-r σ ref-g σ ref-b .

[0053] Understandably, these feature values ​​can be calculated and stored as fixed parameters for direct use later; alternatively, a separate reference color generation unit can be set up to calculate the aforementioned color statistical features based on the input hazy image dataset.

[0054] A mapping relationship can be constructed using the six color statistical features of the first image and the color statistical features of a preset haze-free image dataset. Taking the F1~F4 features in the above specific embodiment as an example, since color calibration targets multi-scale features, this step can directly adjust the stitched feature F1~F4. multi Perform calibration.

[0055] Considering F multi The dimensions are H / 32×W / 32×512. Therefore, the six color statistical features of the first image and the six color statistical features of the preset haze-free image dataset can be projected onto a 512-dimensional space. After projection, the six color statistical features of the first image can be obtained as μ. fog-1 ~μ fog-512 σ fog-1 ~σ fog-512 Such features; similarly, the color statistical features of a pre-defined haze-free image dataset, after projection, can yield μ. ref-1 ~μ ref-512 σ ref-1 ~σ ref-512 Such characteristics.

[0056] Then, we can use the projected 512-dimensional vector to perform an affine transformation, as shown in the following formula:

[0057] ,

[0058] In the formula, i∈{1,2,…,512}, F calibrated-i For the calibrated feature F calibrated The i-th channel, F multi-i For the spliced ​​feature F multi The i-th channel.

[0059] Understandably, the above color statistical features are calculated by distinguishing different channels. In practical applications, if the first image is a grayscale image, then channel-specific calculations are unnecessary; a mean and a standard deviation can be directly calculated from the first image. The preset haze-free image dataset also has only one mean and one standard deviation in its color statistical features. Thus, the calibrated feature F... calibrated Each channel is corrected with the same parameters, eliminating the need to project color statistics features into a 512-dimensional vector.

[0060] Please see Figure 4 In a specific embodiment of the present invention, the color calibration module further includes a fine-tuning unit. Following step 402, the method further includes step S403: fine-tuning the color-calibrated feature map using the fine-tuning unit.

[0061] In this embodiment, to further improve the accuracy of color calibration, step S303 is introduced, which can be expressed by the formula:

[0062] ,

[0063] In the formula, F final The final feature map after calibration is composed of the stitched feature F. multi The color calibration feature maps are obtained through calibration. Of course, the multi-scale features F1 to F4 can also be calibrated separately to obtain their respective color calibration feature maps. α and β are parameters learned through the network.

[0064] Step S104: Using the adaptive fusion module of the trained dehazing model, the second feature map is fused to obtain the third feature map.

[0065] Please see Figure 6 In a specific embodiment of the present invention, the adaptive fusion module includes a channel attention unit, a spatial attention unit, a scale attention unit, and a feature fusion unit. Step S104 specifically includes steps S601 to S604, which will be described in detail below, and can be combined with... Figure 7 Use the architecture diagram in the image to understand the following steps.

[0066] Step S601: Calculate the channel attention weights of the second feature map using the channel attention unit, and multiply the channel attention weights by the second feature map to obtain the first fused feature.

[0067] In a specific embodiment of the present invention, step S601 includes: (1) first processing the input feature map F (e.g., the feature F mentioned above). final (1) Perform global average pooling and global max pooling operations to obtain two channel descriptors; (2) Pass the two channel descriptors through a shared multilayer perceptron (MLP) to obtain two channel attention maps; (3) Add the two channel attention maps and activate them through the Sigmoid activation function to obtain the final channel attention weight M. c (4) Finally, the channel attention weights M c Multiplying the input feature map F by the first fused feature map yields the first fused feature: .

[0068] Step S602: Calculate the spatial attention weight of the first fusion feature using the spatial attention unit, and multiply the spatial attention weight by the first fusion feature to obtain the second fusion feature.

[0069] In a specific embodiment of the present invention, step S602 includes: (1) processing the feature map F after channel attention processing. c (2) Perform average pooling and max pooling along the channel dimension to obtain two spatial feature maps; (3) Concatenate the two spatial feature maps along the channel dimension and obtain the spatial attention weight M through a 7×7 convolutional layer and a sigmoid activation function. s (3) Integrate spatial attention weights with the first fusion feature F c Multiplying, we obtain the second fusion feature: .

[0070] Step S603: Calculate the weight of each first feature map using the scale attention unit to obtain the scale weights, and perform a weighted summation on each first feature map according to the scale weights to obtain the third fused feature.

[0071] In a specific embodiment of the present invention, step S603 includes: (1) for each first feature map { } Calculate the scale descriptors separately. In the above specific embodiment, there are four branches, i.e., n=4; (2) Process all the scale descriptors through a fully connected layer and a Softmax function to obtain the weights of each scale { };(3)Weighted summation of feature maps at each scale to obtain the third fusion feature: Taking the aforementioned four branches as an example, the third fusion feature F scale The dimensions are H / 32×W / 32×128.

[0072] Step S604: After aligning the scales of the third fusion feature with those of the second fusion feature, feature fusion is performed using the feature fusion unit to obtain the third feature map.

[0073] Since steps S601 and S602 are for the spliced ​​feature F after color calibration, final The process can be performed (or, without stitching, by calculating color calibration, channel attention, and spatial attention for each feature at each scale separately), resulting in a second fused feature size of H / 32×W / 32×512. To achieve the fusion of the second and third fused features, the third fused feature first needs to be aligned to 512 dimensions. In this embodiment, the third fused feature can be directly expanded three times along the channel dimension, or it can be adjusted to 512 dimensions through a convolutional layer.

[0074] The fusion process of the feature fusion unit can be expressed by the following formula:

[0075] ,

[0076] In the formula, For the third fusion feature after alignment, F fused λ is the third feature map that is finally output by the adaptive fusion module, and λ is a learnable fusion parameter.

[0077] Please see Figure 8 Step S105: Using the reconstruction module of the trained dehazing model, reconstruct the dehazed image based on the third feature map to obtain the second dehazed image.

[0078] In one specific embodiment of the present invention, the reconstruction module includes an upsampling unit, a detail enhancement unit, and a residual learning unit. The upsampling unit upsamples the third feature map to the original image resolution, the detail enhancement unit enhances image details and edge information, and the residual learning unit learns the residual between the foggy image and the fog-free image, reducing the learning difficulty.

[0079] In a specific embodiment of the present invention, the upsampling unit employs transposed convolution (deconvolution) to progressively upsample the fused feature map to the original image resolution. The detail enhancement unit enhances the image's details and edge information through edge detection and high-frequency information extraction. The residual learning unit predicts the difference between the foggy image and the fog-free image; that is, the dehazing model predicts the residual between the foggy and fog-free images. This residual is then added to the first foggy image to obtain the final fog-free image: I clear =I hazy +Residual.

[0080] The above description of the image processing procedure implicitly expresses the specific structure of the dehazing model. Understandably, after the dehazing model is built, it needs to be trained in order to achieve dehazing processing of the first image. The training process is briefly introduced below.

[0081] (1) Data preparation: Collect a dataset containing paired foggy traffic images and corresponding fog-free images (real labels). Alternatively, a synthetic haze dataset can be used, and a haze effect can be added to clear images using a physical model. This invention uses paired foggy and fog-free traffic image datasets for training. Synthetic haze data is generated using an atmospheric scattering model.

[0082] (2) Loss function design: The loss function includes reconstruction loss, perceptual loss and color consistency loss, and its expression is as follows:

[0083] ,

[0084] In the formula, , , It is the weighting coefficient; L recon For reconstruction loss, it uses L1 loss to calculate the difference between the predicted fog-free image and the true fog-free image; L percep For perceptual loss, it uses a pre-trained VGG network to extract features and calculates the difference between the predicted image and the real image in the feature space; L color As a loss of color consistency, it ensures that the color distribution of the dehazed image is consistent with the reference hazy image.

[0085] (3) Optimization strategy: Use the Adam optimizer and set the initial learning rate to 0.0002; adopt the learning rate decay strategy, multiply the learning rate by 0.5 every 50 epochs; set the batch size to 16 and train for 300 epochs; use weight decay of 0.0001 to prevent overfitting.

[0086] (4) Evaluation indicators:

[0087] Peak signal-to-noise ratio (PSNR): Used to evaluate the quality of reconstructed images;

[0088] Structural Similarity (SSIM): Evaluates the degree of structural similarity between a reconstructed image and a real image;

[0089] Perceived quality: Using the no-reference image quality assessment metrics NIQE and BRISQUE;

[0090] Computational efficiency: Evaluate the number of model parameters, FLOPs, and inference time.

[0091] To verify the effectiveness of the dehazing model of this invention, it was compared with two other dehazing models on a standard dataset. The comparison results are shown in the table below.

[0092] Table 1: Comparison of multiple dehazing models

[0093]

[0094] As shown in the table above, this method improves PSNR by approximately 2.5 dB, SSIM by approximately 0.03, and inference speed by more than 30%, demonstrating excellent dehazing performance.

[0095] It should be noted that the steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they contain the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0096] Please see Figure 9 , Figure 9 An image dehazing apparatus according to an embodiment of the present invention includes a data acquisition unit 901, a first processing unit 902, a second processing unit 903, a third processing unit 904, and a fourth processing unit 905. The data acquisition unit 901 acquires a first image to be processed; the first processing unit 902 uses a multi-scale feature extraction module of a trained dehazing model to extract features of the first image at different scales, obtaining multiple first feature maps; the second processing unit 903 uses a color calibration module of the trained dehazing model to perform color calibration on the first feature maps, obtaining second feature maps; the third processing unit 904 uses an adaptive fusion module of the trained dehazing model to fuse the second feature maps, obtaining a third feature map; and the fourth processing unit 905 uses a reconstruction module of the trained dehazing model to reconstruct the dehazed image based on the third feature maps, obtaining a dehazed second image.

[0097] It should be noted that the image dehazing device of this embodiment is a device corresponding to the image dehazing method described above, and the functional modules in the image dehazing device may correspond to the corresponding steps in the image dehazing method. The image dehazing device of this embodiment can be implemented in conjunction with the image dehazing method; that is, where there is no conflict, the relevant technical details mentioned in the image dehazing method of the above embodiments can also be applied to the image dehazing device of this embodiment.

[0098] Please see Figure 10 , Figure 10An electronic device provided in one embodiment of the present invention includes a processor 1001, a memory 1002 and a communication bus; the communication bus is used to connect the processor 1001 and the memory 1002; the processor 1001 is used to execute a computer program stored in the memory 1002 to implement the above-mentioned image dehazing method.

[0099] The aforementioned electronic device is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0100] The aforementioned electronic devices can be any electronic product capable of human-computer interaction with users, such as personal computers, tablets, smartphones, personal digital assistants (PDAs), game consoles, interactive network television (IPTV), smart wearable devices, etc.

[0101] The aforementioned electronic devices may also include network devices and / or user devices. Among them, network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0102] The networks in which the aforementioned electronic devices are located include, but are not limited to, the Internet, wide area networks, metropolitan area networks, local area networks, and virtual private networks (VPNs).

[0103] The processors mentioned above can be, for example, general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components; the memory mentioned above may include random access memory (RAM), and may also include non-volatile memory, such as at least one disk storage device.

[0104] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, the computer program being used to cause a computer to perform the above-described image dehazing method.

[0105] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. An image defogging method, characterized by, The method comprises the following steps: acquiring a first image to be processed; extracting features of the first image at different scales by using a multi-scale feature extraction module of a trained defogging model to obtain a plurality of first feature maps; performing color calibration on the first feature maps by using a color calibration module of the trained defogging model to obtain second feature maps; performing fusion on the second feature maps by using an adaptive fusion module of the trained defogging model to obtain third feature maps; reconstructing a defogging image based on the third feature maps by using a reconstruction module of the trained defogging model to obtain a second image after defogging; the adaptive fusion module comprises a channel attention unit, a spatial attention unit, a scale attention unit and a feature fusion unit; performing fusion on the second feature maps by using an adaptive fusion module of the trained defogging model to obtain third feature maps, comprising: calculating channel attention weights of the second feature maps by using the channel attention unit, and multiplying the channel attention weights and the second feature maps to obtain first fusion features; calculating spatial attention weights of the first fusion features by using the spatial attention unit, and multiplying the spatial attention weights and the first fusion features to obtain second fusion features; calculating weights of each of the first feature maps by using a scale attention unit to obtain scale weights, and performing weighted summation on each of the first feature maps according to the scale weights to obtain third fusion features; aligning the third fusion features and the second fusion features in scale, and then performing feature fusion by using a feature fusion unit to obtain the third feature maps.

2. The image defogging method of claim 1, wherein, The multi-scale feature extraction module comprises a backbone network and a multi-scale feature pyramid network; extracting features of the first image at different scales by using a multi-scale feature extraction module of a trained defogging model to obtain first feature maps, comprising: extracting basic features of the first image by using the backbone network; generating a plurality of first feature maps from the basic features by using a plurality of parallel convolution branches of the multi-scale feature pyramid network.

3. The image defogging method of claim 2, wherein, The backbone network comprises a first convolution layer, a maximum pooling layer and a plurality of residual block groups connected in sequence, each of the residual block groups is composed of different numbers of residual blocks, and each of the residual blocks comprises two second convolution layers.

4. The image defogging method of claim 2, wherein, Generating a plurality of first feature maps from the basic features by using a plurality of parallel convolution branches of the multi-scale feature pyramid network, comprising: generating a first feature map at a first scale from the basic features by using a convolution layer with a convolution kernel size of 3x3; generating a first feature map at a second scale from the basic features by using a convolution layer with a convolution kernel size of 5x5; generating a first feature map at a third scale from the basic features by using a convolution layer with a convolution kernel size of 5x5; generating a first feature map at a fourth scale from the basic features by using a global average pooling and a convolution layer with a convolution kernel size of 1x1.

5. The image defogging method of claim 1, wherein, The color calibration module comprises a global color statistics unit and a color mapping unit; The color calibration module using the trained defogging model is used for color calibration of the first feature map, including: The color statistics feature of the first image is calculated by using the global color statistics unit; According to the mapping relationship between the color statistics feature of the first image and the color statistics feature of the preset fog-free image data set, the color mapping unit is used for color calibration of the first feature map.

6. The image defogging method of claim 5, wherein, The color calibration module further includes a fine-tuning unit; After the step of using the color mapping unit to calibrate the color of the first feature map, it further includes: The fine-tuning unit is used to fine-tune the feature map after color calibration.

7. An image defogging apparatus characterized by comprising: Including: The data acquisition unit is used to acquire the first image to be processed; The first processing unit is used to extract the features of the first image at different scales by using the multi-scale feature extraction module of the trained defogging model, to obtain a plurality of first feature maps; The second processing unit is used to calibrate the color of the first feature map by using the color calibration module of the trained defogging model, to obtain a second feature map; The third processing unit is used to fuse the second feature map by using the adaptive fusion module of the trained defogging model, to obtain a third feature map; And The fourth processing unit is used to reconstruct the defogging image based on the third feature map by using the reconstruction module of the trained defogging model, to obtain the second image after defogging; The adaptive fusion module includes a channel attention unit, a spatial attention unit, a scale attention unit and a feature fusion unit; The adaptive fusion module of the trained defogging model is used to fuse the second feature map to obtain a third feature map, including: The channel attention weight of the second feature map is calculated by using the channel attention unit, and the channel attention weight and the second feature map are multiplied to obtain a first fusion feature; The spatial attention weight of the first fusion feature is calculated by using the spatial attention unit, and the spatial attention weight and the first fusion feature are multiplied to obtain a second fusion feature; The weight of each first feature map is calculated by using the scale attention unit to obtain the scale weight, and each first feature map is weighted and summed according to the scale weight to obtain a third fusion feature; After the third fusion feature and the second fusion feature are aligned in scale, the feature fusion unit is used for feature fusion to obtain the third feature map.

8. An electronic device, comprising: The processor, the memory and the communication bus are included; the communication bus is used to connect the processor and the memory; the processor is used to execute the computer program stored in the memory to realize the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, A computer program is stored thereon, which is used to make a computer execute the method in any one of claims 1-6.

Images