An image defogging method, program product, device and storage medium

Abstract

Embodiments of the present application disclose an image defogging method, a program product, an equipment and a storage medium, comprising: acquiring a to-be-processed image, inputting the to-be-processed image into a pre-determined target image defogging model; performing depth separable convolution operation on the to-be-processed image layer by layer through a multi-scale encoding-decoding network of the target image defogging model to obtain a deep feature map; performing instance normalization operation on the to-be-processed image through a light-weight convolution series network of the target image defogging model to obtain an estimated transmittance map; determining a preliminary defogging image based on the to-be-processed image, the estimated transmittance map and a pre-determined atmospheric light parameter; generating compensation information based on the deep feature map and the preliminary defogging image, and obtaining a target defogging image according to the compensation information and the preliminary defogging image. The present application can output a high-quality defogging image through a light-weight defogging model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image dehazing method, program product, device and storage medium. Background Technology

[0002] Image dehazing, a fundamental task in the underlying field of computer vision, plays a crucial role in improving the quality of outdoor images. In hazy weather, the scattering and absorption of light by suspended particles in the atmosphere leads to reduced image contrast, color shift, and loss of detail. This degradation not only impairs the subjective visual quality of images but also severely restricts the performance and robustness of advanced vision systems such as autonomous driving, video surveillance, and remote sensing. Therefore, developing dehazing algorithms that are both efficient and robust has significant academic and practical value.

[0003] In recent years, deep learning-based image dehazing methods have become mainstream, but existing methods exhibit a significant polarization in model design: The first category consists of heavyweight deep neural networks that pursue ultimate restoration quality, typically employing complex deep residual stacking or attention mechanisms. While these methods achieve excellent dehazing results, their millions of parameters and enormous computational overhead lead to excessively high inference latency. The large model parameters place extremely high demands on hardware bandwidth and storage space, making it difficult to achieve real-time operation on resource-constrained edge computing devices, such as mobile robots and outdoor vehicle-mounted chips. The second category comprises lightweight networks aimed at improving inference efficiency. These methods typically employ simplified physical assumptions or end-to-end direct mapping strategies. Although they are easy to deploy and infer quickly, their extremely low model capacity often results in incomplete dehazing and loss of high-frequency details due to insufficient feature extraction capabilities. Existing methods either have excessively large models that are difficult to run in real-time, or small models with poor dehazing results. Therefore, current image dehazing methods struggle to meet the dual requirements of low computational efficiency and high restoration quality. Summary of the Invention

[0004] This invention provides an image dehazing method, program product, device, and storage medium that can output high-quality dehazed images through a lightweight dehazing model.

[0005] In a first aspect, embodiments of the present invention provide an image dehazing method, comprising: Obtain the image to be processed and input the image to be processed into a pre-determined target image dehazing model; The target image dehazing model uses a multi-scale encoder-decoder network to perform depth-separable convolution operations on the image to be processed layer by layer to obtain a deep feature map. The image to be processed is subjected to instance normalization operation through the lightweight convolutional concatenation network of the target image dehazing model to obtain an estimated transmittance map. A preliminary dehazing image is determined based on the image to be processed, the estimated transmittance map, and the predetermined atmospheric light parameters; Compensation information is generated based on the deep feature map and the preliminary dehazing image, and the target dehazing image is obtained based on the compensation information and the preliminary dehazing image.

[0006] In a second aspect, embodiments of the present invention provide an image dehazing device, the device comprising: The data acquisition module is used to acquire the image to be processed and input the image to be processed into a pre-determined target image dehazing model; the target image dehazing model includes a feature extraction module, a transmittance estimation module, and an image correction module. The feature extraction module is used to perform depth-separable convolution operations on the image to be processed layer by layer through the multi-scale encoding and decoding network of the feature extraction module to obtain a deep feature map; The transmittance estimation module is used to perform instance normalization operation on the image to be processed through the lightweight convolutional concatenation network of the transmittance estimation module to obtain an estimated transmittance map. The preliminary dehazing module is used to determine a preliminary dehazing image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters; The image correction module is used to generate compensation information based on the deep feature map and the preliminary dehazing image, and to obtain the target dehazing image based on the compensation information and the preliminary dehazing image.

[0007] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image dehazing method as described in any of the embodiments of the present invention.

[0008] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image dehazing method as described in any of the embodiments of the present invention.

[0009] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed by a processor, implements an image dehazing method as described in any of the embodiments of the present invention.

[0010] In this embodiment of the invention, an image to be processed is acquired and input into a pre-determined target image dehazing model. Through the multi-scale encoder-decoder network of the target image dehazing model, depth-separable convolution operations are performed layer by layer on the image to be processed to obtain a deep feature map. Through a lightweight convolutional concatenation network of the target image dehazing model, instance normalization operations are performed on the image to be processed to obtain an estimated transmittance map. A preliminary dehazing image is determined based on the image to be processed, the estimated transmittance map, and pre-determined atmospheric light parameters. Compensation information is generated based on the deep feature map and the preliminary dehazing image, and the target dehazing image is obtained based on the compensation information and the preliminary dehazing image. The method of this embodiment of the invention, by analyzing the image to be processed in parallel, decouples the physical model from the depth features, synchronously obtains the deep feature map and the estimated transmittance map, and reduces the number of model parameters from quadratic to linear with the increase of the number of channels through depth-separable convolution, thus achieving model lightweighting. By independently estimating transmittance to ensure the ability to perceive physical parameters, and by fusing deep feature maps and preliminary dehazing images to obtain compensation information and correct the preliminary dehazing images, the difference between the physical coarse reconstruction results and the real clear images can be captured, thereby improving the quality of the dehazing images. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of an image dehazing method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of a multi-scale encoding and decoding network structure provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of a lightweight convolutional cascaded network structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a preliminary dehazed image provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the image dehazing device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0013] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all structures. The acquisition, storage, use, and processing of data in the technical solutions of this application comply with relevant national laws and regulations. It should be noted that, in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used the relevant content of such solutions.

[0014] Figure 1 This is a flowchart illustrating an image dehazing method provided in an embodiment of the present invention. The method of this embodiment can output high-quality dehazed images using a lightweight dehazing model. This method can be executed by an image dehazing device provided in this embodiment, which can be implemented in software and / or hardware. The following embodiments will illustrate this using the example of the device being integrated into an electronic device. The electronic device can be a server or computer device, etc., used to implement an image dehazing method. (Refer to...) Figure 1 The method may specifically include the following steps:

[0015] Step 101: Obtain the image to be processed and input it into the predetermined target image dehazing model.

[0016] The target image dehazing model is a pre-trained model capable of outputting high-quality dehazed images. The image to be processed is the image that needs to be dehazed. The target image dehazing model includes a feature extraction unit, a transmittance estimation unit, a preliminary dehazing unit, and an image correction unit. The feature extraction unit adopts a multi-scale encoder-decoder structure, with depthwise separable convolution as the core operator; all convolution operations within the network are depthwise separable convolutions. The transmittance estimation unit receives the image to be processed in parallel and synchronously with the feature extraction unit. The transmittance estimation unit includes a lightweight convolutional concatenation network, which consists of multiple cascaded lightweight convolutional layers, instance normalization layers, and activation functions connected in series. The transmittance estimation unit is used to generate a single-channel physical transmittance map. The preliminary dehazing unit performs the inverse physical derivation process of the atmospheric scattering model. The image correction unit is responsible for multi-source feature fusion and residual learning compensation, used to compensate for residual fog, detail blur, and color shift in the preliminary dehazed image and perform detail compensation and color correction.

[0017] Step 102: Using the multi-scale encoding and decoding network of the target image dehazing model, perform depthwise separable convolution operations layer by layer on the image to be processed to obtain deep feature maps.

[0018] The multi-scale encoder-decoder network comprises an encoder and a decoder, both using depthwise separable convolution as their core operator. The encoder downsamples the image to be processed at each level, while the decoder upsamples it to recover the spatial dimensions of the features. All convolution operations within the encoder-decoder network are depthwise separable convolutions, which decompose standard convolutions into depthwise convolutional layers and pointwise convolutional layers. By decoupling spatial feature extraction from channel feature fusion, the number of model parameters is compressed. Finally, a 1 / 1 convolutional layer is introduced at the decoder end to adjust channel dimensions and integrate multi-scale features, outputting a deep feature map. Optionally, in this scheme, depthwise separable convolution operations are performed layer by layer on the image to be processed to obtain the deep feature map. This includes: using the downsampling layer of the encoder in the multi-scale encoder-decoder network to perform multi-scale depthwise convolution and pointwise convolution on the image to be processed, obtaining encoder features; using the upsampling layer of the decoder in the multi-scale encoder-decoder network to perform depthwise convolution and pointwise convolution on the encoder features, obtaining decoder features; and obtaining the deep feature map based on the decoder features.

[0019] Specifically, the encoder includes a first downsampling layer, a second downsampling layer, and a third downsampling layer. The encoder features include the features output by each downsampling layer. The image to be processed is input to the feature extraction unit in the form of three-channel features (e.g., red, yellow, and blue channels), and then deep convolution is performed on it through a feature embedding layer. For example, three 3x3 convolution kernels are used, each corresponding to only one input channel. Spatial sliding convolution is performed independently on each of the three channels, outputting intermediate feature maps for the three channels. This process only extracts spatial features and does not involve inter-channel interaction. The output features are further convolved pointwise. For example, multiple 1x1 convolution kernels are used to linearly combine the above three-channel intermediate features, increasing the channel dimensionality to the initial number of feature channels, realizing information fusion between channels, and obtaining full-resolution features. The full-resolution features are then deep convolved and pointwise convolved through the second downsampling layer to obtain 1 / 2 resolution features. The 1 / 2 resolution features are then deep convolved and pointwise convolved through the third downsampling layer to obtain 1 / 4 resolution features.

[0020] In the encoder-decoder network, the encoder retains the feature map of each downsampling layer (e.g., full-resolution features, 1 / 2-resolution features, and 1 / 4-resolution features); when the decoder upsamples, it directly passes this feature map to the corresponding layer. The decoder in this scheme includes a first upsampling layer, a second upsampling layer, and a third upsampling layer. The first upsampling layer of the decoder performs depthwise convolution and pointwise convolution on the features output from the third downsampling layer. For example, the same number of 3x3 convolution kernels as the number of input channels are used, with each kernel processing one channel independently to extract spatial features; linear combination between channels is achieved through 1x1 convolution kernels to realize feature fusion and dimensionality adjustment. After instance normalization and ReLU activation, the spatial resolution is magnified by a factor of two through bilinear interpolation. The 1 / 2-resolution features passed from the second downsampling layer of the encoder via skip connections are concatenated and fused with the magnified features along the channel dimension to form a hybrid feature tensor. Depthwise convolution and pointwise convolution operations are then performed again on this hybrid feature. The full-resolution features from the encoder's first downsampling layer, passed through skip connections, are concatenated with the amplified features along the channel dimension. After depthwise convolution and pointwise convolution, the original full resolution is restored using bilinear interpolation in the third upsampling layer. Finally, a 1x1 convolutional layer is introduced at the end of the decoder to compress the multi-scale fused features into a unified 16-channel array, resulting in a deep feature map.

[0021] For example, Figure 2 This is a schematic diagram of a multi-scale encoding and decoding network structure provided in an embodiment of the present invention. Figure 2 As shown, the foggy image (the image to be processed) is input into a multi-scale encoder-decoder network, and initial feature embedding is performed through a DWConv (depthseparable convolution) layer. Then, it sequentially passes through two DWConv layers in the encoder for progressive downsampling, extracting multi-scale deep features. The deepest layer in the encoding stage is connected to the decoding stage via skip connections, while intermediate layers also pass shallow features to the corresponding layers in the decoding stage via skip connections. The upsampling path in the decoding stage also consists of two DWConv layers, receiving shallow features from skip connections and fusing them with the upsampled deep features to progressively restore spatial resolution. Finally, after processing through a DWConv layer, a 1-to-1 convolutional layer adjusts the channel dimensions, outputting a deep feature map.

[0022] Step 103: Perform instance normalization operation on the image to be processed through the lightweight convolutional concatenation network of the target image dehazing model to obtain the estimated transmittance map.

[0023] The lightweight convolutional concatenation network consists of multiple cascaded lightweight convolutional layers, instance normalization layers, and an output layer. The output layer incorporates a Sigmoid activation function as a physical constraint, forcibly limiting the network output to a physical range, ensuring the physical validity of the transmittance map and preventing numerical out-of-bounds errors that could compromise the interpretability of the physical model. The lightweight convolutional concatenation network performs instance normalization on the image to be processed, enabling the network to independently perform statistical normalization for each image and adapt to different lighting distributions under varying fog concentrations. In this scheme, the lightweight convolutional concatenation network of the target image dehazing model performs instance normalization on the image to be processed to obtain an estimated transmittance map. This includes: convolution and instance normalization of the image to be processed based on lightweight convolutional layers to obtain fog concentration-related features; and activation processing of these fog concentration-related features using the physical constraint activation function of the output layer to obtain the estimated transmittance map.

[0024] The lightweight convolutional cascaded network comprises two lightweight convolutional layers and an output layer. The first lightweight convolutional layer is used to upscale the features of the image to be processed. For example, a 3x3 convolutional kernel maps the input three-channel foggy image to a sixteen-channel feature space, completing the initial feature embedding. Instance normalization is then performed on these sixteen-channel features: the spatial mean and spatial variance of the image are independently calculated across the sixteen channels, and each channel feature is normalized. Then, an affine transformation is performed using the scaling and translation parameters of the target image dehazing model, allowing the network to adapt to the illumination distribution under different fog concentrations. Finally, a nonlinearity is introduced through an activation function, outputting the first layer of sixteen-channel features. The second lightweight convolutional layer nonlinearly refines the first layer features, similarly performing spatial convolution and instance normalization to obtain the second layer of features. The output layer is used to perform physically constrained mapping on fog concentration-related features. For example, the output layer compresses the sixteen-channel features into a single channel using a 3x3 convolutional kernel, obtaining an unconstrained transmittance prediction value. A sigmoid activation function is introduced as a physical constraint layer, forcing the predicted values ​​to be mapped to a physical range of zero to one. This ensures that transmittance inherently meets the constraints of the atmospheric scattering model regarding the physical meaning of transmittance, i.e., transmittance represents the proportion of light penetrating the haze medium, and physically must be between zero and one. Finally, a single-channel estimated transmittance map is generated. Each pixel value in the estimated transmittance map reflects the haze concentration distribution at the corresponding spatial location: values ​​approaching zero indicate areas of extreme scene depth or dense fog, while values ​​approaching one indicate areas of no fog or light fog.

[0025] Through the concatenated operation of the two lightweight convolutional layers and the output layer, the image to be processed is convolved, instance normalized and activated layer by layer. Finally, the estimated transmittance map is obtained through the physical constraint activation function, which provides key parameters that meet physical validity for the subsequent physical preliminary dehazing module.

[0026] For example, Figure 3This is a schematic diagram of a lightweight convolutional cascaded network structure provided in an embodiment of the present invention. Figure 3 As shown, the lightweight convolutional concatenation network first extracts features through convolutional layers, then performs nonlinear transformations through instance normalization layers and ReLU activation functions; then refines features through convolutional layers, instance normalization layers, and ReLU activation functions; finally, it compresses channels through two-dimensional convolutional layers and applies physical constraints through the Sigmoid activation function to output an estimated transmittance map.

[0027] Step 104: Determine the preliminary dehazing image based on the image to be processed, the estimated transmittance map, and the predetermined atmospheric light parameters.

[0028] Specifically, based on domain big data and atmospheric scattering models, the imaging process of foggy images can be described as follows: ;in The image to be processed. A represents the target dehazed image, where A is the global atmospheric light (atmospheric light parameters). Let be the transmittance of the medium. The imaging process is reversed to solve for this transmittance. To avoid a denominator of 0, the larger of the estimated transmittance map and the preset lower threshold is determined. The difference between the image to be processed and the atmospheric light parameters is determined, and the ratio of this larger value is calculated. This ratio is then added to the atmospheric light parameters to obtain the preliminary dehazed image.

[0029] For example, by reversing the imaging process, in areas with extremely high scene depth and high haze concentration, the transmittance... It will approach 0, therefore a preset lower threshold is introduced for transmittance. conduct The modified preliminary defogging solution formula is as follows, subject to constraints: ; Among them, the lower limit threshold The value is set to 0.1. Specifically, the spatial dimension of the transmittance map t(x) is consistent with the input image, being an H×W single-channel matrix. During the calculation, the single-channel value of the denominator (transmittance map) is expanded to three channels through a broadcast mechanism, meaning that the transmittance value at each spatial location simultaneously affects the red, green, and blue color channels at that location, achieving element-wise division pixel by pixel and channel by channel. Threshold constraints ensure the numerical stability of the reverse solution in the dense fog region, providing a fundamental guarantee for the convergence of the subsequent training process. Atmospheric light A is set as a learnable global scalar parameter of the image dehazing model, adaptively learned by the image dehazing model during training. Based on the physical characteristic that atmospheric light reflects the overall illumination properties of the environment and usually exhibits global consistency, setting global atmospheric light as a learnable global scalar can adaptively capture the overall illumination distribution of the dataset while avoiding spatial noise introduced by the estimation graph form.

[0030] For example, Figure 4 This is a schematic diagram of a preliminary dehazed image provided in an embodiment of the present invention. Figure 4 As shown, the deep feature map and the preliminary dehazed image are taken as input and concatenated along their channel dimensions to form a hybrid feature map. The hybrid feature map is then subjected to channel upscaling via a 1x1 convolutional layer, followed by spatial feature extraction via a 3x3 convolutional layer. This is then followed by nonlinear transformation via an instance normalization layer and the ReLU activation function. Finally, a 2D convolutional layer compresses the channels to output a residual map.

[0031] Step 105: Generate compensation information based on the deep feature map and the preliminary dehazing image, and obtain the target dehazing image based on the compensation information and the preliminary dehazing image.

[0032] The compensation information is pixel-level difference information generated through residual learning to correct defects in the initial dehazed image. The compensation information represents the gap between the initial dehazed image and the true clear image. In one optional implementation, the deep feature map and the initial dehazed image are concatenated and stitched together to obtain a mixed feature; residual learning is performed on the mixed feature to obtain compensation information; and the compensation information and the initial dehazed image are fused to obtain the target dehazed image.

[0033] Specifically, the initial dehazed image and the deep feature map are directly concatenated along the channel dimension to form a hybrid feature with a total of nineteen channels. This hybrid feature integrates physical reconstruction results and deep semantic information. The nineteen-channel hybrid feature is then increased to thirty-two channels through a 1x1 convolutional layer, achieving heterogeneous alignment between physical priors and deep features and expanding the feature interaction space. A 3x3 convolutional layer compresses the channels to sixteen, followed by instance normalization and ReLU activation functions for non-linear feature extraction. Finally, a 3x3 convolutional output layer compresses the channels to three, outputting a predicted residual map, i.e., compensation information. The entire process uses residual learning to capture high-frequency differences between the initial dehazed image and the true clear image, including texture detail enhancement, color correction, and residual fog removal. Furthermore, the compensation information is element-wise superimposed pixel-wise and channel-wise with the initial dehazed image. After superposition, a numerical hard truncation operation is performed to force pixel values ​​to be within the effective range of zero to one, ensuring the physical validity of the output image. Ultimately, a target dehazed image that balances thorough dehazing and texture fidelity is obtained.

[0034] For example, Indicates compensation information; the target dehazed image is ,Will Overlay onto the initial dehazed image get : This process leverages high-frequency details captured by deep learning to dynamically correct reconstruction biases in the physical model, thus balancing thorough dehazing with texture fidelity. Considering that the output tensor value might overflow the effective pixel range after multiple convolutions and residual addition, and to ensure the physical validity of subsequent loss function calculations, a hard numerical truncation operation is introduced at the end of the network. .

[0035] While inverse solving based on a physical model can yield a preliminary dehazed image, the lightweight network has inherent errors in estimating transmittance and atmospheric light, inevitably resulting in residual fog, blurred details, and color shifts in the preliminary dehazed image. By using multi-source feature cascading and residual learning, high-frequency differences between the physical coarse reconstruction result and the real clear image can be captured, enabling detail compensation and color correction, thus forming a collaborative optimization path for physical coarse reconstruction and deep restoration.

[0036] The target image dehazing model in this scheme is a pre-trained model. Optionally, before acquiring the image to be processed, the pre-trained image dehazing model can be trained to obtain the target image dehazing model. Training the pre-trained image dehazing model includes: acquiring a training dataset; selecting a sample from the training dataset as the current sample, inputting the current sample into the pre-established pre-trained image dehazing model, performing dehazing processing on the current sample based on the pre-trained image dehazing model to obtain the current dehazing result; optimizing the trainable parameters of the pre-trained image dehazing model based on a preset loss function, the current dehazing result, and the sample label of the current sample to obtain the current dehazing model; if the current dehazing model does not meet the preset conditions, repeating the above operations until a target image dehazing model that meets the preset conditions is obtained.

[0037] Specifically, after obtaining the training dataset, it can be divided into a training set, a validation set, and a test set. The foggy images in the training set are preprocessed by randomly cropping fixed-size image patches as training samples, and augmentation operations such as horizontal flipping and rotation are used to expand the training data scale, improving the model's robustness and generalization ability. A sample is selected from the training dataset as the current sample and input into the image dehazing model to be trained. The model then performs dehazing on the current sample to obtain the current dehazing result.

[0038] The loss function in this scheme is a weighted combination of four terms: pixel loss, structural loss, perceptual loss, and transmissivity smoothing loss. The loss function is shown in the following equation: ;in, This is the weighting coefficient for pixel loss, and its value can be 1. This is the weighting coefficient for structural loss, and its value can be 0.01. The weighting coefficient for perceived loss can be 0.04. This is the weighting coefficient for the transmittance smoothing loss, and its value can be 0.05.

[0039] For pixel loss, The Charbonnier loss can be used, as shown in the following formula: ; Where N represents the total number of pixels in the image. Represents the output image in pixels The pixel value at the location, i.e., the predicted value. , Represents the real image in pixels Location pixel value, i.e., the true value . It is a very small positive constant used to ensure the smoothness and numerical stability of the function, while making the loss function less sensitive to outliers. Structural loss provides explicit supervision at the structural level of the image, ensuring that the dehazing result maintains the structural integrity of the original scene; the structural loss is defined as follows: SSIM stands for Structural Similarity Index.

[0040] Perceptual loss measures the differences between images at the feature level, helping to capture structural details and texture information, thereby enhancing the perceptual quality of the image. Since the feature maps processed by the ReLU activation function are very sparse, which may limit the model's performance, the feature layers after convolution are chosen as the basis for calculating the perceptual loss to more fully preserve feature information, thus improving training effectiveness and image restoration quality. The definition of perceptual loss is as follows: Where M is the set of deep convolutional neural network (VGG16) layers selected for calculating the loss. They represent the first The number of channels, height, and width of the layer feature map. This represents the L1 norm (i.e., the sum of the absolute values ​​of element-wise differences). Perceptual loss compensates for the limitation of pixel loss, which only focuses on low-level pixel values, by comparing the similarity of images in deep semantic features.

[0041] Transmittance smoothing loss enables the network to dehaze smooth texture regions while preserving structural information of object edges and suppressing artifacts caused by high-frequency noise due to a lack of ground truth supervision. The definition of transmittance smoothing loss is as follows: ;in, This indicates that the transmittance map estimated by the network is in coordinates. The coordinates at that location and These represent the gradient operators in the horizontal and vertical directions, respectively, which are the differences between adjacent pixels. This is a constant used to ensure numerical stability, and its value is 10. -6 Transmittance smoothing loss, as a regularization term, effectively guides the network to generate physically reasonable, edge-clear, and noise-free transmittance maps.

[0042] Furthermore, the trainable model parameters of the model to be trained are adjusted according to the loss function. These trainable model parameters include network weight parameters and learnable scalar parameters (e.g., global atmospheric light parameters A). Preset conditions could be, for example, reaching the maximum number of training batches or the loss function reaching convergence. When the dehazing model to be trained does not meet the preset conditions, its model parameters are adjusted until a target image dehazing model that meets the preset conditions is obtained.

[0043] The technical solution of this embodiment involves acquiring an image to be processed and inputting it into a pre-determined target image dehazing model. Through the multi-scale encoding and decoding network of the target image dehazing model, depth-separable convolution operations are performed layer by layer on the image to be processed to obtain a deep feature map. Through a lightweight convolutional concatenation network of the target image dehazing model, instance normalization operations are performed on the image to be processed to obtain an estimated transmittance map. A preliminary dehazing image is determined based on the image to be processed, the estimated transmittance map, and pre-determined atmospheric light parameters. Compensation information is generated based on the deep feature map and the preliminary dehazing image, and the target dehazing image is obtained based on the compensation information and the preliminary dehazing image. This technical solution, by analyzing the image to be processed in parallel, decouples the physical model from the depth features, simultaneously obtaining the deep feature map and the estimated transmittance map. Through depth-separable convolution, the number of model parameters decreases from quadratic to linear with the increase of the number of channels, achieving model lightweighting. By independently estimating the transmittance map to ensure the ability to perceive physical parameters, and by fusing the deep feature map and the preliminary dehazed image to obtain compensation information and correct the preliminary dehazed image, the difference between the physical coarse reconstruction result and the real clear image can be captured, thereby improving the quality of the dehazed image.

[0044] Figure 5 This is a schematic diagram of an image dehazing apparatus provided in an embodiment of the present invention. This apparatus is suitable for performing the image dehazing method provided in an embodiment of the present invention. Figure 5 As shown, the device may specifically include:

[0045] The data acquisition module 501 is used to acquire the image to be processed and input the image to be processed into a predetermined target image dehazing model; the target image dehazing model includes a feature extraction module, a transmittance estimation module, and an image correction module. The feature extraction module 502 is used to perform depth-separable convolution operations on the image to be processed layer by layer through the multi-scale encoding and decoding network of the feature extraction module to obtain a deep feature map; Transmittance estimation module 503 is used to perform instance normalization operation on the image to be processed through the lightweight convolutional concatenation network of the transmittance estimation module to obtain an estimated transmittance map. The preliminary dehazing module 504 is used to determine a preliminary dehazing image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters. The image correction module 505 is used to generate compensation information based on the deep feature map and the preliminary dehazing image, and to obtain the target dehazing image based on the compensation information and the preliminary dehazing image.

[0046] Optionally, the feature extraction module 502 is specifically used to: perform multi-scale depth convolution and pointwise convolution on the image to be processed through the downsampling layer of the encoder of the multi-scale encoder-decoder network to obtain encoder features; The encoder features are obtained by performing depthwise convolution and pointwise convolution on the upsampling layer of the decoder of the multi-scale encoder-decoder network, and the deep feature map is obtained based on the decoder features.

[0047] Optionally, the transmittance estimation module 503 is specifically used to: perform convolution and instance normalization on the image to be processed based on the lightweight convolutional layer to obtain fog concentration-related features; The concentration-related features are activated by the physical constraint activation function of the output layer to obtain the estimated transmittance map.

[0048] Optionally, the preliminary defogging module 504 is specifically used to: determine the larger value between the estimated transmittance map and the preset lower threshold. Determine the ratio of the difference between the image to be processed and the atmospheric light parameters to the larger value; The ratio and the atmospheric light parameters are added together to obtain the preliminary dehazing image.

[0049] Optionally, the image correction module 505 is specifically used to: perform channel cascade stitching on the deep feature map and the preliminary dehazed image to obtain a hybrid feature; The compensation information is obtained by performing residual learning on the hybrid features; The compensation information and the preliminary dehazed image are fused to obtain the target dehazed image.

[0050] Optionally, the device further includes a model training submodule, which is specifically used for: acquiring a training dataset; Select a sample from the training dataset as the current sample, input the current sample into the pre-established image dehazing model to be trained, and perform dehazing processing on the current sample based on the image dehazing model to be trained to obtain the current dehazing result; Based on the preset loss function, the current dehazing result, and the sample label of the current sample, optimize the trainable parameters of the image dehazing model to be trained, and obtain the current dehazing model; If the current dehazing model does not meet the preset conditions, repeat the above operation until the target image dehazing model that meets the preset conditions is obtained.

[0051] The image dehazing apparatus provided in this embodiment of the invention can execute the image dehazing method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method. Content not described in detail in this embodiment can be referred to the description in any method embodiment of the invention.

[0052] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, with reference to... Figure 6 , Figure 6 The electronic device 12 shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application. Figure 6 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0053] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0054] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0055] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0056] A program / utility 40 having a set (at least one) of program modules 46 may be stored, for example, in system memory 28. Such program modules 46 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 46 typically perform the functions and / or methods described in the embodiments of this application.

[0057] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although... Figure 6 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0058] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing an image dehazing method provided in this embodiment of the invention: acquiring an image to be processed, inputting the image to be processed into a predetermined target image dehazing model; performing depth-separable convolution operations layer by layer on the image to be processed through the multi-scale encoding and decoding network of the target image dehazing model to obtain a deep feature map; performing instance normalization operations on the image to be processed through the lightweight convolutional concatenation network of the target image dehazing model to obtain an estimated transmittance map; determining a preliminary dehazing image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters; generating compensation information based on the deep feature map and the preliminary dehazing image, and obtaining a target dehazing image based on the compensation information and the preliminary dehazing image.

[0059] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an image dehazing method as provided in all embodiments of this invention: acquiring an image to be processed; inputting the image to be processed into a predetermined target image dehazing model; performing depth-separable convolution operations layer by layer on the image to be processed through the multi-scale encoding and decoding network of the target image dehazing model to obtain a deep feature map; performing instance normalization operations on the image to be processed through the lightweight convolutional concatenation network of the target image dehazing model to obtain an estimated transmittance map; determining a preliminary dehazing image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters; generating compensation information based on the deep feature map and the preliminary dehazing image, and obtaining a target dehazing image based on the compensation information and the preliminary dehazing image. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electronic device, apparatus, or device that is electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used or combined with an electronic device, apparatus, or device by instructions to execute it.

[0060] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in conjunction with an electronic device, apparatus, or device that executes instructions.

[0061] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0062] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An image dehazing method, characterized in that, The method includes: Obtain the image to be processed and input the image to be processed into a pre-determined target image dehazing model; The target image dehazing model uses a multi-scale encoder-decoder network to perform depth-separable convolution operations on the image to be processed layer by layer to obtain a deep feature map. The image to be processed is subjected to instance normalization operation through the lightweight convolutional concatenation network of the target image dehazing model to obtain an estimated transmittance map. A preliminary dehazing image is determined based on the image to be processed, the estimated transmittance map, and the predetermined atmospheric light parameters; Compensation information is generated based on the deep feature map and the preliminary dehazing image, and the target dehazing image is obtained based on the compensation information and the preliminary dehazing image.

2. The method according to claim 1, characterized in that, The target image dehazing model uses a multi-scale encoder-decoder network to perform depthwise separable convolution operations on the image to be processed layer by layer to obtain deep feature maps, including: The encoder features are obtained by performing multi-scale depth convolution and pointwise convolution on the image to be processed through the downsampling layer of the encoder of the multi-scale encoder-decoder network. The encoder features are obtained by performing depthwise convolution and pointwise convolution on the upsampling layer of the decoder of the multi-scale encoder-decoder network, and the deep feature map is obtained based on the decoder features.

3. The method according to claim 1, characterized in that, The lightweight convolutional concatenation network comprises two lightweight convolutional layers and an output layer; through the lightweight convolutional concatenation network of the target image dehazing model, instance normalization is performed on the image to be processed to obtain an estimated transmittance map, including: Based on the lightweight convolutional layer, the image to be processed is convolved and instance normalized to obtain fog concentration-related features; The concentration-related features are activated by the physical constraint activation function of the output layer to obtain the estimated transmittance map.

4. The method according to claim 1, characterized in that, Determining a preliminary dehazed image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters includes: The larger value between the estimated transmittance map and the preset lower threshold is determined; Determine the ratio of the difference between the image to be processed and the atmospheric light parameters to the larger value; The ratio and the atmospheric light parameters are added together to obtain the preliminary dehazing image.

5. The method according to claim 1, characterized in that, The process includes generating compensation information based on the deep feature map and the preliminary dehazed image, and obtaining the target dehazed image based on the compensation information and the preliminary dehazed image, including: The deep feature map and the preliminary dehazed image are concatenated and stitched together to obtain a hybrid feature; The compensation information is obtained by performing residual learning on the hybrid features; The compensation information and the preliminary dehazed image are fused to obtain the target dehazed image.

6. The method according to claim 1, characterized in that, Before acquiring the image to be processed, the method further includes: Obtain the training dataset; Select a sample from the training dataset as the current sample, input the current sample into the pre-established image dehazing model to be trained, and perform dehazing processing on the current sample based on the image dehazing model to be trained to obtain the current dehazing result; The trainable parameters of the image dehazing model are optimized based on the preset loss function, the current dehazing result, and the sample label of the current sample to obtain the current dehazing model; the loss function includes pixel loss function, structural loss function, perceptual loss function, and transmittance smoothing loss function. If the current dehazing model does not meet the preset conditions, repeat the above operation until the target image dehazing model that meets the preset conditions is obtained.

7. An image dehazing device, characterized in that, The device includes: The data acquisition module is used to acquire the image to be processed and input the image to be processed into a pre-determined target image dehazing model; the target image dehazing model includes a feature extraction module, a transmittance estimation module, and an image correction module. The feature extraction module is used to perform depth-separable convolution operations on the image to be processed layer by layer through the multi-scale encoding and decoding network of the feature extraction module to obtain a deep feature map; The transmittance estimation module is used to perform instance normalization operation on the image to be processed through the lightweight convolutional concatenation network of the transmittance estimation module to obtain an estimated transmittance map. The preliminary dehazing module is used to determine a preliminary dehazing image based on the image to be processed, the estimated transmittance map, and predetermined atmospheric light parameters; The image correction module is used to generate compensation information based on the deep feature map and the preliminary dehazing image, and to obtain the target dehazing image based on the compensation information and the preliminary dehazing image.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image dehazing method as described in any one of claims 1-6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the image dehazing method as described in any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the image dehazing method as described in any one of claims 1 to 6.

Images