Image defogging method and system based on adaptive convolution and texture prior

Abstract

The invention discloses an image defogging method and system based on adaptive convolution and texture prior, and belongs to the technical field of computer vision. The method comprises the following steps: acquiring an original image; inputting the original image into a self-adaptive multi-scale convolution module in the defogging model for processing to obtain a coding feature map; according to a discrete feature codebook obtained through pre-training, optimizing the coding feature map through a texture restoration module in a defogging model to obtain a texture restoration feature map; and processing the texture restoration feature map through a decoder in a defogging model to obtain a defogged target image. According to the scheme, on one hand, the feature extraction effect of non-uniform fog spots in a complex real scene is improved through the adaptive multi-scale convolution module, and on the other hand, the smoothed texture in the feature extraction process is restored through the texture restoration module, so that the defogging effect of the image is improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to an image dehazing method and system based on adaptive convolution and texture prior. Background Technology

[0002] Haze and smog cause reduced visibility, degraded contrast, color distortion, and loss of detail in outdoor images due to the absorption and scattering of light by suspended particles in the atmosphere. Therefore, image dehazing has always been a fundamental and important visual task.

[0003] Current image dehazing methods typically utilize convolutional neural networks or Transformer architectures. These networks are trained on large-scale synthetic pairwise datasets to learn an end-to-end mapping from foggy to clear images, or they dehaze by predicting physical parameters. However, existing methods still fall short in recovering high-frequency details when dealing with non-uniform fog and extremely dense fog. During global dehazing, most models struggle to distinguish between dense and light fog regions, leading to over-enhancing of light fog areas or failure to dehaze dense fog areas. Furthermore, when recovering texture details severely obscured by dense fog, existing loss functions tend to smooth the recovery, losing high-frequency texture and edge information, resulting in dehazed images that lack visual realism and clarity. Summary of the Invention

[0004] Therefore, it is necessary to provide an image dehazing method and system based on adaptive convolution and texture prior to address the above problems, thereby improving the image dehazing effect.

[0005] This application provides an image dehazing method based on adaptive convolution and texture prior, the method comprising:

[0006] Obtain the original image; The original image is input into the adaptive multi-scale convolution module in the dehazing model for processing to obtain an encoded feature map; the convolution kernel in the adaptive multi-scale convolution module has a receptive field corresponding to a pixel. Based on the discrete feature codebook obtained through pre-training, the encoded feature map is optimized by the texture restoration module in the dehazing model to obtain a texture restoration feature map; the discrete feature codebook includes multiple high-definition texture feature vectors; The texture restoration feature map is processed by the decoder in the dehazing model to obtain the dehazed target image.

[0007] In one optional implementation, the adaptive multi-scale convolution module includes a geometric shape prediction module, a dynamic resampling module, and a feature modulation module; The original image is input into the adaptive multi-scale convolution module of the dehazing model for processing to obtain an encoded feature map, including: The original image is input into the geometric shape prediction module to obtain a height scale map and a width scale map; the height scale map and the width scale map are used to represent the receptive field expansion intensity of each coordinate in the vertical and horizontal directions, respectively; The height and width scale maps are input into the dynamic resampling module to generate an adaptive sampling grid. The original image features are then resampled and aggregated based on the adaptive sampling grid to obtain a sampled feature map. The original image is input into the feature modulation module for processing to obtain the predicted modulation coefficients and bias coefficients. The sampled feature map is modulated according to the modulation coefficient and the bias coefficient to obtain the coded feature map.

[0008] In one optional implementation, the original image is input into the geometric shape prediction module to obtain a height scale map and a width scale map, including: The original image is used to extract features through a feature extraction network to obtain the first feature map; The height-scale map is obtained using the following formula. and width scale diagram :

[0009]

[0010] in Represents a high-level convolutional mapping; Represents the width convolution mapping, For the Sigmoid function, The upper limit of the preset height; This is the upper limit of the preset width; This is the first feature map.

[0011] In one optional implementation, the step of inputting the height scale map and width scale map into the dynamic resampling module to generate an adaptive sampling grid, and resampling and aggregating the original image features according to the adaptive sampling grid to obtain a sampled feature map includes: The sampling grid size is determined based on the global mean of the height and width scale maps for each coordinate. Based on the height and width scale maps of each coordinate and the sampling grid size, an adaptive sampling interval is determined for each coordinate to generate an adaptive sampling grid; the adaptive sampling grid has a corresponding adaptive sampling interval for each coordinate. Based on the adaptive sampling grid, the original image features are resampled by bilinear interpolation in a continuous space to obtain the expanded resampled features; Based on the sampling grid size, a corresponding aggregation convolution kernel is selected to perform intra-block aggregation on the resampled features, restoring the original spatial resolution and obtaining the sampling feature map.

[0012] In one optional implementation, modulating the sampled feature map according to the modulation coefficients and bias coefficients to obtain the coded feature map includes: The encoded feature map is obtained using the following formula:

[0013] in, The modulation coefficient; This is the bias coefficient; This is element-wise multiplication; For sampling feature maps; This is for encoding feature maps.

[0014] In one optional implementation, the step of optimizing the encoded feature map using a texture restoration module in a dehazing model based on a pre-trained discrete feature codebook to obtain a texture restoration feature map includes: The encoded feature map is mapped from the original vector space to the feature vector space corresponding to the discrete feature codebook to obtain each vector to be processed. High-resolution texture feature vectors similar to each vector to be processed are selected from the discrete feature codebook. The encoded feature map is optimized based on high-resolution texture feature vectors similar to each vector to be processed, to obtain the texture restoration feature map.

[0015] In one optional implementation, optimizing the encoded feature map based on high-resolution texture feature vectors that are similar to each vector to be processed to obtain the texture restoration feature map includes: Map the high-resolution texture feature vectors that are similar to each vector to be processed back to the original vector space to obtain a quantized feature map; The encoded feature map is input into the affine parameter prediction network to obtain the scaling factor and bias factor. The quantized feature map is processed according to the scaling factor and the bias factor to obtain a third feature map; The third feature map is fused with the encoded feature map to obtain the texture restoration feature map.

[0016] In an optional implementation, the method further includes: Acquire both fogged images and real images of the samples; The sample hazy image is input into the dehazing model to obtain the dehazed predicted image; Based on the predicted image and the sample real image, the final loss function value is obtained through a preset loss function, and the parameters of the dehazing model are updated according to the final loss function value.

[0017] In one optional implementation, the preset loss function is as follows:

[0018]

[0019] in, To predict the image, This is the final loss function value. For the sample real image, and The balance coefficient changes dynamically with each training round; in, Pixel reconstruction loss is used to characterize the difference in pixel values ​​between the predicted image and the sample real image; Multi-scale perceptual loss is used to characterize the differences between the predicted image and the sample real image at different levels of feature maps; This is a boundary-aware loss used to characterize the difference in edge intensity between the predicted image and the real sample image; The semantic consistency constraint loss is used to characterize the distance between the predicted image and the sample foggy image in the deep feature space; The structure-preserving loss is used to characterize the difference in gradient between the feature maps of the predicted image and the feature maps of the sample real image.

[0020] Furthermore, this application also provides an image dehazing system based on adaptive convolution and texture prior, the system comprising: The raw image acquisition unit is used to acquire the raw image. The encoding unit is used to input the original image into the adaptive multi-scale convolution module for processing to obtain an encoded feature map; the convolution kernel in the adaptive multi-scale convolution module has a receptive field corresponding to a pixel. The texture restoration unit is used to optimize the encoded feature map based on the pre-trained discrete feature codebook to obtain a texture restoration feature map; the discrete feature codebook includes multiple high-resolution texture feature vectors. The decoding unit processes the texture restoration feature map through the decoder to obtain the dehazed target image.

[0021] Compared with the prior art, the technical solution provided in this application has the following advantages: After acquiring the original image, it can be input into the adaptive multi-scale convolution module of the dehazing model for processing to obtain an encoded feature map. The convolution kernels in this model have receptive fields corresponding to pixels, allowing for dynamic adjustment of the receptive field shape and sampling based on the fog concentration and scene structure in the original image, thus improving the accuracy of the encoded feature map. Then, based on the high-resolution texture feature vectors from the pre-trained discrete feature codebook, the texture restoration module in the dehazing model optimizes the encoded feature map to obtain a texture restoration feature map. Finally, the decoder processes the texture restoration feature map to obtain the dehazed target image. This approach improves the feature extraction effect for non-uniform fog spots in complex real-world scenes through the adaptive multi-scale convolution module and restores the textures smoothed out during feature extraction through the texture restoration module, thereby enhancing the dehazing effect. Attached Figure Description

[0022] Figure 1 A flowchart of an image dehazing method based on adaptive convolution and texture prior is shown in one embodiment of the present invention.

[0023] Figure 2 A flowchart of an image dehazing method based on adaptive convolution and texture prior is shown in one embodiment of the present invention.

[0024] Figure 3 A schematic diagram of an adaptive multi-scale convolution module according to an embodiment of this application is shown.

[0025] Figure 4 The illustration shown is a schematic diagram of the principle of a discrete codebook-guided texture restoration module according to an embodiment of this application.

[0026] Figure 5 A flowchart illustrating the overall model involved in an embodiment of this application is shown.

[0027] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention. Detailed Implementation

[0028] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0029] In the fields of computer vision and image processing, image dehazing has always been a fundamental and highly regarded low-level vision task. Haze and fog, due to the absorption and scattering of light by suspended particles in the atmosphere, cause outdoor images to suffer from reduced visibility, degraded contrast, color distortion, and loss of detail. This not only severely impacts the visual quality of images but also significantly hinders the performance and reliability of subsequent advanced computer vision tasks. Therefore, researching efficient and robust single-image dehazing algorithms has significant application value and practical implications for improving the environmental adaptability of outdoor vision systems.

[0030] Existing image dehazing techniques are mainly divided into two categories: physically based methods and deep learning-based methods. Early physically based methods relied primarily on atmospheric scattering models, estimating transmittance and atmospheric light through manually designed prior assumptions by mining the statistical characteristics of clear images, and then inversely inferring the clear image. However, these methods tend to fail when faced with complex scenes that do not conform to the prior assumptions. In recent years, with the rapid development of deep learning technology, data-driven dehazing methods have gradually become mainstream. These methods typically utilize convolutional neural networks or Transformer architectures, learning an end-to-end mapping from hazy to clear images by training on large-scale synthetic pairwise datasets, or by predicting physical parameters for dehazing.

[0031] Existing methods still fall short in restoring high-frequency details when dealing with non-uniform fog and extremely dense fog. Most models struggle to distinguish between dense and light fog regions during global dehazing, leading to over-enhancing of light fog areas or failure to dehaze dense fog areas. Furthermore, when restoring texture details severely obscured by dense fog, existing loss functions tend to smooth the restoration, losing high-frequency texture and edge information, resulting in a visually unrealistic and less sharp image after dehazing.

[0032] Therefore, to address the aforementioned problems, this application provides an image dehazing method based on adaptive convolution and texture prior. In the encoder stage, this invention employs an adaptive multi-scale convolution module instead of standard convolution. This module can dynamically adjust the shape and sampling position of the receptive field according to the fog concentration and scene geometry, thereby accurately capturing anisotropic structural features during multi-level downsampling. At the connection between the encoder and decoder, this invention introduces a discrete codebook-guided texture restoration module. This module utilizes a pre-trained high-quality discrete codebook as a prior knowledge base, and performs texture restoration on damaged deep features through feature space retrieval and matching. The decoder upsamples and fuses the features that have undergone structural enhancement and texture restoration, outputting a high-fidelity, clear image.

[0033] Figure 1 A flowchart of an image dehazing method based on adaptive convolution and texture prior is shown in one embodiment of the present invention, as follows: Figure 1 As shown, the method flow includes: Step 101: Obtain the original image.

[0034] In the embodiments of this application, the original image can be acquired in real time by an image acquisition device, or it can be read from a pre-stored image dataset.

[0035] For example, image acquisition equipment can be surveillance cameras, vehicle-mounted cameras, drone vision systems, or mobile terminal camera modules. When image acquisition is carried out in hazy or low-visibility conditions, suspended particles in the atmosphere absorb and scatter the propagating light, resulting in degradation phenomena such as brightness reduction, contrast decrease, and color shift in the acquired images. Furthermore, the detailed textures of distant targets are often severely obscured or blurred by the fog.

[0036] In some embodiments, preprocessing operations can be performed on the original image before it is input into the dehazing model. For example, the image can be scaled to match its resolution to the input size of the dehazing model; the image pixels can also be normalized to map pixel values ​​to a specified range, thereby improving the stability of the neural network training and inference process. The preprocessed image is then fed as input data into the encoder part of the dehazing model for feature extraction.

[0037] Step 102: Input the original image into the adaptive multi-scale convolution module in the dehazing model for processing to obtain the encoded feature map.

[0038] In the adaptive multi-scale convolution module, the convolution kernel has a receptive field corresponding to the pixel.

[0039] Specifically, after acquiring the original image, the original image is input into the adaptive multi-scale convolution module in the dehazing model for feature extraction to obtain the encoded feature map.

[0040] In this embodiment, the adaptive multi-scale convolution module can constitute the encoder structure of the dehazing model. Its main function is to extract deep features that can characterize scene structure information and fog distribution information from the input image layer by layer. Unlike the convolution operation in traditional convolutional neural networks that uses a fixed-size convolution kernel, the convolution operation in this application introduces an adaptive receptive field mechanism. That is, when the convolution kernel extracts features at different pixel locations, its sampling position and receptive field range can be dynamically adjusted according to the image content. Thus, in areas with more complex texture structures, a smaller and more refined receptive field can be used for feature extraction to preserve image edge and detail information; while in areas with thicker fog or less texture information, the receptive field range can be appropriately expanded to capture a larger range of contextual information to better characterize the spatial distribution of fog.

[0041] Step 103: Based on the discrete feature codebook obtained through pre-training, the encoded feature map is optimized by the texture restoration module in the dehazing model to obtain the texture restoration feature map.

[0042] The discrete feature codebook includes multiple high-definition texture feature vectors.

[0043] In this embodiment, the texture restoration module introduces a discrete feature codebook as a prior knowledge base for textures to compensate for the loss of texture information caused by fog occlusion or feature compression during the encoding process. The discrete feature codebook can be constructed through a pre-training process. For example, a large number of clear, fog-free images can be selected as training samples, local texture features in the images can be obtained through a feature extraction network, and these texture features can be discretized using vector quantization or clustering algorithms to form a discrete codebook containing multiple typical texture feature vectors.

[0044] In the actual dehazing process, the texture restoration module maps local feature vectors in the encoded feature map to the feature space of the discrete feature codebook and calculates the similarity between the local feature and each texture feature vector in the codebook. Subsequently, based on the similarity results, the texture feature vector that is closest to the current feature is retrieved from the discrete codebook, and this texture feature vector is used to compensate for or fuse the original encoded feature, thereby restoring the texture details weakened by the fog.

[0045] Through the aforementioned feature retrieval and texture compensation mechanisms, prior texture information from high-resolution images can be introduced into the deep feature space. This enables the network to generate clearer details with realistic textures when restoring image structures, thus avoiding the over-smoothing problem that occurs in traditional dehazing models when reconstructing images. After processing by the texture restoration module, a texture restoration feature map containing richer high-frequency texture information can be obtained.

[0046] Step 104: The texture restoration feature map is processed by the decoder in the dehazing model to obtain the dehazed target image.

[0047] After obtaining the texture restoration feature map, the texture restoration feature map is processed by the decoder in the dehazing model to generate the dehazed target image.

[0048] In this embodiment, the decoder is mainly used to gradually restore the deep features obtained in the encoding stage to the spatial resolution of the original image and map the features to the final image pixel values. During the decoding process, the spatial size of the feature map can be restored layer by layer through upsampling operations, such as deconvolution, interpolation upsampling, or pixel rearrangement to improve the resolution of the feature map. After multi-layer decoding and feature fusion, the feature mapping is finally converted into pixel values ​​in the image space through one or more convolution operations, thereby outputting the dehazed target image. Compared with the input original image, the haze effect in the target image is significantly reduced, the overall image contrast is improved, and the edge structure and texture details in the image are effectively restored, resulting in a clearer image with higher visual quality.

[0049] In summary, after acquiring the original image, it can be input into the adaptive multi-scale convolution module of the dehazing model for processing to obtain an encoded feature map. The convolution kernels in this model have receptive fields corresponding to pixels, allowing for dynamic adjustment of the receptive field shape and sampling based on the fog concentration and scene structure in the original image, thus improving the accuracy of the encoded feature map. Then, based on the high-resolution texture feature vectors from the pre-trained discrete feature codebook, the texture restoration module in the dehazing model optimizes the encoded feature map to obtain a texture restoration feature map. Finally, the decoder processes the texture restoration feature map to obtain the dehazed target image. This approach improves the feature extraction effect for non-uniform fog spots in complex real-world scenes through the adaptive multi-scale convolution module and restores the textures smoothed out during feature extraction through the texture restoration module, thereby enhancing the dehazing effect.

[0050] Figure 2 A flowchart of an image dehazing method based on adaptive convolution and texture prior is shown in one embodiment of the present invention, as follows: Figure 2 As shown, the method flow includes: Step 201: Obtain the original image.

[0051] Step 201 can be referred to step 101 above, and will not be repeated here.

[0052] Step 202: Input the original image into the geometric shape prediction module to obtain the height scale map and the width scale map; The height scale map and the width scale map are used to represent the receptive field expansion intensity of each coordinate in the vertical and horizontal directions, respectively.

[0053] Figure 3 A schematic diagram of an adaptive multi-scale convolution module according to an embodiment of this application is shown. Figure 3 As shown, the adaptive multi-scale convolution module mainly consists of three parts: a geometric shape prediction module, a dynamic resampling module, and a feature modulation module. The geometric shape prediction module is used to perceive the local structural orientation and scale requirements of the input features. The dynamic resampling module is responsible for performing adaptive sampling and convolution operations on feature points in continuous space. The feature modulation module introduces a feature modulation mechanism based on the rectangular convolution output.

[0054] The geometric shape prediction module takes the original image X as input and predicts the height scale map pixel by pixel. With width scale diagram Where H is the pixel height of the original image and W is the pixel width of the original image. and Representing the position respectively The intensity of the receptive field extension along the vertical and horizontal directions is used to modulate the spatial distribution range of the convolution sampling points.

[0055] Given an input image The first feature map is obtained by extracting features from the original image using a feature extraction network. ;in ; This means that the input image is processed through a feature extraction network. Feature extraction is performed.

[0056] Two parallel scale prediction branches are introduced to estimate the spatial scale of the convolutional sampling grid in the height and width directions, respectively. Specifically, the height scale map is obtained using the following formula. and width scale diagram :

[0057]

[0058] in This represents a high-level convolutional mapping (composed of multiple convolutional layers); This represents a width-based convolutional mapping (consisting of multiple convolutional layers). For the Sigmoid function, The upper limit of the preset height; Set the upper bound of the preset width (e.g., 7); This is the first feature map.

[0059] Step 203: Input the height scale map and the width scale map into the dynamic resampling module to generate an adaptive sampling grid, and resample and aggregate the original image features according to the adaptive sampling grid to obtain a sampling feature map.

[0060] To achieve step 203 above, this application first determines the sampling grid size based on the global mean of the height scale map and the width scale map of each coordinate. Then, based on the height and width scale maps of each coordinate and the sampling grid size, the adaptive sampling interval for each coordinate is determined to generate an adaptive sampling grid; the adaptive sampling grid has a corresponding adaptive sampling interval for each coordinate. Based on the adaptive sampling grid, the original image features are resampled by bilinear interpolation in a continuous space to obtain the expanded resampled features; Based on the sampling grid size, the corresponding aggregation convolution kernel is selected, and the resampled features are aggregated within the block to restore the original spatial resolution and obtain the sampling feature map.

[0061] The above steps can be implemented as follows: In the dynamic resampling module, to obtain the overall sampling scale requirement under the current feature level, the dynamic resampling module takes the global mean of the height scale map and the width scale map in the spatial dimension:

[0062] and It reflects the average geometric scale of the feature map in a global sense (i.e., in height and width).

[0063] Then, a scale normalization factor is introduced in the embodiments of this application. The mean scale is normalized to control the impact of scale prediction on the sampling size, and continuous values ​​are mapped to a discrete convolutional kernel size space. To ensure the spatial central symmetry of the sampling grid, an odd-numbered mapping function is defined. The scale value is constrained to an odd integer. The final sampling grid size is determined by the following formula:

[0064] and These represent the minimum and maximum allowed sampling sizes, which are 3 and 7 respectively. This is to avoid insufficient information due to an excessively small receptive field, or redundant calculations and noise due to an excessively large receptive field.

[0065] Odd number mapping function Defined as:

[0066] This mapping ensures that the sampling grid always has a unique center location, thus maintaining consistency with the symmetric structure of the convolution operation. This is crucial when determining the sampling scale. Then, the module constructs a zero-centered regular grid. The grid is composed of It consists of several relative coordinate points, describing the spatial topology of a standard convolutional kernel. Its coordinates are usually uniformly distributed in the normalized space.

[0067] The module utilizes the scale parameter predicted pixel by pixel. and The base mesh is adaptively scaled to generate the final sampled coordinate field. For the location in the feature map... , its first Coordinates of each sampling point It can be represented as:

[0068] in Indicates position The center coordinates on the feature map, Basic grid The first in Each relative coordinate point. This process scales the adaptive sampling interval, enabling the convolutional receptive field to adaptively expand or shrink according to the local image structure.

[0069] Since the predicted sampling coordinates are usually located in continuous space, the module uses bilinear interpolation to adjust the input feature map. Perform resampling. For any sampling location... Let its top-left corner integer pixel coordinates be Then its four neighboring pixel set Defined as:

[0070] Bilinear interpolation kernel function Defined as the product of a one-dimensional linear kernel along two coordinate axes. For any sampling position p, any neighboring pixels... The interpolation weights are calculated as follows:

[0071] Location eigenvalues ​​at Can be written as

[0072] In its implementation, this interpolation process is achieved by weighted summation of the features of four neighboring pixels, directly related to the continuous coordinate offset. This allows the sampling operation to effectively propagate gradients to the geometry prediction branch during backpropagation. Through this bilinear interpolation mechanism, the module obtains resampled features that adaptively adapt to the geometry. .

[0073] The expanded resampled features obtained by bilinear interpolation are denoted as Its dimensions are:

[0074] Specifically, a feature map is generated in a continuous coordinate domain, centered at each spatial location (i,j) of the original feature map. OK, A regular grid composed of columns is used to obtain the corresponding data from the original feature map through interpolation. Each sampled feature value is then used. The sampled points at that location are then rearranged into spatial blocks and stitched into the expanded feature map. Thus, and These represent the number of sampling points in the vertical and horizontal directions, respectively, which determine the topological scale of the sampling point array.

[0075] The module will Considered as being by An extended feature map is formed by stitching together non-overlapping local rectangular sampling blocks in a spatial dimension, where the spatial size of each sampling block is... To aggregate the features of the point set within the block and restore them to the original spatial resolution, based on the current sampling scale... Select matching rectangular aggregation kernels And perform intra-block aggregation independently for each sampling block. This process can be implemented by standard 2D convolution, with the kernel size and stride both set to... This ensures that the sampling blocks do not overlap and restores the output resolution to [normal value]. , denoted as:

[0076] in Represents a two-dimensional convolution operator. To extract from a pre-defined multi-scale rectangular kernel set by Selected learnable weights, The step size is used to obtain the output Z (which is the above-mentioned sampled feature map).

[0077] Step 204: Input the original image into the feature modulation module for processing to obtain the predicted modulation coefficients and bias coefficients.

[0078] To further enhance the flexibility of feature representation, the module introduces a feature modulation mechanism on top of the rectangular convolution output, adjusting the amplitude of the convolution response element by element. Modulation coefficients With bias term The features from the original input are obtained by parallel modulation networks and bias networks, respectively. This was predicted.

[0079] Step 205: Modulate the sampled feature map according to the modulation coefficient and the bias coefficient to obtain the coded feature map.

[0080] The modulation network models the relationship between input features and output amplitude through multiple layers of convolution and nonlinear mapping. Its output is then processed... The activation function imposes constraints to avoid excessive amplitude scaling, while the bias network learns spatially relevant feature translation terms to compensate for systematic biases in the convolutional response. Both affect the feature map. Linear modulation is performed to obtain the final output characteristics.

[0081] In this embodiment, the encoded feature map can be obtained using the following formula:

[0082] Where M is the modulation coefficient; b is the bias coefficient; Z represents element-wise multiplication; Z is the sampled feature map; Y is the encoded feature map.

[0083] Step 206: Map the encoded feature map from the original vector space to the feature vector space corresponding to the discrete feature codebook to obtain each vector to be processed.

[0084] Figure 4 The illustration shown is a schematic diagram of the principle of a discrete codebook-guided texture restoration module according to an embodiment of this application. Figure 4 As shown, the discrete codebook-guided texture restoration module is mainly deployed at the connection bottleneck between the encoder and the decoder, aiming to solve the problem of high-frequency texture loss and detail smoothing that occurs after deep features have been downsampled multiple times.

[0085] This module introduces a pre-trained discrete feature codebook as an external prior knowledge base. By mapping blurred deep features to a discrete latent space, retrieving the best-matching clear texture prototype, and utilizing an adaptive affine transformation mechanism to resolve the distribution difference between discrete and continuous features, this module achieves micro-texture restoration of dehazed images.

[0086] First, the feature input and codebook space are defined. The module receives features from the encoder end. As input, its dimensions are denoted as (B, C, H', W'), where B is the batch size, C is the number of channels, and H' and W' are the height and width of the downsampled feature map, respectively. The module is pre-loaded with a discrete codebook pre-trained on a large-scale, high-resolution image dataset. The codebook contains K high-dimensional feature vectors, each representing a typical high-resolution texture pattern. The codebook is in the form of a set. Defined as:

[0087] in Represents the k-th codeword vector. For feature embedding dimension. This codebook It serves as a dictionary for texture reconstruction. To accurately match textures even with domain differences, the module performs nearest neighbor retrieval based on reweighted distance. To achieve nearest neighbor retrieval, the encoded features must first be linearly mapped to an embedding space consistent with the codewords to obtain each spatial location. Corresponding feature vector .

[0088] Step 207: Select high-resolution texture feature vectors that are similar to each vector to be processed from the discrete feature codebook.

[0089] Specifically, in this embodiment, the texture restoration module calculates the Euclidean distance between the feature and all codewords in the codebook, using the underlying distance metric... Defined as,

[0090] Considering the significant differences in texture distribution between degraded and clear images, direct matching based on Euclidean distance is easily affected by high-frequency noise or anomalous textures. The texture restoration module introduces a frequency prior based on codeword statistical characteristics.

[0091] All positions in the training set were assigned to the first position. Number of times per code character Recorded as:

[0092] in It is an indicator function used to count the number of samples that meet the conditions; it takes the value 1 if the condition is true and 0 otherwise. Used to determine image v in the training set The codeword index used for the position. Then, the count is normalized to obtain the usage frequency. : in, ,and This is a constant used for smoothing to prevent certain codewords from never being hit, thus avoiding... Take 1e4.

[0093] Then, the distance term is adaptively reweighted for each codeword. The corresponding frequency descriptor obtained from the previous step This is used to reflect the stability and reliability of the texture prototype in the training distribution. Based on this, a frequency modulation function is constructed to calculate the frequency weights. :

[0094] in These are adjustable hyperparameters used to control the strength of the influence of prior weights on the matching process. In the implementation, the function... Using a monotonically increasing exponent exp:

[0095] After introducing frequency weights, the weighted distance It is represented as:

[0096] This weighted distance takes into account both feature similarity and texture prior reliability.

[0097] The module is in all codeword index sets Perform a search and select the codeword with the smallest weighted distance as the position. The index of the matching result is given by the following formula:

[0098] Based on the obtained index map, the module extracts the corresponding codeword vectors from the codebook and performs spatial recombination, as follows: For any coordinate on the feature map... The module obtains the best matching index based on the above process. Extract the corresponding high-dimensional codeword vector from the codebook. And map it back to the original spatial coordinates.

[0099] Step 208: Optimize the encoded feature map based on the high-definition texture feature vectors similar to each vector to be processed to obtain the texture restoration feature map.

[0100] Furthermore, the encoded feature map is optimized based on high-resolution texture feature vectors similar to each vector to be processed to obtain the texture restoration feature map, including: High-resolution texture feature vectors similar to each vector to be processed are mapped back to the original vector space to obtain a quantized feature map; the encoded feature map is input into an affine parameter prediction network to obtain a scaling factor and a bias factor; the quantized feature map is processed according to the scaling factor and the bias factor to obtain a third feature map; the third feature map is fused with the encoded feature map to obtain the texture restoration feature map.

[0101] Specifically, by traversing all pixel positions, the corresponding high-dimensional codeword vector is extracted from the codebook. After mapping it back to the original spatial coordinates, the system constructs the final discrete quantization feature. At this time Composed of clear, high-quality features, it theoretically eliminates noise and blur interference caused by fog, but its feature distribution may be discontinuous with the semantic environment of the current image.

[0102] To solve the problem of quantization features With original features To address the mismatch between semantics and distribution, this invention designs an affine parameter prediction network. This network uses the original features... Using the input as a condition, local semantic intensity and illumination distribution information of the image are extracted through multi-layer convolution and activation functions to dynamically predict the scaling factor γ and bias factor β used for feature modulation. This prediction process can be summarized as follows:

[0103] in This represents an affine parameter prediction network that extracts semantic and imaging attribute information relevant to the current scene from the original encoded features. The network uses... As input, it first passes through several layers. Convolution models the local context and progressively compresses the stable feature distribution. A non-linear activation function, ReLU, is inserted between convolutional layers.

[0104] Then, after feature extraction, the network splits into two parallel prediction branches, one for generating scaling parameters and the other for generating scaling parameters. With bias parameters Each branch consists of one layer. Convolutional structures are used to linearly map intermediate features to quantized features. The parameter space is aligned with the number of channels and the spatial resolution, thus yielding:

[0105] Branches typically employ restricted activation functions to constrain the scaling magnitude, preventing feature over-amplification or suppression. The branch directly outputs a bias response that can be positive or negative, used to compensate for the semantic offset introduced by discrete quantization.

[0106] The module uses the generated modulation parameters γ and β to quantize the features. Feature transformation is performed to inject the semantic structure information of the original features into the high-resolution quantized texture, achieving texture alignment. The corrected texture features... The calculation formula is as follows:

[0107] in This represents element-wise multiplication. Through this step, discrete codebook features are remapped back to a continuous feature space that conforms to the semantics of the current image.

[0108] The module employs residual connections for feature fusion and output. To preserve the original image's geometric structure as much as possible while restoring texture, the module combines the corrected texture features... After a simple 3×3 convolution process, and compared with the original input features Perform element-wise addition. The final output feature. The calculation formula is expressed as follows:

[0109] Step 209: The texture restoration feature map is processed by the decoder in the dehazing model to obtain the dehazed target image.

[0110] Figure 5 A flowchart illustrating an overall model according to an embodiment of this application is shown. Figure 5 As shown, in the encoder stage, this invention employs an adaptive multi-scale convolution module instead of standard convolution. This module dynamically adjusts the shape and sampling position of the receptive field of the input X based on fog concentration and scene geometry, thereby accurately capturing anisotropic structural features during multi-level downsampling. At the connection between the encoder and decoder, this invention introduces a discrete codebook-guided texture restoration module. This module utilizes a pre-trained high-quality discrete codebook as a prior knowledge base, and performs texture restoration on damaged deep features through feature space retrieval and matching. The decoder upsamples and fuses the features that have undergone structural enhancement and texture restoration, thereby ensuring that the output Y is a high-fidelity, clear image.

[0111] The adaptive multi-scale convolution module extracts depth structure information and anisotropic texture features from images through shape-adaptive sampling and multi-aspect-ratio convolution branches. This invention introduces an adaptive geometric transformation mechanism composed of a shape and offset prediction network to dynamically evaluate the optimal height and width of the convolution kernel and the precise offset of sampling points in continuous space. This module uses various combinations of rectangular convolution kernels with different aspect ratios to accurately capture non-uniform fog features that are strip-shaped or irregularly distributed. The module can adaptively adjust the geometry of the sampling grid according to local transmittance changes in the input image, thereby automatically selecting the optimal spatial sampling strategy under conditions of non-uniform fog, abrupt depth changes, and complex textures, effectively improving the model's adaptability and detail reconstruction capabilities in complex and realistic fog scenes.

[0112] The discrete codebook-guided texture restoration module utilizes a pre-trained discrete feature space as a generative texture prior to address the issues of high-frequency detail loss and excessive texture smoothing in deep features under dense fog. This invention introduces a feature retrieval mechanism based on vector quantization. By calculating the distance between deep, degraded features and standard feature vectors in a high-quality codebook, the module accurately retrieves the most matching high-definition texture prototype in the feature space. Simultaneously, the module employs an adaptive affine transformation mechanism composed of convolutional networks to dynamically predict modulation parameters based on the semantic distribution of input features, performing spatial adaptive correction and fusion on the retrieved discrete features. This module transforms the dehazing task from simple pixel regression to texture feature matching and reconstruction, thereby improving the model's texture restoration capability in heavily foggy areas while maintaining the consistency of the original semantic structure.

[0113] Furthermore, in this embodiment of the application, before performing dehazing on the original image through the above-described process, the dehazing model can be iteratively trained through a similar process, and the original image after iterative training can be used for dehazing, which can improve the dehazing model's ability to dehaze the original image.

[0114] Specifically, the training process may include the following steps: Acquire both a foggy sample image and a real sample image; the real sample image can be an image acquired from the target area in a fog-free environment; while the foggy sample image can be an image acquired from the target area in a foggy environment.

[0115] Input the hazy sample image into the dehazing model to obtain the dehazed predicted image; Based on the predicted image and the sample real image, the final loss function value is obtained through a preset loss function, and the parameters of the dehazing model are updated according to the final loss function value.

[0116] Specifically, in the embodiments of this application, the final loss function obtained by the preset loss function is... It is defined as the weighted sum of pixel reconstruction loss and multi-component perceptual feature loss.

[0117] The formal representation of this preset loss function is:

[0118]

[0119] in For the dehazed image predicted by the network, For a clear and realistic reference image, and This is a balance coefficient that changes dynamically with each training round.

[0120] Define the original foggy input image (i.e., the foggy sample image mentioned above) as: To achieve a smooth transition from coarse-grained recovery to fine-grained enhancement, this module designs a linear decay and growth strategy based on the current training epoch. Let the current training epoch be... The maximum number of training rounds is Structural weights Initialized to 1.0 and linearly decaying to 0.5 during training, while detail weights... The weights then increase linearly from 0.1 to 0.5. This dynamic scheduling mechanism ensures that the model is primarily guided by pixel-level gradients in the early stages of optimization, avoiding getting trapped in local minima. As training progresses, the weights of the perceptual loss gradually dominate, forcing the model to approximate the distribution of the real image in the manifold space. The weight calculation formula is as follows:

[0121] As a fundamental constraint, pixel reconstruction loss The L1 distance is used to represent the difference in pixel values ​​between the predicted image and the sample real image. Compared to mean squared error, L1 loss is more robust in handling outliers and can effectively mitigate the smoothing blurring effect of the dehazed image. For a size of... The input tensor, and the formula for calculating the loss of this part are: in as well as These represent the dehazed image predicted by the network and the real reference image, respectively. The values ​​for the dimensions are: B is the batch size, C is the number of channels, and H and W are the feature map height and width, respectively (which are equal to the pixel height and width of the original image). They are respectively in The value of is taken; this constraint ensures the basic fidelity of the dehazing result in terms of brightness and color, laying the foundation for subsequent advanced feature recovery.

[0122] To capture high-level semantic information and texture details of images, this module integrates multi-scale perceptual loss. This loss is used to characterize the differences between the predicted image and the real sample image at different levels of feature maps. The loss utilizes a pre-trained VGG19 network as a feature extractor, extracting feature maps from five different depth levels of the network. These levels cover everything from low-level edge texture to high-level object shape semantics. Indicates the VGG network's... The feature mapping function of the layer, To assign weights to corresponding layers, this embodiment uses an increasing weight configuration of [1 / 32, 1 / 16, 1 / 8, 1 / 4, 1] to enhance the consistency of deep semantics. The perceptual loss calculation formula is as follows:

[0123] To address the edge degradation problem that is prone to occur in defogging tasks, a boundary-aware loss function is designed. This characterizes the difference in edge intensity between the predicted image and the real sample image. Gradient responses of the images are calculated separately by introducing convolution kernels. Let... and These are the convolution kernels in the width and height directions, respectively, and I is the image to be input (including...). as well as ), and These are the horizontal and vertical gradient maps obtained after convolving the input image with the corresponding convolution kernel, respectively. The loss is a constant. First, the gradient maps of the predicted image and the real image are calculated, and then the L1 distance between their edge intensity maps is calculated. This loss forces the network to focus on high-frequency boundary information, thereby generating images with sharp edges and clear contours. The specific calculation process is as follows:

[0124]

[0125]

[0126] To prevent semantic drift or false textures during the dehazing process, a semantic consistency constraint loss is introduced. This is used to characterize the distance between the predicted image and the sample hazy image in the deep feature space. The loss term calculates the distance between the predicted image and the original hazy input image. Distance in deep feature space. Although dehazing aims to remove fog, the semantic categories and spatial layout of objects in the image should remain consistent with the input. By assigning a small weight coefficient of 0.05, this constraint effectively anchors the semantic content of the image without hindering dehazing, preventing excessive background enhancement or object structure distortion. The formula is expressed as:

[0127] To further improve the structural integrity at the image feature level, the structure preservation loss is calculated. This loss is used to characterize the difference in gradient between the feature maps of the predicted image and the feature maps of the sampled real image. Instead of directly comparing the numerical values ​​of the feature maps, this loss focuses on the spatial gradient of the feature maps. By calculating the difference between the predicted feature map and the downsampled real feature map and taking the mean of their gradients, this loss constrains the consistency of high-frequency components in the feature space, complementing the perceptual loss and further suppressing mesh artifacts. Its calculation basis is as follows:

[0128] in, The operation of averaging across all spatial locations is used to characterize the operation of resizing. Resize is a downsampling operation. The above loss components are balanced in the total loss function by specific weighting coefficients, which effectively improves the image resolution capability of the model in complex hazy scenes.

[0129] In summary, after acquiring the original image, it can be input into the adaptive multi-scale convolution module of the dehazing model for processing to obtain an encoded feature map. The convolution kernels in this model have receptive fields corresponding to pixels, allowing for dynamic adjustment of the receptive field shape and sampling based on the fog concentration and scene structure in the original image, thus improving the accuracy of the encoded feature map. Then, based on the high-resolution texture feature vectors from the pre-trained discrete feature codebook, the texture restoration module in the dehazing model optimizes the encoded feature map to obtain a texture restoration feature map. Finally, the decoder processes the texture restoration feature map to obtain the dehazed target image. This approach improves the feature extraction effect for non-uniform fog spots in complex real-world scenes through the adaptive multi-scale convolution module and restores the textures smoothed out during feature extraction through the texture restoration module, thereby enhancing the dehazing effect.

[0130] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an optional embodiment of the present invention. This electronic device can be a computer device used to execute the above-described method. Figure 6As shown, the electronic device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processor can process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces).

[0131] The processor 10 may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0132] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0133] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the use of the electronic device based on the display of a mini-program landing page. Furthermore, the memory 20 may include high-speed random access memory (RAM), and may also include non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. The memory 20 may include volatile memory, such as RAM; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive; the memory 20 may also include combinations of the above types of memory.

[0134] The electronic device also includes a communication interface 30 for communicating with other devices or communication networks.

[0135] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0136] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. An image dehazing method based on adaptive convolution and texture prior, characterized in that, The method includes: Obtain the original image; The original image is input into the adaptive multi-scale convolution module in the dehazing model for processing to obtain an encoded feature map; the convolution kernel in the adaptive multi-scale convolution module has a receptive field corresponding to a pixel. Based on the discrete feature codebook obtained through pre-training, the encoded feature map is optimized by the texture restoration module in the dehazing model to obtain a texture restoration feature map; the discrete feature codebook includes multiple high-definition texture feature vectors; The texture restoration feature map is processed by the decoder in the dehazing model to obtain the dehazed target image.

2. The method according to claim 1, characterized in that, The adaptive multi-scale convolution module includes a geometric shape prediction module, a dynamic resampling module, and a feature modulation module. The original image is input into the adaptive multi-scale convolution module of the dehazing model for processing to obtain an encoded feature map, including: The original image is input into the geometric shape prediction module to obtain a height scale map and a width scale map; the height scale map and the width scale map are used to represent the receptive field expansion intensity of each coordinate in the vertical and horizontal directions, respectively; The height and width scale maps are input into the dynamic resampling module to generate an adaptive sampling grid. The original image features are then resampled and aggregated based on the adaptive sampling grid to obtain a sampled feature map. The original image is input into the feature modulation module for processing to obtain the predicted modulation coefficients and bias coefficients. The sampled feature map is modulated according to the modulation coefficient and the bias coefficient to obtain the coded feature map.

3. The method according to claim 2, characterized in that, The original image is input into the geometric shape prediction module to obtain a height scale map and a width scale map, including: The original image is used to extract features through a feature extraction network to obtain the first feature map; The height-scale map is obtained using the following formula. and width scale diagram : in Represents a high-level convolutional mapping; Represents the width convolution mapping, For the Sigmoid function, The upper limit of the preset height; This is the upper limit of the preset width; This is the first feature map.

4. The method according to claim 2, characterized in that, The step of inputting the height and width scale maps into the dynamic resampling module to generate an adaptive sampling grid, and resampling and aggregating the original image features based on the adaptive sampling grid to obtain a sampled feature map includes: The sampling grid size is determined based on the global mean of the height and width scale maps for each coordinate. Based on the height and width scale maps of each coordinate and the sampling grid size, an adaptive sampling interval is determined for each coordinate to generate an adaptive sampling grid; the adaptive sampling grid has a corresponding adaptive sampling interval for each coordinate. Based on the adaptive sampling grid, the original image features are resampled by bilinear interpolation in a continuous space to obtain the expanded resampled features; Based on the sampling grid size, a corresponding aggregation convolution kernel is selected to perform intra-block aggregation on the resampled features, restoring the original spatial resolution and obtaining the sampling feature map.

5. The method according to claim 2, characterized in that, The step of modulating the sampled feature map according to the modulation coefficient and the bias coefficient to obtain the coded feature map includes: The encoded feature map is obtained using the following formula: in, The modulation coefficient; This is the bias coefficient; This is element-wise multiplication; For sampling feature maps; This is for encoding feature maps.

6. The method according to any one of claims 1 to 5, characterized in that, The step of optimizing the encoded feature map using the texture restoration module in the dehazing model based on the pre-trained discrete feature codebook to obtain a texture restoration feature map includes: The encoded feature map is mapped from the original vector space to the feature vector space corresponding to the discrete feature codebook to obtain each vector to be processed. High-resolution texture feature vectors similar to each vector to be processed are selected from the discrete feature codebook. The encoded feature map is optimized based on high-resolution texture feature vectors similar to each vector to be processed, to obtain the texture restoration feature map.

7. The method according to claim 6, characterized in that, The step of optimizing the encoded feature map based on high-definition texture feature vectors that are similar to each vector to be processed to obtain the texture restoration feature map includes: Map the high-resolution texture feature vectors that are similar to each vector to be processed back to the original vector space to obtain a quantized feature map; The encoded feature map is input into the affine parameter prediction network to obtain the scaling factor and bias factor. The quantized feature map is processed according to the scaling factor and the bias factor to obtain a third feature map; The third feature map is fused with the encoded feature map to obtain the texture restoration feature map.

8. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain both fogged images and real images of the samples; The sample hazy image is input into the dehazing model to obtain the dehazed predicted image; Based on the predicted image and the sample real image, the final loss function value is obtained through a preset loss function, and the parameters of the dehazing model are updated according to the final loss function value.

9. The method according to claim 8, characterized in that, The preset loss function is as follows: in, To predict the image, This is the final loss function value. For the sample real image, and The balance coefficient changes dynamically with each training round; in, Pixel reconstruction loss is used to characterize the difference in pixel values ​​between the predicted image and the sample real image; Multi-scale perceptual loss is used to characterize the differences between the predicted image and the sample real image at different levels of feature maps; This is a boundary-aware loss used to characterize the difference in edge intensity between the predicted image and the real sample image; The semantic consistency constraint loss is used to characterize the distance between the predicted image and the sample foggy image in the deep feature space; The structure-preserving loss is used to characterize the difference in gradient between the feature maps of the predicted image and the feature maps of the sample real image.

10. An image dehazing system based on adaptive convolution and texture prior, characterized in that, The system includes: The raw image acquisition unit is used to acquire the raw image. The encoding unit is used to input the original image into the adaptive multi-scale convolution module for processing to obtain an encoded feature map; the convolution kernel in the adaptive multi-scale convolution module has a receptive field corresponding to a pixel. The texture restoration unit is used to optimize the encoded feature map based on the pre-trained discrete feature codebook to obtain a texture restoration feature map; the discrete feature codebook includes multiple high-resolution texture feature vectors. The decoding unit processes the texture restoration feature map through the decoder to obtain the dehazed target image.

Images