General image defogging method and system based on transform domain characteristics guidance

Abstract

A general image defogging method and system based on transform domain characteristic guidance, comprising the following steps: obtaining a to-be-processed image or video frame and extracting spatial basic features; performing frequency domain transformation on the spatial basic features, combining frequency band statistical priori to construct frequency domain enhanced features; converting the input image to HSV space, generating color perception guidance features according to saturation, brightness and their joint description quantity; obtaining multi-domain fusion features through pixel-level residual enhancement and channel-level convolution fusion, and outputting a clear image or a clear frame sequence. The present application solves the problem of insufficient collaborative modeling of existing defogging methods for global fog layer, detail blur and color offset by jointly utilizing frequency domain structure degradation characteristics and color perception degradation characteristics, and has good universality and system integration value.

Description

Technical Field

[0001] This invention relates to a general image dehazing method and system guided by transform domain characteristics, belonging to the fields of computer vision and deep learning technology. Background Technology

[0002] Image dehazing is an important research task in computer vision, aiming to recover clear scene information from degraded images affected by atmospheric scattering such as fog, smoke, and water vapor. Image dehazing technology is widely used in downstream vision tasks such as autonomous driving, intelligent monitoring, robot navigation, remote sensing, and object detection, and is of great significance for improving the perception capabilities of vision systems in complex environments. However, foggy weather causes light absorption and scattering during propagation, leading to problems such as decreased contrast, blurred details, color distortion, and information loss in acquired images, severely impacting the performance and reliability of subsequent vision algorithms.

[0003] With the development of deep learning technology, learning-based image dehazing methods have made significant progress. Compared to traditional methods based on dark channel priors, color attenuation priors, and other human-designed methods, deep learning methods can utilize large amounts of data to learn the mapping relationship between hazy and clear images, thus achieving better dehazing results. Currently, most mainstream methods are based on convolutional neural networks or Transformer networks, extracting features in the RGB space and completing image restoration, achieving good results in image detail reconstruction and visual quality improvement.

[0004] To further improve dehazing performance, researchers have begun to explore the use of frequency domain features, color space features, and physical prior information to assist image restoration. For example, Fourier transform analysis of image spectral features enhances global structure modeling capabilities using frequency domain information; or HSV color space analysis of image brightness and saturation distributions alleviates color shift issues during dehazing. These studies demonstrate that different transform domains can provide supplementary information distinct from the RGB space, playing a positive role in improving image restoration quality.

[0005] However, existing technologies still suffer from the following problems. On the one hand, most dehazing methods primarily perform feature learning in the RGB space, failing to adequately utilize the features of fog degradation in the frequency and color domains, making it difficult to fully characterize the combined degradation caused by fog on image structural and color information. On the other hand, existing frequency domain methods typically use frequency domain features only as auxiliary information, lacking a frequency band modeling mechanism specifically for fog degradation characteristics, making it difficult to effectively utilize the degradation patterns contained in different frequency regions. Furthermore, most existing color space methods rely on manually designed color priors or independently process color features, making it difficult to effectively integrate with depth features, thus limiting color recovery capabilities.

[0006] Therefore, how to fully explore the characteristic patterns of fog degradation in different transform domains and construct an image dehazing method that can jointly utilize frequency domain structural information and color space perception information to achieve synergistic optimization of image structural detail restoration and color distortion correction has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing image dehazing methods, which primarily rely on RGB space feature learning, struggle to fully utilize the characteristic patterns of fog degradation in different transform domains, and lack the ability to collaboratively model image structural degradation and color distortion. This invention creatively proposes a general image dehazing method and system guided by transform domain characteristics. By jointly mining structural degradation features in the frequency domain and perceptual degradation features in the HSV color space, this invention achieves collaborative modeling and fusion of information from multiple transform domains, improving image detail recovery, color restoration, and overall visual quality. This, in turn, enhances the accuracy, stability, and robustness of image dehazing in complex real-world scenarios.

[0008] The present invention is achieved using the following technical solution.

[0009] A general image dehazing method guided by transform domain characteristics includes the following steps:

[0010] Step 1: Obtain the image to be dehazed, construct a frequency domain statistical prior, and extract shallow spatial features.

[0011] Specifically, a training dataset containing fog images and corresponding clear images is collected, and Fourier transforms are performed on the fog images and clear images respectively, and the corresponding amplitude spectrum information is extracted.

[0012] Furthermore, the average amplitude spectrum of all foggy images and clear images is calculated separately:

[0013]

[0014]

[0015] Where N represents the number of training samples; and These represent the average amplitude spectra of all foggy and clear images, respectively. and These represent the frequency coordinates of the spectrum in the horizontal and vertical directions, respectively. and Let represent the amplitude spectra corresponding to the (i)th foggy image and the clear image, respectively.

[0016] Construct a frequency domain statistical prior plot based on the differences between the average amplitude spectra:

[0017]

[0018] in, This represents a frequency domain statistical prior plot, used to characterize the degree of influence of fog degradation in different frequency regions.

[0019] Furthermore, using the center of the spectrum as the center, the frequency domain statistical prior plot is divided into frequency bands according to the frequency radius, and the frequency radius is defined as:

[0020]

[0021] in, This represents the coordinates of the spectrum center, where r represents the distance from the current frequency point to the spectrum center.

[0022] Generate a corresponding frequency band mask based on a preset frequency range, and construct a frequency domain prior set by combining it with a frequency domain statistical prior map, which will be used in the subsequent frequency domain degradation feature modeling process.

[0023] Input the image to be dehazed into the feature extraction module to obtain shallow spatial features:

[0024]

[0025] Where I represents the input image to be dehazed. This represents the shallow feature extraction process. This represents the extracted shallow spatial features.

[0026] Step 2: Perform frequency domain transformation on the shallow spatial features, and extract frequency domain guided features by combining the pre-constructed frequency domain statistical prior.

[0027] Specifically, a two-dimensional Fourier transform is performed on the shallow spatial features to obtain the corresponding frequency domain representation:

[0028]

[0029] in, This indicates the Fourier transform operation. It represents the frequency domain characteristics.

[0030] Furthermore, the frequency domain features are decomposed into amplitude spectrum and phase spectrum:

[0031]

[0032] in, Indicates amplitude spectrum, Represents the phase spectrum. It is the imaginary unit.

[0033] Since fog degradation is mainly manifested in changes in the amplitude spectrum, the frequency domain statistical prior map constructed in step 1 is used to guide the modeling of the amplitude spectrum. Specifically, the amplitude spectrum is fused with the frequency domain statistical prior map to obtain the frequency domain guided features:

[0034]

[0035] in, This represents the frequency domain statistical prior plot. This indicates a feature splicing operation.

[0036] Subsequently, frequency domain degradation modeling is performed on the frequency domain guiding features to obtain the enhanced frequency domain features:

[0037]

[0038] in, This represents the frequency domain feature coding process. This represents the enhanced frequency domain characteristics.

[0039] Furthermore, to preserve the original spectral structure information, the enhanced frequency domain features are residually fused with the original frequency domain features:

[0040]

[0041] Finally, the fused frequency domain features and the original phase spectrum are subjected to an inverse Fourier transform to restore the spatial domain:

[0042]

[0043] in, This indicates the inverse Fourier transform operation. This indicates frequency domain degradation characteristics.

[0044] By introducing frequency domain statistical prior maps to guide the modeling of the amplitude spectrum, this step can enhance the network's ability to represent the spectral features of fog degradation, improve the image's global contrast recovery ability, and enhance its ability to recover structural details.

[0045] Step 3: Convert the image to be dehazed to the HSV color space, construct a fog perception prior based on saturation and brightness information, and extract color perception features.

[0046] Specifically, the input image to be dehazed is converted from the RGB color space to the HSV color space:

[0047]

[0048] in, This indicates the input image to be dehazed; This indicates a conversion operation from RGB to HSV color space; , and These represent the hue channel, saturation channel, and luminance channel, respectively.

[0049] Since fog degradation typically manifests as decreased image saturation and increased brightness, saturation and brightness information are used to construct a fog perception prior. Specifically, saturation features, brightness features, and joint perception features are defined:

[0050]

[0051]

[0052] in, Indicates saturation characteristics; Indicates brightness characteristics; This represents a fog perception feature constructed jointly by brightness and saturation, used to enhance the response capability of high-brightness, low-saturation regions.

[0053] Furthermore, the above features are concatenated to construct an HSV fog perception prior:

[0054]

[0055] in, This represents the prior features of HSV fog perception.

[0056] Subsequently, spatial feature encoding and channel feature encoding are performed on the HSV fog perception prior to obtain spatial perception features and channel perception features:

[0057]

[0058]

[0059] in This represents the spatial feature encoding process; This represents the channel feature encoding process; Indicates spatial perception characteristics; This indicates channel-sensing characteristics.

[0060] Furthermore, spatial perception features and channel perception features are fused to obtain color perception features:

[0061]

[0062] in, This represents the characteristics of color perception.

[0063] By constructing an HSV fog perception prior based on saturation and brightness information, this step can enhance the network's ability to represent fog-degraded color features and improve the accuracy and naturalness of color restoration during the defogging process.

[0064] Step 4: Enhance the spatial features using color perception features and fuse them with frequency domain degradation features to obtain multi-transform domain fusion features, and output the dehazing result.

[0065] Specifically, the color perception features obtained in step 3 are introduced into the shallow spatial features obtained in step 2 to enhance the spatial features:

[0066]

[0067] in, Indicates shallow spatial characteristics; Indicates color perception characteristics; The moderating coefficient representing the color perception characteristics; This indicates the enhanced spatial features.

[0068] Furthermore, the enhanced spatial features are fused with the frequency domain degradation features obtained in step 2:

[0069]

[0070] in, Indicates frequency domain degradation characteristics; Indicates feature concatenation operation; This indicates the combined characteristics after fusion.

[0071] Subsequently, feature mapping is performed on the joint features to obtain multi-transform domain fused features:

[0072]

[0073] in, This represents the feature fusion process; This represents the multi-transform domain fusion feature.

[0074] Through the above fusion process, structural degradation information in the frequency domain, color perception information in the HSV color space, and local texture information in the spatial domain are collaboratively modeled to achieve complementary fusion of features from multiple transform domains, providing richer degradation characterization information for subsequent image restoration processes.

[0075] Furthermore, the multi-transform domain fusion feature is input into the image restoration module for feature reconstruction to obtain the final dehazing result.

[0076]

[0077] in, This indicates the image restoration process. This represents the output dehazed image.

[0078] By reconstructing the fused features through the image restoration module, we can make full use of the structural degradation information in the frequency domain, the color perception information in the HSV color space, and the texture detail information in the spatial domain to achieve synergistic optimization of image structure restoration and color correction, and obtain clear and natural dehazing results.

[0079] Specifically, to objectively evaluate the dehazing effect, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM) can be used as evaluation metrics. PSNR measures the pixel-level error between the restored image and the reference sharp image, while SSIM measures the structural similarity between the restored image and the reference sharp image.

[0080] Based on the above method, the present invention further proposes a general image dehazing method and system guided by transform domain characteristics, including: an image input and spatial representation subsystem 10, a frequency band prior modeling subsystem 20, a saturation and brightness guidance subsystem 30, and a multi-domain fusion and dehazing output subsystem 40.

[0081] The connections between the above components are as follows:

[0082] The image input and spatial representation subsystem 10 receives the image to be processed and outputs basic spatial features; the frequency band prior modeling subsystem 20 receives the basic spatial features and outputs frequency domain enhancement features; the saturation and brightness guidance subsystem 30 receives the image to be processed and the basic spatial features and outputs color perception guidance features; the multi-domain fusion and dehazing output subsystem 40 receives the basic spatial features, frequency domain enhancement features, and color perception guidance features, and completes pixel-level residual enhancement, channel stitching, convolutional fusion, and clear image reconstruction.

[0083] Beneficial effects

[0084] Compared with the prior art, the present invention has the following advantages:

[0085] 1. This invention introduces frequency band statistical priors on spatial basic features and uses the low-frequency and mid-frequency degradation rules in the amplitude spectrum to guide feature enhancement, which helps to alleviate the problems of global fog residue and structural blur, and makes the restoration of image contrast, boundary contours and detail texture more stable.

[0086] 2. This invention constructs a color perception prior by using saturation, brightness and their joint descriptive quantities, and extracts haze-sensitive color information by using a dual-path encoding method of spatial and channel, which can better distinguish between real color changes and haze shifts, and is conducive to improving the color naturalness and brightness consistency of dehazing results.

[0087] 3. This invention employs pixel-level residual injection and channel-level convolution fusion to combine spatial, frequency, and color transformation domain features. The structure is clear and easy to embed into existing dehazing networks or video enhancement systems, which can improve the versatility, robustness, and deployment adaptability in complex scenarios. Attached Figure Description

[0088] Figure 1 This is a flowchart of the method.

[0089] Figure 2 This is a schematic diagram of the system composition of the present invention. Detailed Implementation

[0090] To better illustrate the purpose and advantages of the present invention, the method of the invention will be further described below in conjunction with the accompanying drawings and examples.

[0091] Example

[0092] like Figure 1 As shown, a general image dehazing method and system based on transform domain characteristics includes the following steps:

[0093] Step 1: Input the image or video frame to be processed, construct a frequency band prior, and extract basic spatial features. Specifically, input the fogged image into a spatial backbone network composed of convolutional layers and feature encoding layers to obtain shallow spatial features. Where C, H, and W represent the number of channels, height, and width, respectively. This feature is simultaneously fed into the frequency band prior modeling path and the multi-domain fusion path, and provides a spatial reference for the subsequent alignment of the color-guided feature.

[0094] Step 2: Generate frequency domain enhancement features. First, for... A two-dimensional Fourier transform is performed, and the frequency domain features are decomposed into amplitude spectrum features and phase spectrum features. Since fogging degradation mainly affects a portion of the amplitude spectrum, this example constructs a frequency band prior map based on the statistical differences in the amplitude spectra of fogged and clear samples. Different frequency range masks are then used to represent the intensity of degradation. The following steps are then performed:

[0095]

[0096] in, This indicates frequency domain guidance characteristics. This represents the amplitude spectrum characteristics obtained after Fourier transforming shallow spatial features. This indicates channel splicing.

[0097] Subsequently, the frequency-domain guided features are input into the frequency-domain encoder for feature enhancement, and residual connections are used to preserve the original spectral structure information. Finally, inverse Fourier transform is used to recover the spatial domain, and frequency-domain enhanced features are obtained through convolutional mapping. This step improves the network's ability to recover global structural information and detailed texture information by explicitly modeling the impact of fog degradation on different frequency bands.

[0098] Step 3: Construct saturation and brightness guiding features. Convert the input image I from RGB space to HSV space and extract saturation and brightness channel information. Since hazy areas typically exhibit increased brightness and decreased saturation, to highlight the common high-brightness, low-saturation characteristics of hazy areas, this example constructs an HSV color prior using saturation, brightness, and their joint descriptor.

[0099]

[0100] Where S represents the saturation channel and V represents the brightness channel. This indicates channel splicing.

[0101] Subsequently, spatial weight encoders and channel weight encoders are used to model the HSV color prior, extracting spatial and channel-aware information, which are then fused to obtain color-aware guided features. This step enhances the network's ability to represent color degradation features in fogged areas, improving the color naturalness and brightness consistency of the restored image.

[0102] Step 4: Perform pixel-level residual enhancement, multi-domain feature fusion, and dehazing output. Inject color-perceptual guided features into the spatial base features:

[0103]

[0104] in, This is a learnable scaling parameter used to control the contribution of color-guided features to the spatial representation. Representing the basic characteristics of space, Indicates color-guided features. Color domain enhancement features are then added. These color domain enhancement features are subsequently fused with frequency domain enhancement features.

[0105]

[0106] in, Indicates a convolutional fusion layer. This involves enhancing the frequency domain features. The fused features contain three complementary types of information: spatial details, frequency domain structure, and color perception. These are then input into the dehazing reconstruction network to output a clear image. For video frame sequences, the above processing can be performed on each frame and combined with the temporal information in the network to maintain visual continuity in the restoration results of adjacent frames.

[0107] Among them, a general image dehazing method and system based on transform domain characteristics in this application solves the problem that structural degradation and color shift are difficult to restore simultaneously in complex foggy scenes by using frequency band prior modeling, saturation and brightness guidance, pixel-level residual enhancement and multi-domain convolution fusion, effectively improving the clarity, naturalness and stability of the dehazing results.

[0108] Figure 1 The schematic diagram of the system composition of a general image dehazing method and system based on transform domain characteristics provided in this application example includes an image input and spatial representation subsystem 10, a frequency band prior modeling subsystem 20, a saturation and brightness guidance subsystem 30, and a multi-domain fusion and dehazing output subsystem 40.

[0109] The image input and spatial representation subsystem 10 is used to receive the image or video frame to be dehazed, complete the extraction of spatial backbone features, and output the basic spatial features to the subsequent subsystems.

[0110] The frequency band prior modeling subsystem 20 is used to perform Fourier transform, amplitude spectrum decomposition, frequency band prior splicing, frequency domain coding, residual enhancement and inverse transform on the spatial basic features, and output frequency domain enhanced features.

[0111] The saturation and brightness guidance subsystem 30 is used to convert the input image to the HSV space, construct saturation, brightness and brightness-saturation joint descriptors, and obtain color perception guidance features through dual-path encoding.

[0112] The multi-domain fusion and dehazing output subsystem 40 is used to receive spatial basic features, frequency domain enhancement features and color perception guidance features, perform pixel-level residual injection, channel stitching and convolutional fusion, and output clear images or clear frame sequences through the dehazing reconstruction network.

[0113] The connection relationships between the above-mentioned components are as follows: the image input and spatial representation subsystem 10 is connected to the frequency band prior modeling subsystem 20, the saturation and brightness guidance subsystem 30, and the multi-domain fusion and dehazing output subsystem 40, respectively; the frequency band prior modeling subsystem 20 and the saturation and brightness guidance subsystem 30 are both connected to the multi-domain fusion and dehazing output subsystem 40; the multi-domain fusion and dehazing output subsystem 40 outputs the dehazing result as the final output terminal.

[0114] The above detailed description further illustrates the purpose, technical solution, and beneficial effects of the invention. It should be understood that the above description is merely a specific example of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A general image dehazing method and system guided by transform domain characteristics, characterized in that, Includes the following steps: Step 1: Obtain the image to be dehazed, construct a frequency domain statistical prior, and extract shallow spatial features; Step 2: Perform frequency domain transformation on the shallow spatial features, and extract frequency domain guided features by combining the pre-constructed frequency domain statistical priors; Step 3: Convert the image to be dehazed to the HSV color space, construct a fog perception prior based on saturation and brightness information, and extract color perception features; Step 4: Enhance spatial features using color perception features and fuse them with frequency domain degradation features to obtain multi-transform domain fusion features, generating a clear image or clear frame sequence after dehazing.

2. The method as described in claim 1, characterized in that, The acquisition of basic spatial features in step 1 specifically includes: The image to be processed is input into a spatial backbone network consisting of convolutional layers and feature encoding layers to obtain basic spatial features. The spatial basic features are simultaneously provided to the frequency band prior modeling process, the color-guided modeling process, and the multi-domain fusion process.

3. The method as described in claim 1, characterized in that, The construction of frequency domain enhancement features in step 2 specifically includes: Basic spatial features Performing a two-dimensional Fourier transform yields the frequency domain features. ,Will Decomposed into amplitude spectrum features and phase spectrum characteristics And a frequency band prior map is generated based on the statistical difference in amplitude spectrum between fogged and clear samples. .

4. The method as described in claim 3, characterized in that, The frequency band prior map mentioned in step 2 The degradation intensity of haze at different frequency scales is represented by masks covering multiple frequency bands. and After channel splicing, the signal is input into the frequency domain encoder, and then subjected to residual enhancement, inverse Fourier transform, and convolutional projection to obtain frequency domain enhanced features. .

5. The method as described in claim 1, characterized in that, Step 3, which involves constructing color-perception guided features, specifically includes: The image to be processed is converted from RGB space to HSV space to obtain saturation channel S and luminance channel V, and then constructed. , , and will , and Channel splicing is performed to form HSV color prior.

6. The method as described in claim 5, characterized in that, In step 3, the HSV color prior is input into the spatial weight encoder and the channel weight encoder respectively to obtain spatial distribution features and channel relationship features, and the two are added together to form the color perception guidance feature. .

7. The method as described in claim 1, characterized in that, The fusion described in step 4 specifically includes: pass Color perception-guided features are injected into the spatial base features at a learnable scale, whereby... Learnable scaling parameters; then through Color enhancement spatial features and frequency domain enhancement features are channel-stitched and convolutionally fused, and the fusion result is input into the dehazing reconstruction network.

8. A general image dehazing system guided by transform domain characteristics, characterized in that, include: Image Input and Spatial Representation Subsystem 10: Used to receive images or video frames to be processed, extract basic spatial features, and provide spatial representations to subsequent subsystems; Frequency band prior modeling subsystem 20: used to perform frequency domain transformation, amplitude spectrum decomposition, frequency band prior construction, frequency domain coding, residual enhancement and inverse transformation on spatial basic features, and output frequency domain enhanced features; Saturation and brightness guidance subsystem 30: used to convert the input image to HSV space, construct saturation, brightness and brightness-saturation joint descriptors, and generate color perception guidance features; Multi-domain fusion and dehazing output subsystem 40: used to perform pixel-level residual enhancement and convolutional fusion of spatial basic features, frequency domain enhancement features and color perception guided features to output clear images or clear frame sequences.

Images