Transform-based efficient image defogging method

Abstract

The invention discloses an efficient image defogging method based on Transform. The efficient image defogging method is characterized by comprising the following steps: acquiring an image to be defogged; constructing a defogging model based on a Transform, wherein the defogging model is used for defogging the image; training the defogging model to obtain a trained defogging model; and inputting a to-be-defogged image into the trained defogging model to realize defogging of the image. According to the method, modeling is carried out in parallel through features of different scales and different receptive fields, the adaptive capacity of the network to a multi-scale fog structure and a complex scene is enhanced, the model can give consideration to remote global dependence and local detail information at the same time, and therefore the overall definition and structural consistency of a defogged image are improved.

Description

Technical Field

[0001] This invention belongs to the field of fully automated products and relates to an efficient image dehazing method based on Transformer. Background Technology

[0002] Current image dehazing techniques mainly include traditional prior-based methods, deep learning methods based on convolutional neural networks (CNNs), and novel end-to-end algorithms based on Transformers. Although these methods have made some progress, they still have many technical bottlenecks and shortcomings, and cannot fully meet the requirements for high-quality, high-speed dehazing in complex real-world scenes.

[0003] (I) Traditional Prior-Based Image Dehazing Methods Traditional dehazing methods are typically based on physical models or statistical priors, including dark channel prior (DCP), color attenuation prior (CAP), and maximum contrast prior (MIP). While these methods are computationally simple, they generally suffer from the following problems: 1. The prior assumptions are too idealistic and have poor adaptability to real-world scenarios. For example, the prior assumption of the dark channel, "most local areas in a fog-free image have at least one color channel with a very low value," will fail severely in the following scenarios: colored lights and backlighting scenes cause abnormal brightness distribution; nighttime scenes have complex lighting, and the prior conditions cannot be met. 2. Poor performance in high-density fog environments. In complex scenarios such as extremely dense fog, the transmittance is difficult to estimate accurately, resulting in incomplete defogging and extremely unnatural colors.

[0004] 3. It cannot adapt to different scenarios and struggles to handle various fog distributions. Due to its lack of learning ability, it cannot adaptively adjust to changes in the environment, making it difficult to handle non-uniform fog, multi-scale fog, etc.

[0005] (II) Deficiencies of CNN-based Deep Learning Dehazing Algorithms In recent years, a large number of end-to-end dehazing models based on convolutional neural networks have emerged, such as AOD-Net, GCANet, FFA-Net, and MSBDN. These methods have achieved good results by learning from large-scale datasets, but they still have the following core problems: 1. Heavy reliance on synthetic data and weak generalization ability to real-world scenes. Most models are trained on synthetic datasets such as RESIDE, which lack sufficient understanding of real fog structures, lighting models, and sensor noise distribution, resulting in uneven brightness and obvious fog residue after defogging.

[0006] 2. Insufficient detail recovery and texture smoothing. Due to the limited receptive field of CNN convolution kernels, it is difficult to simultaneously handle large-scale fog distribution and fine-grained texture. Therefore, during dehazing, edge blurring, difficulty in recovering high-frequency details, and overly smooth textures are common problems.

[0007] 3. Multi-scale structures are complex, resulting in slow inference speed and high computational cost. Models such as MSBDN and FFA-Net often use multi-branch, multi-scale pyramid structures to improve performance, but they suffer from drawbacks such as large model size and the need for a large amount of computational resources.

[0008] (III) Deficiencies of Transformer-based dehazing algorithms In recent years, methods like DehazeFormer, TransWeather, and Restormer have increasingly adopted the Transformer self-attention structure to improve dehazing quality through global modeling. However, the following important technical issues still exist: 1. High computational complexity and low efficiency in high-resolution image processing. The computational cost of the standard self-attention mechanism increases quadratically with image resolution, resulting in extremely slow dehazing during inference on 720P and 1080P images, and making it difficult to deploy on mobile devices.

[0009] 2. There are shortcomings in texture detail restoration. Since Transformer is better at global relation modeling, the lack of local convolution or multi-scale modules often leads to insufficient image texture.

[0010] While existing image dehazing techniques have achieved some success, they still generally suffer from the following shortcomings and defects, particularly in terms of image detail restoration, generalization ability to real-world scenes, and model inference speed under high-resolution conditions, where significant bottlenecks exist: (1) The defogging effect has insufficient generalization ability in real-world scenarios. Existing advanced dehazing methods, including FFA-Net and MSBDN based on multi-scale convolutional networks, and TransWeather and DehazeFormer based on Transformers, generally rely on a large amount of synthetic fog data for training. Their models perform well on standard synthetic datasets, but they are prone to problems such as color shift, insufficient restoration of edge and texture details, obvious residual fog, and incomplete dehazing in real complex fog environments (such as non-uniform fog, strong fog, backlight, night scenes, etc.). These problems lead to insufficient reliability of the models in real-world application scenarios, and they cannot meet the high requirements of scenarios such as monitoring, autonomous driving, and aerial remote sensing.

[0011] (2) In high-resolution scenarios, the model inference speed is slow and the computation and memory overhead is high. Most existing state-of-the-art (SOTA) dehazing algorithms are complex in structure and computationally intensive. In particular, Transformer-type models are prone to high self-attention computational complexity in high-resolution images (such as 720p and 1080p), which can lead to a significant increase in inference latency. Multi-scale convolution / fusion structures also generate a large amount of memory access overhead during the inference stage. In scenarios such as autonomous driving and high-definition video surveillance, these high latency and high resource consumption issues cannot meet the engineering deployment requirements. Summary of the Invention

[0012] To address the core technical problems of traditional dehazing algorithms, such as insufficient generalization ability in real-world scenes, slow inference speed, and high computational overhead, this invention adopts the following technical solution: an efficient image dehazing method based on Transformer, comprising the following steps: Obtain the image to be dehazed; Construct a Transformer-based dehazing model for image dehazing; The dehazing model is trained to obtain a well-trained dehazing model; The image to be dehazed is input into a trained dehazing model to achieve image dehazing.

[0013] Furthermore: the defogging model includes: Shallow feature extraction module: used to extract shallow features from dehazed images; Deep feature extraction module: adopts an encoder-decoder structure, and performs deep feature extraction based on the shallow feature extraction module. Channel reduction module: Used to reduce the number of channels to 3 based on the extracted deep features transmitted by the deep feature extraction module.

[0014] Furthermore: the encoder-decoder structure includes: an encoder and a decoder; The encoder includes three residual blocks—a first residual block, a second residual block, and a third residual block—that are sequentially connected by a splicing operation. The first residual block, the second residual block, and the third residual block have the same structure; The first residual block includes: First-scale perceptual embedding module: used to extract image features using depthwise separable deformable convolution, which can generate local feature maps of different shapes and encode multi-scale pixel value sequences; First multi-scale Transformer block: Used to fuse features from different branches to obtain multi-scale image features based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module. Taylor approximate attention mechanism is introduced, and features from different branches are fused under the linear attention mechanism based on Taylor expansion to obtain multi-scale image features. The encoder includes a fourth residual block, a fifth residual block, and a sixth residual block; The fourth residual block, the fifth residual block, and the sixth residual block have the same structure; The fourth, fifth, and sixth residual blocks are connected in the dimension by connecting multiple tensor graphs. The fourth residual block includes: The fourth multi-scale Transformer block introduces the Taylor approximate attention mechanism, which fuses features from different branches under the linear attention mechanism based on Taylor expansion to obtain multi-scale image features; Fourth-scale perceptual embedding module: Based on the multi-scale image features transmitted by the fourth multi-scale Transformer block, it generates local feature maps of different shapes and encodes them to generate multi-scale pixel value sequences.

[0015] Furthermore: the fourth multi-scale Transformer block includes: First branch: Second branch, Third branch, and Selective kernel feature fusion module: The outputs of the first branch, the second branch, and the third branch are connected to the input of the selective kernel feature fusion module; The first branch includes: a first DCN module, a first HARDWISH activation function, and a first FL-MSA module; The first DCN module, the first HARDWISH activation function, and the first FL-MSA module are connected sequentially. The second branch includes: a second DCN module, a second HARDWISH activation function, and a second FL-MSA module; The first HARDWISH activation function, the second DCN module, the second HARDWISH activation function, and the second FL-MSA module are connected in sequence. The third branch includes: a third DCN module, a third HARDWISH activation function, and a third FL-MSA module; The second HARDWISH activation function, the third DCN module, the third HARDWISH activation function, and the third FL-MSA module are connected in sequence.

[0016] Furthermore: The first scale-aware embedding module is used to extract image features using depthwise separable deformable convolution, generating local feature maps of different shapes and encoding a multi-scale pixel value sequence as follows: S31: Input Features ; S32: Multi-branch parallel execution of deformable convolution operations; for a certain branchs spatial point p The standard deformable convolution formula at the given location is shown in the following equation.

[0017] in: This indicates the width of the convolution kernel in that branch; This represents the relative coordinates of the Kth sampling position at the center of the regular convolution; The offset is predicted by the subnetwork, as shown in the following formula:

[0018] in, ; These are the weights of the convolution kernel. Sampling points are typically real-number coordinates. This indicates bilinear interpolation calculation; The standard deformable convolution is decomposed into two operations: DepthWise and PointWise. The DepthWise operation performs independent deformable convolution sampling on each channel, while the PointWise operation performs fusion. The two operations are shown in the following equations:

[0019] in: For DepthWise convolution in channels c The k Individual core weights;

[0020] in: The weights for the 1×1 convolution kernel are from the input channel. c Mapped to output channel .

[0021] Further: The first multi-scale Transformer block: the process of fusing features from different branches based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module to obtain multi-scale image features is as follows: Assume the input query vector Query, key vector Key, and value vector Value are of size . Given the feature map, the standard attention calculation formula is as follows:

[0022] Rewrite formula (2-1) in a more general form, as shown in formula (2-2):

[0023] in: Representing the first of the Query matrix Line, function Represents the similarity function, when for When this happens, equation (2-2) degenerates into equation (2-1); according to the Taylor expansion, exist The location can be expanded into Substituting this expansion into formula (2-2) yields formula (2-3):

[0024] right and conduct After normalization and setting the normalization norm to 0.5, we obtain... and The normalization norm is set to 0.5, so and The dot product result is in the range [-0.25, 0.25]. When within range, and The result is approximate, when using Alternative Then, equation (2-3) is transformed into equation (2-4):

[0025] Finally, equation (2-4) is rewritten using the associative law of matrix multiplication, resulting in equation (2-5):

[0026] Where: N represents the total number of tokens, j is the "index of the token being followed", representing the traversal index of all key-value tokens; for a size of The feature map, the computational complexity of the original multi-head attention mechanism is shown in equation (2-6).

[0027] The computational complexity of the proposed fast linear multi-head attention mechanism is shown in Equation (2-7):

[0028] In most cases, The product is much greater than The value of the fast linear multi-head attention mechanism reduces the computational complexity from Reduced to .

[0029] A high-efficiency image dehazing device based on Transformer, comprising: Acquisition module: Used to acquire the image to be dehazed; Build modules: Used to build Transformer-based dehazing models for dehazing images; Training module: Used to train the dehazing model to obtain a trained dehazing model; Implementation module: This module is used to input the image to be dehazed into the trained dehazing model to achieve image dehazing.

[0030] A readable storage medium storing a program module, characterized in that the program module, when run in a processor, can implement the method as described in any one of the above.

[0031] This invention provides an efficient image dehazing method based on Transformer, offering a high-quality, high-speed image dehazing approach suitable for high-resolution real-time applications. It proposes a Transformer-based dehazing network (MBTaylorFormer) that balances image dehazing performance with computational overhead. This network features a multi-scale branch embedding structure, enabling the simultaneous generation of multi-scale feature maps to enhance image dehazing. Furthermore, it optimizes existing attention mechanisms by using a linearized self-attention mechanism to optimize model parameters and computational cost.

[0032] To address the problems of high computational complexity, poor adaptability to high-resolution images, and insufficient restoration of details in fogged areas in existing Transformer dehazing methods, several structural improvements were proposed and significant technical effects were achieved.

[0033] First, this invention introduces the Taylor approximate attention mechanism, reducing the quadratic computational complexity of traditional softmax-attention to linear complexity. This effectively reduces the computational load and memory usage of the model on high-resolution images, significantly improving inference efficiency while maintaining global modeling capabilities. Experimental results show that, at the same input resolution, the computational cost (FLOPs) and inference time of this model are significantly lower than those of the traditional Transformer structure.

[0034] Secondly, this invention designs a multi-branch Transformer architecture and a perceptual embedding module. By modeling features at different scales and receptive fields in parallel, it enhances the network's adaptability to multi-scale fog structures and complex scenes, enabling the model to simultaneously take into account both long-distance global dependencies and local detail information, thereby improving the overall clarity and structural consistency of dehazed images.

[0035] Experiments show that the present invention achieves higher PSNR and SSIM scores on multiple publicly available dehazing datasets, outperforming existing mainstream methods in dehazing performance. Furthermore, the present model demonstrates greater stability in terms of fog edge, texture detail, and brightness recovery. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of the method; Figure 2 This is a schematic diagram of a scale-aware embedding module, where (a) represents a scene with an offset of 3 in the deformable convolution, (b) represents a scene with an offset of -3 in the deformable convolution, and (c) represents a scene with an offset of -3 in the deformable convolution. Figure 3 The figures show the comparative experimental results, where (a) is the input, (b) is the FAA-net network, (c) is the SGID-PFF network, (d) is the Restormer network, (e) is the method of this application, and (f) is the GT network. Detailed Implementation

[0038] It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0039] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] Figure 1 This is a flowchart of the method; An efficient image dehazing method based on Transformer includes the following steps: S1: Obtain the image to be dehazed; S2: Construct a Transformer-based dehazing model for image dehazing; S3: Train the dehazing model to obtain a trained dehazing model; S4: Input the image to be dehazed into the trained dehazing model to achieve image dehazing.

[0041] Steps S1 / S2 / S3 / S4 are executed sequentially; Furthermore, the defogging model includes: Shallow feature extraction module: used to perform shallow feature extraction on the dehazed image; the shallow feature extraction module uses a 3×3 convolutional layer for an input hazy image. ,get ; Deep feature extraction module: adopts an encoder-decoder structure, and performs deep feature extraction based on the shallow feature extraction module. Channel reduction module: Used to reduce the number of channels to 3 based on the extracted deep features transmitted by the deep feature extraction module.

[0042] The channel reduction module uses a 3×3 convolutional layer to reduce the number of channels to 3, resulting in... The final dehazed image is This network combines the hierarchical extraction capabilities of U-Net with the dependency modeling capabilities of Transformer over long distances between pixels, thereby reducing overall computational overhead and the number of parameters while maintaining strong feature representation capabilities.

[0043] The encoder-decoder structure includes: an encoder and a decoder; The encoder includes three residual blocks—a first residual block, a second residual block, and a third residual block—connected sequentially by a Concat operation. The meaning of the Concat operation: Concatenate means to join together. This operation refers to directly connecting multiple tensors together along a specified dimension to form a new tensor, without performing any numerical calculations or weighted fusion. The first residual block, the second residual block, and the third residual block have the same structure; The first residual block includes: The first-level scale-aware embedding module (SAEM) is used for image feature extraction using depthwise separable deformable convolutions. It generates local feature maps of different shapes and encodes multi-scale pixel value sequences. This module introduces different receptive field scales: small-scale branches focus on details, while large-scale branches focus on the global fog structure. Using deformable sampling makes it more sensitive to irregular fog boundaries and regions with abrupt changes in brightness. This module provides subsequent Transformer modules with multi-scale information from the outset, improving robustness to complex fog scenes.

[0044] The Patch Embedding module employs a multi-branch parallel structure, with each branch consisting of a set of deformable convolutions (DCNs) and non-linear activation functions. Different branches use convolution kernel parameters of different scales, thus forming feature extraction paths with different receptive field sizes. Within each branch, deformable convolutions are used to spatially sample the input features, allowing the sampling location to adaptively adjust according to the image content to better fit the spatial distribution characteristics of the fog area.

[0045] Figure 2 This is a schematic diagram of a scale-aware embedding module, where (a) represents a scene with an offset of 3 in the deformable convolution, (b) represents a scene with an offset of -3 in the deformable convolution, and (c) represents a scene with an offset of -3 in the deformable convolution. The first multi-scale Transformer Module (MSTM) is used to fuse features from different branches based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module. It introduces the Taylor approximate attention mechanism and the linear attention mechanism based on Taylor expansion to obtain multi-scale image features. Technical solution that introduces Taylor's approximate attention mechanism The core idea of ​​this technical solution is to perform a low-order Taylor expansion of the exponential function in softmax attention, approximating the exponential operation with a polynomial form, thereby avoiding the explicit construction of QK^ The fully connected attention matrix. Specifically, after normalizing the Query and Key features to ensure numerical stability, exp(Q_i^ K_j) is approximated as the sum of constant and linear terms, making the calculation of attention weights decomposed into a summation of several global statistics independent of the query.

[0046] The role of the multi-scale Transformer module is to address the limitations of the single-path attention mechanism in traditional single-branch Transformers for dehazing tasks, where tokens of different scales are forced to share the same attention modeling method. Under the linear attention mechanism based on Taylor expansion used in this paper, the expressive power of the model is limited, making it necessary to use the multi-scale Transformer module for structural compensation.

[0047] The encoder includes a fourth residual block, a fifth residual block, and a sixth residual block; The fourth residual block, the fifth residual block, and the sixth residual block have the same structure; The fourth, fifth, and sixth residual blocks are connected in the dimension by connecting multiple tensor graphs. The fourth residual block includes The fourth multi-scale Transformer block introduces the Taylor approximate attention mechanism, which fuses features from different branches under the linear attention mechanism based on Taylor expansion to obtain multi-scale image features; Fourth-scale perceptual embedding module: Based on the multi-scale image features transmitted by the fourth multi-scale Transformer block, it generates local feature maps of different shapes and encodes them to generate multi-scale pixel value sequences.

[0048] Furthermore, the fourth multi-scale Transformer block includes: The first branch: Its main function is to capture local details and restore edge textures and fine structures; The second branch is mainly used to model the structure of objects and their local context, connecting details and global information. The third branch provides a larger receptive field, primarily used for modeling large-scale fog distributions and capturing global brightness and low-frequency information.

[0049] The function of the selective kernel feature fusion module is: ①Features extracted from branches at different scales may be redundant or even conflicting; the SKFF module reduces conflicts and improves fusion quality through weight selection; ② This model uses a linear attention mechanism based on Taylor expansion. The expressive power of the linear attention mechanism is weaker than that of the original attention mechanism. By using a multi-branch + SKFF structure, the overall expressive power is enhanced, which has a compensatory effect.

[0050] The outputs of the first branch, the second branch, and the third branch are connected to the input of the Selective Kernel Feature Fusion (SKFF) module. The first branch includes: a first DCN module, a first HARDWISH activation function, and a first FL-MSA module; The DCN module is used to enhance the network's ability to model multi-scale, irregular fog structures. The FL-MSA module enables global modeling with linear computational complexity, allowing Transformer to efficiently process high-resolution dehazed images.

[0051] The purpose of the HardWish activation function is to improve training stability and reduce computational overhead while ensuring non-linear expressive power.

[0052] The first DCN module, the first HARDWISH activation function, and the first FL-MSA module are connected sequentially. The second branch includes: a second DCN module, a second HARDWISH activation function, and a second FL-MSA module; The first HARDWISH activation function, the second DCN module, the second HARDWISH activation function, and the second FL-MSA module are connected in sequence. The third branch includes: a third DCN module, a third HARDWISH activation function, and a third FL-MSA module; The second HARDWISH activation function, the third DCN module, the third HARDWISH activation function, and the third FL-MSA module are connected in sequence.

[0053] Furthermore, the scale-aware module uses depth-separable deformable convolutions for image feature extraction. This module can generate local feature maps of different shapes and encode them into tokens, avoiding the shortcomings of insufficient modeling of fog scale and shape when using convolution kernels of fixed size to extract feature maps.

[0054] The first scale-aware embedding module is used to extract image features using depthwise separable deformable convolution. The process of generating local feature maps of different shapes and encoding multi-scale pixel value sequences is as follows: S31: Input Features ; S32: Multi-branch parallel execution of deformable convolution operations; for a certain branch s spatial point p The standard deformable convolution formula at the given location is shown in the following equation. (1-1) in: This indicates the width of the convolution kernel in that branch; This represents the relative coordinates of the Kth sampling position at the center of the regular convolution; The offset is predicted by the subnetwork, as shown in the following formula: (1-2) in, ; These are the weights of the convolution kernel. Sampling points are typically real-number coordinates. This indicates bilinear interpolation calculation; To reduce computational cost, this model uses depthwise separable convolution to improve upon the standard deformable convolution, decomposing the standard deformable convolution into two operations: DepthWise and PointWise. DepthWise performs separate convolutions for each channel without mixing with other channels; PointWise combines information from different channels at the same spatial location. The DepthWise operation performs independent deformable convolution sampling on each channel, and the PointWise operation then fuses them. The two steps are shown in the following equations:

[0055] in: For DepthWise convolution in channels c The k Individual core weights; (1-4) in: The weights for the 1×1 convolution kernel are from the input channel. c Mapped to output channel .

[0056] Further: The first multi-scale Transformer block: the process of fusing features from different branches based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module to obtain multi-scale image features is as follows: The multi-scale Transformer module receives multi-branch tokens from the previous layer and processes them using a multi-head self-attention mechanism. Considering that this model is used in edge devices, and that traditional self-attention mechanisms have too many parameters, this algorithm proposes a Faster Linear multi-head Self-Attention (FL-MHSA) mechanism to reduce the number of parameters in the model. The main improvement of FL-MHSA is that it replaces the exp(x) function in the original attention mechanism with a first-order Taylor expansion and rewrites the formula using the matrix multiplication associative law, ultimately reducing computational complexity.

[0057] For traditional self-attention mechanisms, assuming the input query vector Query(Q) (attention to "what I am looking for"), key vector Key(K) (attention to "what I am"), and value vector Value(V) (attention to "what information I contain") are of size Given the feature map, the standard attention calculation formula is as follows:

[0058] Rewrite formula (2-1) in a more general form, as shown in formula (2-2):

[0059] in: Representing the first of the Query matrix Line, function Represents the similarity function, when for When this happens, equation (2-2) degenerates into equation (2-1); according to the Taylor expansion, exist The location can be expanded into Substituting this expansion into formula (2-2) yields formula (2-3):

[0060] To reduce errors, and conduct After normalization and setting the normalization norm to 0.5, we obtain... and The normalization norm is set to 0.5, so and The dot product result is in the range [-0.25, 0.25]. When within range, and The result is approximate. Considering the number of parameters in the model, this model ignores the Pienos remainder term in the Taylor expansion. When using... Alternative Then, equation (2-3) is transformed into equation (2-4):

[0061] Finally, equation (2-4) is rewritten using the associative law of matrix multiplication, resulting in equation (2-5):

[0062] Where: j is the "index of the token being followed", representing the traversal index of all key-value tokens; N represents the total number of tokens (i.e. the number of key / value tokens participating in the attention calculation); For size The feature map, the computational complexity of the original multi-head attention mechanism is shown in equation (2-6).

[0063] The computational complexity of the proposed fast linear multi-head attention mechanism is shown in Equation (2-7):

[0064] In most cases, The product is much greater than Therefore, the fast linear multi-head attention mechanism proposed in this paper successfully reduces the computational complexity from [value missing]. Reduced to .

[0065] A high-efficiency image dehazing device based on Transformer, comprising: Acquisition module: Used to acquire the image to be dehazed; Build modules: Used to build Transformer-based dehazing models for dehazing images; Training module: Used to train the dehazing model to obtain a trained dehazing model; Implementation module: This module is used to input the image to be dehazed into the trained dehazing model to achieve image dehazing.

[0066] A readable storage medium storing a program module, characterized in that the program module, when run in a processor, can implement any of the methods described above.

[0067] Example: 1. Implementation Environment The dataset used for training and validation in this experiment was the publicly available RESIDE dataset, which contains five subsets. Model training used the ITS (Indoor Training Set) and OTS (Outdoor Training Set) subsets, while model validation used the SOTS (Synthetic Objective Testing Set) subset. During training, the batch size was set to 8, the Adam optimizer with a learning rate of 0.0003 was used, and L1 Loss was employed. Training was performed on a single NVIDIA RTX 4060 GPU for 300 epochs. The experimental results were evaluated using Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and the number of model parameters. PSNR is a pixel-level error metric used to measure the difference between the restored image and the original image. The unit of PSNR is usually dB; a higher dB value indicates a closer reconstructed image to the original. SSIM evaluates the quality of image restoration from the perspective of the human visual system, primarily assessing brightness, contrast, and structure. A SSIM value closer to 1 indicates a more realistic visual effect.

[0068] 2. Comparison of experimental results

[0069] 3. The dehazing model proposed in this application achieves improvements in PSNR and SSIM compared to current state-of-the-art CNN-based models, while significantly reducing the number of parameters. It also optimizes the dehazing effect and network model size. Compared to the Transformer-based Restormer model, the proposed model achieves better dehazing results with a lower number of parameters. In practical dehazing performance, FAA-Net performs poorly and exhibits strong graininess in the images; the SGID-PFF network performs well but suffers from severe color distortion; the Restormer network outperforms the former two, but some texture loss still exists in the images. In contrast, the proposed model performs exceptionally well in dehazing, achieving better results not only in metrics but also in visual quality compared to other models.

[0070] Figure 3 The figures show the comparative experimental results, where (a) is the input, (b) is the FAA-net network, (c) is the SGID-PFF network, (d) is the Restormer network, (e) is the method of this application, and (f) is the GT network.

[0071] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An efficient image dehazing method based on Transformer, characterized in that: Includes the following steps: Obtain the image to be dehazed; Construct a Transformer-based dehazing model for image dehazing; The dehazing model is trained to obtain a well-trained dehazing model; The image to be dehazed is input into a trained dehazing model to achieve image dehazing.

2. The efficient image dehazing method based on Transformer according to claim 1, characterized in that: The defogging model includes: Shallow feature extraction module: used to extract shallow features from dehazed images; Deep feature extraction module: adopts an encoder-decoder structure, and performs deep feature extraction based on the shallow feature extraction module. Channel reduction module: Used to reduce the number of channels to 3 based on the extracted deep features transmitted by the deep feature extraction module.

3. The efficient image dehazing method based on Transformer according to claim 2, characterized in that: The encoder-decoder structure includes: an encoder and a decoder; The encoder includes three residual blocks—a first residual block, a second residual block, and a third residual block—that are sequentially connected by a splicing operation. The first residual block, the second residual block, and the third residual block have the same structure; The first residual block includes: First-scale perceptual embedding module: used to extract image features using depthwise separable deformable convolution, which can generate local feature maps of different shapes and encode multi-scale pixel value sequences; First multi-scale Transformer block: Used to fuse features from different branches to obtain multi-scale image features based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module. Taylor approximate attention mechanism is introduced, and features from different branches are fused under the linear attention mechanism based on Taylor expansion to obtain multi-scale image features. The encoder includes a fourth residual block, a fifth residual block, and a sixth residual block; The fourth residual block, the fifth residual block, and the sixth residual block have the same structure; The fourth, fifth, and sixth residual blocks are connected in the dimension by connecting multiple tensor graphs. The fourth residual block includes: The fourth multi-scale Transformer block introduces the Taylor approximate attention mechanism, which fuses features from different branches under the linear attention mechanism based on Taylor expansion to obtain multi-scale image features; Fourth-scale perceptual embedding module: Based on the multi-scale image features transmitted by the fourth multi-scale Transformer block, it generates local feature maps of different shapes and encodes them to generate multi-scale pixel value sequences.

4. The efficient image dehazing method based on Transformer according to claim 3, characterized in that: The fourth multi-scale Transformer block includes: First branch: Second branch, Third branch, and Selective kernel feature fusion module: The outputs of the first branch, the second branch, and the third branch are connected to the input of the selective kernel feature fusion module; The first branch includes: a first DCN module, a first HARDWISH activation function, and a first FL-MSA module; The first DCN module, the first HARDWISH activation function, and the first FL-MSA module are connected sequentially. The second branch includes: a second DCN module, a second HARDWISH activation function, and a second FL-MSA module; The first HARDWISH activation function, the second DCN module, the second HARDWISH activation function, and the second FL-MSA module are connected in sequence. The third branch includes: a third DCN module, a third HARDWISH activation function, and a third FL-MSA module; The second HARDWISH activation function, the third DCN module, the third HARDWISH activation function, and the third FL-MSA module are connected in sequence.

5. The efficient image dehazing method based on Transformer as described in claim 1, wherein the method is characterized in that: The first scale-aware embedding module is used to extract image features using depthwise separable deformable convolution. The process of generating local feature maps of different shapes and encoding multi-scale pixel value sequences is as follows: S31: Input Features ; S32: Multi-branch parallel execution of deformable convolution operations; for a certain branch s spatial point p The standard deformable convolution formula at the given location is shown in the following equation. in: This indicates the width of the convolution kernel in that branch; This represents the relative coordinates of the Kth sampling position at the center of the regular convolution; The offset is predicted by the subnetwork, as shown in the following formula: in, ; These are the weights of the convolution kernel. Sampling points are typically real-number coordinates. This indicates bilinear interpolation calculation; The standard deformable convolution is decomposed into two operations: DepthWise and PointWise. The DepthWise operation performs independent deformable convolution sampling on each channel, while the PointWise operation performs fusion. The two operations are shown in the following equations: in: For DepthWise convolution in channels c The k Individual core weights; in: The weights for the 1×1 convolution kernel are from the input channel. c Mapped to output channel .

6. The efficient image dehazing method based on Transformer according to claim 3, characterized in that: The first multi-scale Transformer block: The process of fusing features from different branches to obtain multi-scale image features based on the multi-scale pixel value sequence transmitted by the first scale-aware embedding module is as follows: Assume the input query vector Query, key vector Key, and value vector Value are of size . Given the feature map, the standard attention calculation formula is as follows: Rewrite formula (2-1) in a more general form, as shown in formula (2-2): in: Representing the first of the Query matrix Line, function Represents the similarity function, when for When this happens, equation (2-2) degenerates into equation (2-1); according to the Taylor expansion, exist The location can be expanded into Substituting this expansion into formula (2-2) yields formula (2-3): right and conduct After normalization and setting the normalization norm to 0.5, we obtain... and The normalization norm is set to 0.5, so and The dot product result is in the range [-0.25, 0.25]. When within range, and The result is approximate, when using Alternative Then, equation (2-3) is transformed into equation (2-4): Finally, equation (2-4) is rewritten using the associative law of matrix multiplication, resulting in equation (2-5): Where: N represents the total number of tokens, and j is the "index of the token being followed", representing the traversal index of all Key-Value tokens; for a size of The feature map, the computational complexity of the original multi-head attention mechanism is shown in equation (2-6). The computational complexity of the proposed fast linear multi-head attention mechanism is shown in Equation (2-7): In most cases, The product is much greater than The value of the fast linear multi-head attention mechanism reduces the computational complexity from Reduced to .

7. A high-efficiency image dehazing device based on Transformer, characterized in that: include: Acquisition module: Used to acquire the image to be dehazed; Build modules: Used to build Transformer-based dehazing models for dehazing images; Training module: Used to train the dehazing model to obtain a trained dehazing model; Implementation module: This module is used to input the image to be dehazed into the trained dehazing model to achieve image dehazing.

8. A readable storage medium storing a program module, characterized in that, The program module, when run in a processor, can implement the method as described in any one of claims 1-6.

Images