Frequency domain decoupling flare removal method

Abstract

The invention discloses a frequency domain decoupling flare removal method, and relates to the technical field of image processing. The invention discloses a novel frequency domain flare removal method, namely a frequency decoupling flare removal network (FDDNet), and the method achieves the efficient flare removal through the decoupling of flare features and normal texture features in a frequency domain. The network is mainly composed of two key modules: a frequency decoupling module (FDM) and a frequency fusion module (FFM). According to the FDM, features are divided into low-frequency components and high-frequency components by setting masks, and the low-frequency components and the high-frequency components are optimized respectively, so that solar flares are effectively removed, and meanwhile structural details are reserved; and the FFM isolates and fuses the characteristic amplitudes and phase information of the encoder and the decoder, so that the flare interference in the phase is relieved, and the quality of the reconstructed image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a frequency domain decoupling flare removal method. Background Art

[0002] Flare is a common optical artifact that typically occurs when a strong light source enters a camera lens, especially when photographing scenes with bright light sources at night. Since lenses can become stained with fingerprints, dust, or wear during everyday use, the scattering and reflection of light are often amplified, exacerbating the effects of flare artifacts. Lens flare not only reduces image contrast and visual beauty but also obscures details around the light source, hindering computer vision tasks such as detection, segmentation, and optical flow estimation. Therefore, developing reliable and efficient nighttime glare removal algorithms has become a critical issue that needs to be addressed.

[0003] In the early days, people usually reduced the negative impact of flare on cameras through advanced optical design and materials, such as adding glass elements to the lens or applying anti-reflective coatings. However, this method is not only costly, but also usually only optimizes light of specific wavelengths and angles, with limited effectiveness. With the development of machine learning technology, researchers have begun to explore learning-based flare removal methods. Traditional methods usually first detect the shape and position of the flare and then repair the affected area, but these methods often fail to achieve satisfactory results when dealing with complex and diverse real-world scenes.

[0004] In recent years, several deep learning-based flare removal methods have been proposed. For example, Wu et al. synthesized flare-impaired images by directly adding flare images to scene images and used them to train a neural network. Qiao et al. utilized unpaired flare data to design a network consisting of a light source detection module, a flare detection and removal module, and a generation module. These methods primarily target daytime flare removal or specific flare patterns, but their performance in nighttime flare removal is limited. For nighttime scenes, Dai et al. first proposed the nighttime flare dataset, Flare7k, and further enhanced the synthesized Flare7k data with a newly captured real-world dataset, Flare-R. This enhancement effectively improves the model's ability to handle nighttime flares, as artificial and solar spectra have different diffraction patterns. However, while these data-driven methods can partially remove nighttime flares, flares in real scenes often cover large or even entire image areas, making them incapable of handling large-scale flare artifacts. To address this, FFormer proposes a global analysis method combined with fast Fourier transforms to enhance the model's receptive field and address flare from a global perspective. However, existing methods have difficulty in effectively decoupling flare from image content structure while removing flare artifacts, resulting in the inability to recover local areas damaged by artifact removal.

[0005] Therefore, a new solution to the above problems needs to be proposed. Summary of the Invention

[0006] The purpose of the present invention is to provide a frequency domain decoupling and flare removal method, and propose a corresponding frequency domain decoupling and flare removal network module to achieve deep decoupling of flare characteristics and content characteristics from the perspective of frequency domain to solve the technical problems existing in the background.

[0007] To achieve the above object, the present invention provides the following technical solution: a frequency domain decoupling flare removal method, comprising at least the following steps:

[0008] S1: First, build a frequency domain decoupling and flare removal network module. The frequency domain decoupling and flare removal network module achieves an efficient flare removal effect by decoupling the frequency characteristics of flare and content image in the frequency domain. The frequency domain decoupling and flare removal network module includes a frequency domain decoupling module and a frequency domain fusion module. The frequency domain decoupling module is FDM, and the frequency fusion module is FFM.

[0009] S2: The frequency domain decoupling module uses a mask to divide the image frequency domain information into low-frequency and high-frequency components, and dynamically optimizes their weights to effectively decouple flare from content information while maximally preserving content structure information.

[0010] S3: By improving the jump connection of the traditional Uformer, a frequency domain feature fusion module is proposed. The frequency domain feature fusion module suppresses the negative impact of strong light spots such as flares or abnormal brightness areas on the reconstructed image by fusing the amplitude and phase values ​​of the features in the encoding and decoding stages in the frequency domain respectively, retaining the original spatial structure of the image, thereby effectively repairing the local damage caused by removing the flare.

[0011] Furthermore, the frequency domain decoupling and flare removal network module adopts a common U-shaped structure and replaces the basic block with the frequency domain decoupling module to achieve the frequency domain characteristic decoupling of the flare features and the content features. The frequency domain feature fusion module is used to replace the traditional jump connection, thereby suppressing the negative impact of strong light spots on the reconstructed image.

[0012] Furthermore, the process of removing flare from an image by the frequency domain decoupling and flare removal network module includes at least the following steps:

[0013] S1.1: First, the input flare damaged image I is mapped to obtain low-level features X0;

[0014] S1.2: Next, the obtained low-level features are passed through N encoder stages, each encoder sub-stage consisting of an FDM and a down-sampling layer. The features are passed through the FDM to learn the characteristics of the flare in the frequency domain, and the down-sampling layer consists of a 2*2 convolution to halve the input feature size and double the channel number. The FDM is used as an intermediate layer of the network to enhance the perception of global structural information of the image;

[0015] S1.3: In the decoding stage, N sub-decoders are also used, and the decoder consists of an FDM and an up-sampling layer. Notably, since the FFM replaces the traditional skip connection, it can better suppress the abnormal influence of local regions on lighting and enhance the visual perception of the reconstructed image.

[0016] S1.4: After the decoding stage, the frequency domain decoupling flare removal network module outputs the restored image and the predicted flare.

[0017] Further, the FDM separates high and low frequency information in the frequency spectrum by designing a mask, and optimizes the weights of high and low frequency information to deeply decouple the flare and the content structure, while preventing the influence on the texture and other detailed information in the high frequency information.

[0018] Since high and low frequency features are located in different positions in the frequency spectrum, a simple mask generation module is further designed in the frequency domain decoupling module. The mask generation module first determines the size of the mask according to the shape of the input feature, and then calculates the radius r of the low frequency region. Specifically, the radius r of the low frequency region is controlled by a proportion parameter a, and the calculation formula is as follows:

[0019] r = a min(center h , center w )

[0020] where center h and center w are the center points of the height and width of the input feature, and a represents the relative proportion of the low frequency region occupying the entire frequency spectrum.

[0021] Subsequently, by calculating the distance of each pixel in the mask to the center point, a low frequency mask Mlow is generated with the center of the frequency spectrum as the center and the radius as r. The mask is expanded to the batch dimension and the channel dimension, so that it can be operated pixel by pixel with the input feature in subsequent frequency domain feature processing. The complement of the low frequency region mask is used as the high frequency mask MHIGH. The mask is directly multiplied with the calculated frequency spectrum of the feature to obtain high frequency features and low frequency features, respectively:

[0022] Next, define two N-dimensional global dynamic weights applied to each channel of the feature. The coefficients of these weights are determined by the input features so that the model can perceive the target features more quickly. Specifically, first define the multi-channel global filter D:

[0023] D(X)=F -1 (K(X)·F(X))

[0024] Where F(X) represents the two-dimensional Fourier transform; F -1 is the inverse Fourier transform, which is used to convert frequency domain data back to the time domain; K(X) represents the filter;

[0025] Then initialize the weights based on the input features:

[0026]

[0027] Where N is the filter dimension; F is the number of channels of the frequency component; B is the batch size; is the average value of the image; H and W are the height and width of the feature respectively;

[0028] Next, we define learnable parameters to adjust the frequency domain weight coefficients to guide the model to perceive the frequency difference between flares and content features. This process is expressed as:

[0029]

[0030] where w low represents the initial learnable parameters, r low Represents the weighted parameter, x low Represents the frequency domain representation of the input feature x.

[0031] Furthermore, the FFM fuses the amplitude and phase values ​​in the characteristic frequency domains of the encoder and decoder respectively. By suppressing the interference of flares on the phase values, the FFM can effectively preserve the spatial structure of the reconstructed image.

[0032] First, Fourier transform is performed on the decoder and the encoder features of the corresponding stage to convert the features from the spatial domain to the frequency domain;

[0033] Then the frequency domain is converted into amplitude and phase values. The process can be expressed as follows:

[0034]

[0035] Where Re is the real part of the complex number; Im is the imaginary part; arctan is the inverse tangent function; F(u,v) is the result of Fourier transform;

[0036] Then, the amplitude value of the encoder is concatenated with the amplitude value of the decoder feature, and the phase value is concatenated with the phase value, avoiding the feature interference caused by the simple concat method in the traditional U-shaped architecture, and guiding the model to perceive the characteristics of various frequency components through the multi-head self-attention mechanism, thereby further improving the accuracy of feature fusion.

[0037] Subsequently, the enhanced amplitude value and phase value are respectively inverse Fourier transformed into the spatial domain, and then enter the next stage decoder.

[0038] Compared with the prior art, the beneficial effects of the present application are:

[0039] 1. The present application proposes a novel frequency domain decoupling flare network module, which separates the flare characteristics in the frequency domain in terms of angle and depth, and maximizes the preservation of content structure.

[0040] 2. The present application proposes a frequency domain decoupling module, which optimizes the low-frequency and high-frequency information of the features by setting a threshold, while preserving the original detail information while decoupling the flare information.

[0041] 3. The present application proposes a frequency domain fusion module, which can effectively preserve the spatial structure of the reconstructed image while relieving the abnormal influence of local area illumination by suppressing the interference of flare on the phase value.

[0042] 4. Extensive experiments have proved that the method of the present application has obvious advantages in removing flare. DETAILED DESCRIPTION

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed for the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0044] Figure 1 The performance comparison diagram of the present application on the synthetic data set is shown in the figure;

[0045] Figure 2 The overall framework of the present application is shown in the figure;

[0046] Figure 3 The frequency domain fusion module of the present application is shown in the figure;

[0047] Figure 4 The qualitative comparison diagram of the present application is shown in the figure. DETAILED DESCRIPTION

[0048] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0049] A frequency domain decoupling flare removal method, at least comprising the following steps:

[0050] S1: First, build a frequency domain decoupling flare removal network module, which realizes efficient flare removal by decoupling the frequency characteristics of flare and content images in the frequency domain. The frequency domain decoupling flare removal network module includes a frequency domain decoupling module and a frequency domain fusion module. The frequency domain decoupling module is FDM, and the frequency fusion module is FFM.

[0051] S2: Use the frequency domain decoupling module to set a mask to divide the image frequency domain information into low-frequency and high-frequency parts, and dynamically optimize their weights to effectively decouple flare and content information while maximizing the preservation of content structure information.

[0052] S3: Propose a frequency domain feature fusion module by improving the skip connection of the traditional Uformer. The frequency domain feature fusion module suppresses the negative impact of strong flares or abnormal brightness areas on the reconstructed image by fusing the amplitude and phase values of the features in the encoding and decoding stages in the frequency domain, and preserves the original spatial structure in the image, thereby effectively repairing the local damage caused by removing the flare.

[0053] Referring to Figure 2 , the frequency domain decoupling flare removal network module adopts a common U-shaped structure and replaces the basic block with a frequency domain decoupling module to realize the frequency domain characteristic decoupling of flare features and content features. The frequency domain feature fusion module is used to replace the traditional skip connection, thereby suppressing the negative impact of strong flares on the reconstructed image.

[0054] The process of the frequency domain decoupling flare removal network module for removing flare from an image includes at least the following steps:

[0055] S1.1: First, map the input flare damaged image I to obtain low-level features X0;

[0056] S1.2: Next, the obtained low-level features are passed through N encoder stages, each encoder sub-stage consisting of a FDM and a downsampling layer. The features are passed through the FDM to learn the characteristics of the flare in the frequency domain, and the downsampling layer consists of a 2*2 convolution to halve the input feature size and double the channel number. The FDM is used as the middle layer of the network to enhance the perception of global structure information of the image.

[0057] S1.3: In the decoding stage, N sub-decoders are also used. The decoder consists of FDM and upsampling layers. It is worth noting that FFM replaces traditional skip connections, which can better suppress the influence of abnormal lighting in local areas and enhance the visual perception of the reconstructed image.

[0058] S1.4: After the decoding stage, the frequency domain decoupling and flare removal network module outputs the restored image and the predicted flare.

[0059] FDM separates high- and low-frequency information in the spectrum by designing a mask, and deeply decouples flare from content structure by optimizing the weights of high- and low-frequency information respectively, while preventing the impact on detailed information such as texture in high-frequency information.

[0060] The structure of FDM is as follows Figure 2 As shown in Figure 2b. Since high-frequency and low-frequency features are located at different positions in the frequency spectrum, a simple mask generation module is further designed in the frequency domain decoupling module. The mask generation module first determines the size of the mask based on the shape of the input feature and then calculates the radius r of the low-frequency region. Specifically, the radius r of the low-frequency region is controlled by a scale parameter α, which is calculated as follows:

[0061] r=α·min(center h ,center w )

[0062] Among them, center h and center w are the center points of the height and width of the input feature, respectively, and α represents the relative proportion of the low-frequency area in the entire spectrum;

[0063] Subsequently, by calculating the distance from each pixel in the mask to the center point, a low-frequency mask Mlow with the center of the spectrum as the center and a radius of r is generated. The mask will be expanded to the batch dimension and channel dimension so that it can be operated pixel by pixel with the input features in the subsequent frequency domain feature processing; the complement of the low-frequency area mask is used as the high-frequency mask MHIGH; the mask is directly dot-producted with the calculated feature spectrum to obtain high-frequency features and low-frequency features respectively:

[0064] Next, define two N-dimensional global dynamic weights applied to each channel of the feature. The coefficients of these weights are determined by the input features so that the model can perceive the target features more quickly. Specifically, first define the multi-channel global filter D:

[0065] D(X)=F -1 (K(X)·F(X))

[0066] Where F(X) represents the two-dimensional Fourier transform; F -1is the inverse Fourier transform, which is used to convert frequency domain data back to the time domain; K(X) represents the filter;

[0067] Then initialize the weights based on the input features:

[0068]

[0069] Where N is the filter dimension; F is the number of channels of the frequency component; B is the batch size; is the average value of the image; H and W are the height and width of the feature respectively;

[0070] Next, we define learnable parameters to adjust the frequency domain weight coefficients to guide the model to perceive the frequency difference between flares and content features. This process is expressed as:

[0071]

[0072] where w low represents the initial learnable parameters, r low Represents the weighted parameter, x low Represents the frequency domain representation of the input feature x.

[0073] FFM fuses the amplitude and phase values ​​in the characteristic frequency domains of the encoder and decoder respectively. By suppressing the interference of flares on the phase values, FFM can effectively preserve the spatial structure of the reconstructed image.

[0074] Frequency domain fusion module such as Figure 3 As shown in Figure 1. First, the decoder and the encoder features of the corresponding stage are Fourier transformed to convert the features from the spatial domain to the frequency domain.

[0075] Then the frequency domain is converted into amplitude and phase values. The process can be expressed as follows:

[0076]

[0077] Where Re is the real part of the complex number; Im is the imaginary part; arctan is the inverse tangent function; F(u,v) is the result of Fourier transform;

[0078] Next, the encoder's amplitude values ​​are concatenated with the decoder's amplitude values, and the phase values ​​are concatenated with the phase values. This avoids feature interference (such as the negative impact of flare) caused by the simple concatenation method in the traditional U-shaped architecture. The combined frequency domain features at different levels are passed through a multi-head self-attention mechanism to guide the model to perceive the characteristics of various frequency components, further improving the accuracy of feature fusion.

[0079] Subsequently, the enhanced amplitude and phase values ​​are subjected to inverse Fourier transform, converted into the spatial domain, and then enter the next stage decoder.

[0080] Based on the above, the following experiments are proposed for comparison and verification:

[0081] Quantitative comparison:

[0082] Table I compares the performance of our method and the current advanced method on the paired test set. The results clearly show that our method achieves the best performance in all indicators. In particular, on the two paired data sets, our method has significantly improved the PSNR and SSIM indicators compared to the existing optimal method, proving its effectiveness in removing night flare. Our method also performs well on the G-PSNR and S-PSNR flare region evaluation indicators proposed by Flare7K++, achieving the best results.

[0083] Table I: Comparison of indicators on Flare7k dataset

[0084] Real Methods PSNR SSIM LPIPS G-PSNR S-PSNR Sharmaetal. 20.492 0.826 0.1115 17.79 12.648 Wuetal. 24.613 0.871 0.0598 21.772 16.728 Zhou et al 25.184 0.872 0.0548 22.112 20.543 BracketFlare 26.587 0.885 0.0559 23.41 22.281 Flare7k 26.978 0.890 0.0466 23.507 21.563 IR-SDE 27.121 0.891 0.047 23.642 22.143 Flare7k++(U-Former) 27.633 0.894 0.0428 23.949 22.603 Kotpetal. 27.662 0.897 0.0422 23.987 22.847 RDDM 24.412 0.853 0.156 21.422 21.285 WaveDM 23.892 0.827 0.09 20.966 20.934 Ours 28.030 0.9027 0.0422 24.509 23.300

[0085] Qualitative comparison:

[0086] We conducted intuitive comparisons on the real test set and synthetic test set of Flare7K++, as shown in Figure 4 The experimental results show that our method effectively alleviates the problems existing in the recovery of local details by the existing methods. As shown in the first row of the figure, the existing method often destroys the local details of the light source after removing the flare, while our method can successfully recover these areas.

[0087] Subsequently, we conducted intuitive comparisons on the non-paired test set of real night flare contaminated images, and the results are shown in the figure. Our method performs significantly better than the existing advanced method in handling flare artifacts. For example, in the first two rows of the figure, the recovery results of the existing method still retain obvious white artifacts, even after the flare removal process. In contrast, our method can effectively eliminate these unnecessary artifacts and generate cleaner and better visual effect images. This further highlights the superior ability of our method to suppress flare while maintaining the integrity of the image content.

[0088] In summary:

[0089] This paper proposes a novel frequency domain flare removal method - frequency decoupling flare removal network (FDDNet), which achieves efficient flare removal by decoupling flare features from normal texture features in the frequency domain; the network is mainly composed of two key modules: frequency decoupling module (FDM) and frequency fusion module (FFM); among them, FDM divides the features into low-frequency and high-frequency components by setting masks, and optimizes them separately, thereby effectively removing flares while retaining structural details; FFM isolates and fuses the feature amplitude and phase information of the encoder and decoder, alleviates the flare interference in the phase, and improves the quality of the reconstructed image. Figure 1 As shown in the figure, the present invention compares the method with the existing advanced methods in terms of G-PSNR and S-PSNR, two indicators proposed by Dai et al. to evaluate the flare removal effect. It can be clearly observed from the figure that the method of the present invention shows significant advantages in both indicators, indicating its superiority in processing flare areas and restoring image quality. FDDNet outperforms the existing advanced methods in the flare removal task and has stronger robustness and visual effects.

[0090] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A frequency domain decoupling flare removal method, characterized by: At least the following steps are included: S1: First, build a frequency domain decoupling and flare removal network module. The frequency domain decoupling and flare removal network module achieves an efficient flare removal effect by decoupling the frequency characteristics of flare and content image in the frequency domain. The frequency domain decoupling and flare removal network module includes a frequency domain decoupling module and a frequency domain fusion module. The frequency domain decoupling module is FDM, and the frequency fusion module is FFM. S2: The frequency domain decoupling module uses a mask to divide the image frequency domain information into low-frequency and high-frequency components, and dynamically optimizes their weights to effectively decouple flare from content information while maximally preserving content structure information. S3: By improving the jump connection of the traditional Uformer, a frequency domain feature fusion module is proposed. The frequency domain feature fusion module suppresses the negative impact of strong light spots such as flares or abnormal brightness areas on the reconstructed image by fusing the amplitude and phase values ​​of the features in the encoding and decoding stages in the frequency domain respectively, retaining the original spatial structure of the image, thereby effectively repairing the local damage caused by removing the flare.

2. The frequency domain decoupling flare removal method according to claim 1, characterized in that: The frequency domain decoupling and flare removal network module adopts a common U-shaped structure and replaces the basic block with the frequency domain decoupling module to achieve frequency domain characteristic decoupling of flare features and content features. The frequency domain feature fusion module is used to replace the traditional jump connection, thereby suppressing the negative impact of strong light spots on the reconstructed image.

3. The frequency domain decoupling flare removal method according to claim 1, characterized in that: The process of removing flare from an image by the frequency domain decoupling and flare removal network module includes at least the following steps: S1.1: First, the input flare damaged image I is mapped to obtain low-level features X0; S1.2: Next, the obtained low-level features are passed through N encoder stages. Each encoder sub-stage consists of an FDM and a downsampling layer. The features are passed through the FDM to learn the characteristics of the flare in the frequency domain. The downsampling layer consists of a 2x2 convolution, which halves the input feature size and doubles the number of channels. The FDM is used as the middle layer of the network to enhance the perception of the global structural information of the image. S1.3: In the decoding stage, N sub-decoders are also used. The decoder consists of FDM and upsampling layers. It is worth noting that FFM replaces traditional skip connections, which can better suppress the influence of abnormal lighting in local areas and enhance the visual perception of the reconstructed image. S1.4: After the decoding stage, the frequency domain decoupling and flare removal network module outputs the restored image and the predicted flare.

4. The frequency domain decoupling flare removal method according to claim 1, characterized in that: The FDM separates high- and low-frequency information in the spectrum by designing a mask, and deeply decouples flare from content structure by optimizing the weights of high- and low-frequency information respectively, while preventing the impact on detail information such as texture in high-frequency information. Since high-frequency and low-frequency features are located at different positions in the spectrum, a simple mask generation module is further designed in the frequency domain decoupling module. The mask generation module first determines the size of the mask according to the shape of the input feature, and then calculates the radius r of the low-frequency area. Specifically, the radius r of the low-frequency area is controlled by a scale parameter α, and its calculation formula is as follows: r=α·min(center h ,center w ) Among them, center h and center w are the center points of the height and width of the input feature, respectively, and α represents the relative proportion of the low-frequency area in the entire spectrum; Subsequently, by calculating the distance from each pixel in the mask to the center point, a low-frequency mask Mlow with the center of the spectrum as the center and a radius of r is generated. The mask will be expanded to the batch dimension and channel dimension so that it can be operated pixel by pixel with the input features in the subsequent frequency domain feature processing; the complement of the low-frequency area mask is used as the high-frequency mask MHIGH; the mask is directly dot-producted with the calculated feature spectrum to obtain high-frequency features and low-frequency features respectively: Next, define two N-dimensional global dynamic weights applied to each channel of the feature. The coefficients of these weights are determined by the input features so that the model can perceive the target features more quickly. Specifically, first define the multi-channel global filter D: D(X)=F -1 (K(X)·F(X)) Where F(X) represents the two-dimensional Fourier transform; F -1 is the inverse Fourier transform, which is used to convert frequency domain data back to the time domain; K(X) represents the filter; Then initialize the weights based on the input features: Where N is the filter dimension; F is the number of channels of the frequency component; B is the batch size; is the average value of the image; H and W are the height and width of the feature respectively; Next, we define learnable parameters to adjust the frequency domain weight coefficients to guide the model to perceive the frequency difference between flares and content features. This process is expressed as: where w low represents the initial learnable parameters, r low Represents the weighted parameter, x low Represents the frequency domain representation of the input feature x.

5. The frequency domain decoupling flare removal method according to claim 1, characterized in that: The FFM fuses the amplitude and phase values ​​in the characteristic frequency domains of the encoder and decoder respectively. By suppressing the interference of flares on the phase values, the FFM can effectively preserve the spatial structure of the reconstructed image. First, Fourier transform is performed on the decoder and the encoder features of the corresponding stage to convert the features from the spatial domain to the frequency domain; Then the frequency domain is converted into amplitude and phase values. The process can be expressed as follows: Where Re is the real part of the complex number; Im is the imaginary part; arctan is the inverse tangent function; F(u,v) is the result of Fourier transform; Next, the encoder amplitude value is concatenated with the decoder amplitude value, and the phase value is concatenated with the phase value. This avoids feature interference caused by the simple concatenation method in the traditional U-shaped architecture. The combined frequency domain features at different levels are passed through a multi-head self-attention mechanism to guide the model to perceive the characteristics of various frequency components, further improving the accuracy of feature fusion. Subsequently, the enhanced amplitude and phase values ​​are subjected to inverse Fourier transform, converted into the spatial domain, and then enter the next stage decoder.

Images