Super high-definition image moire removal method based on pyramid learnable band-pass filter
By combining the pyramid learnable bandpass filter module (P-LBF) and the cross-layer feature fusion module (CLF), the problem of moiré frequency coupling in ultra-high-definition images is solved, achieving more effective moiré removal and image restoration.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-06
- Publication Date
- 2026-03-27
AI Technical Summary
Existing demoiring techniques cannot effectively remove moiré patterns from ultra-high-definition images, especially when dealing with frequency coupling and complex moiré patterns.
Multi-scale frequency domain filtering is performed using a pyramid learnable bandpass filter module (P-LBF), and features are aligned and fused through feature alignment blocks (FA) and feature fusion blocks (FF). Combined with a cross-layer feature fusion module (CLF), effective interaction of features at different depths is achieved.
It effectively models and removes complex and coupled moiré patterns in ultra-high-definition images, improving image restoration quality, preserving image details, and demonstrating superior demoiré effect compared to existing methods.
Smart Images
Figure CN120070260B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to a kind of super high-definition image moire method based on pyramid learnable band-pass filter, more particularly to the image moire effect processing field based on deep learning technology. BACKGROUND
[0002] With the development of the times, electronic devices are everywhere in life, and the scene of using mobile phones and other electronic devices to record information by taking pictures on the screen is also becoming more and more common. However, due to the mismatch between the color filter array of the camera sensor and the sub-pixel of the screen or the interference between the color filter array and the high-frequency information, color interference signals of various colors and shapes often appear in the captured image. These interference signals usually appear in the form of stripes, mesh or corrugation, with various colors and irregularities. This interference signal is called moire. Moire is almost distributed in the entire space of the image, and the frequency is very complex, which seriously reduces the quality of the image and has a great influence on downstream computer vision tasks such as target detection and image classification. Therefore, it is worth studying to recover a clean image from a moire image. In theory, moire removal can be regarded as a traditional image restoration process, which aims to eliminate the moire pattern noise in the photographed image and restore the color space information. However, unlike other image restoration tasks such as image denoising, image deblurring and image deraining, the moire pattern and the original image information are mixed in a wide range of spatial and frequency domains, rather than simply existing in the high-frequency or low-frequency region of the image. This makes the restoration of moire images more challenging than other image restoration tasks.
[0003] With the development of deep learning, people have found that deep learning performs well in the field of image processing, and its application in image moire removal is also possible. Deep learning can use its powerful feature learning and pattern recognition capabilities to reduce or eliminate the influence of moire. Currently, many researchers have used deep learning methods to study the image moire removal task and have achieved certain results. However, existing research still has some shortcomings in processing super high-definition images. For example, DMCNN ignores the relationship between moire components at different scales, and the features extracted at each scale are not sufficient. FHDeNet removes moire from global and local scales, but ignores the relationship between different scales, making it difficult to handle the case where the moire frequencies of super high-definition images are coupled with each other. MopNet classifies and processes moire, but the moire of super high-definition images is very complex and cannot be well classified. MBCNN uses a single scale at the same semantic level for processing, making it difficult to model the complex moire of super high-definition images, and the features of different depths lack information interaction. Although ESDNet is designed for super high-definition images, it lacks explicit modeling of moire and has limited ability to handle uncommon moire.
[0004] To solve the above problems, the application provides a super high-definition image moire removal method based on a pyramid learnable band-pass filter, which can effectively model the coupled moire in the super high-definition image to restore a clear image. SUMMARY
[0005] The technical problem solved by the application: In view of the problem that the existing moire removal technology cannot effectively remove the moire in the super high-definition image, the application proposes a pyramid learnable band-pass filter module (P-LBF) under the same semantics from the perspective of the frequency domain aliasing of the moire in the super high-definition image, which decouples and filters the coupled moire from different scales, and then uses a feature alignment block (FA) and a feature fusion block (FF) to align and effectively fuse the features filtered under multiple scales, thereby effectively modeling the coupled moire. Secondly, the network for processing super high-definition images often has a deep depth, and most of the existing methods use simple pixel-level addition or channel connection to operate the features at different depths. In order to explore a more effective feature connection method at different depths, the application proposes a cross-layer feature fusion module (CLF).
[0006] Implementation steps: The application provides a super high-definition image moire removal method based on a pyramid learnable band-pass filter, and the basic steps are as follows:
[0007] Step 1: Construct an image moire removal network model based on a pyramid learnable band-pass filter. The network model takes an encoder-decoder network as the main framework, and uses a cross-layer feature fusion module (CLF) to strengthen the connection between the encoder and the decoder.
[0008] Step 2: Given a moire image, first reduce the calculation cost through a pixel shuffle, and then extract shallow features through a convolution layer.
[0009] Step 3: Input the shallow features into the encoder-decoder network.
[0010] The encoder module and the decoder module of the encoder-decoder network (each layer of the encoder module and the decoder module adopts the same structure) are 3-layer structures, which are used to gradually remove the moire from the image polluted by the moire under different scale receptive fields. Bilinear interpolation downsampling is used between every two encoder levels to halve the feature size to gradually increase the receptive field. Similarly, bilinear interpolation is also used between every two decoders to double the feature resolution and gradually change to the same size as the input image.
[0011] Step 4: In each layer of the encoder and decoder module, the input features are first further extracted using the dilated residual dense block (DRDB), then the coupled moire is decoupled filtered using the pyramid learnable band-pass filter module (P-LBF), and finally the output features F R The output features of the P-LBF are added to the output features of the DRDB to obtain the final output features of the current layer of the encoder / decoder module.
[0012] Step 5: In order to realize more effective information interaction between the shallow features in the encoder and the deep features in the decoder, the network is enabled to better preserve image details while removing moire through the cross-layer feature fusion module (CLF).
[0013] Step 6: The constructed image moire removal network model is trained.
[0014] Step 7: The trained image moire removal network model receives the verification dataset images that need to be processed for moire removal, and outputs the images after moire removal processing.
[0015] Further, the specific structure of the pyramid learnable band-pass filter module P-LBF is as follows:
[0016] The P-LBF module is composed of a pyramid filter block PF, a feature alignment block FA, and a feature fusion block FF. The PF is a pyramid structure composed of basic filter units BU, which decouples the coupled moire at different scales. The FA block aligns the multi-scale features. The FF block effectively fuses the aligned features.
[0017] The specific structure of the BU block is as follows: the input features are first processed by an expanded convolution with a convolution kernel size of 3*3 and a ReLU activation function, and then connected with the features before convolution in the channel dimension. Then, the above-mentioned repeated processing is performed 4 times, and the expansion rates of the 5 expanded convolutions are 1, 2, 3, 2, and 1, respectively. After the 5 expanded convolutions and the ReLU activation function processing, a convolution with a convolution kernel of 3*3 is performed to change the channel number to 64, i.e., to obtain a 64-channel feature map. Then, the learnable band-pass weight and the 64-channel feature map are input to the block frequency domain inverse transform (FDIT) to learn the band-pass weight, and then a 3*3 convolution and a feature scaling layer (FSL) are used. The FSL linearly constrains the output of the convolution layer to prevent the generation of excessive gradients. The FSL includes a learnable parameter s initialized to 0.1, which will be updated together with other learnable parameters in the network during the training phase. The features after the FSL are added to the original input features of the BU block as the output features of the BU block.
[0018] The specific structure of the PF block is as follows: the input features of the PF block are first respectively subjected to 2 times down-sampling and 4 times down-sampling together with the original scale features to obtain three features F x1 , F x2 , and F x3 of different scales. Then the three features of different scales are respectively given to the BU block for decoupling filtering to remove the coupled moire. Finally, the features after down-sampling are restored to a scale comparable to the original scale using bilinear up-sampling and connected in the channel dimension to obtain the pyramid filtered features F c .
[0019] The specific structure of the FA block is as follows: the FA takes the pyramid filtered features F c as input features. First, a 1-dimensional vector v c is obtained in the channel dimension using global average pooling, then a weight distribution in the channel dimension is learned using three fully connected layers and a sigmoid layer, and finally the learned weight distribution is multiplied with the features F c to obtain the aligned features F c′ .
[0020] The specific structure of the FF block is as follows: first, one-dimensional features v h and v w are obtained in the H and W dimensions respectively using average pooling, then a weight distribution in the H and W dimensions is obtained using a sigmoid layer, and then the features F c′ are multiplied with the weights in different dimensions respectively to obtain the features F h and F w . Finally, F h , F w , and F c′ are added and fused using a 1x1 convolution to obtain the output features of the FF.
[0021] Further, the specific structure of the CLF module is as follows:
[0022] The CLF accepts two input features, which are the output features F i E of the i-th (i = 1, 2) layer encoder and the features obtained by bilinear up-sampling twice on the output features of the i+1-th (i = 1, 2) layer decoder The features F i E of the i-th level encoder are input into a 1x1 convolution layer, an LReLU layer, and a 1x1 convolution layer to generate a scaling factor γ, and the features F i E are input into a 1x1 convolution layer, an LReLU layer, and a 1x1 convolution layer to generate a translation factor β. The dimensions of the parameters γ and β are the same as of the same dimensionality for guiding the decoder features transformations. Specifically, the features are scaled and shifted by the following equation:
[0023]
[0024] to get the adjusted features To better preserve the image details, a channel-wise concatenation is performed between the output features F i E and the modulated output features of the decoder Finally, a 3x3 convolution is used to fuse the concatenated features and reduce the feature dimensionality, resulting in the output features of the CLF, which are further processed as the input features of the i-th (i=1,2) layer decoder decoder.
[0025] Further, the step 6 is specifically as follows:
[0026] The training method of the network model is that the moire image I moire in the data set is input first; then the desired image I pred processed by removing moire is obtained through the designed network model; finally, the image I pred processed by removing moire output by the model is constantly optimized using the loss function, so that it gradually resembles the real clear image I clear prepared in the data set.
[0027] During the training process, the loss function Loss adopts the L1 function combined with the perceptual loss Lp as the loss function. The loss function is specifically represented as:
[0028] Loss = L1(I pred , I clear ) + k * Lp(I pred , I clear )
[0029] Where Lp(I pred , I clear ) = || μ(I pred ) - μ(I clear ) ||, μ is the pre-trained VGG-16 network. k is a model hyperparameter for balancing the L1 loss and the perceptual loss. The training adopts a multi-supervision strategy to supervise the step-by-step output of the decoder module, which is specifically represented as:
[0030] Loss_total = Loss(I pred1 , I clear1 ) + Loss(I pred2 , Iclear2 )+Loss(I pred3 ,I clear3 )
[0031] where I pred1 , I pred2 , I pred3 are the predicted images of the three decoder layers of the decoder module respectively, I clear1 , I clear2 , I clear3 are the real clear images of corresponding size.
[0032] As a preferred, the data set during training adopts the public data set UHDM data set, the data set contains 5000 groups of 4K images of different scenes, each group of images contains two images, namely the moire image I moire contaminated by moire and the original clear image I clear . 5000 groups of images, 4500 groups of images are used as training data set, and the other 500 groups of images are used as verification data set. Among them, the moire image I moire as the input image data in the network model training process, and the original clear image I clear as the reference image for comparison with the model predicted image I pred in the network model training process.
[0033] Further, the specific method of step 7 is as follows:
[0034] Load the image moire removal network model weight trained by step 6, update the parameters in the model. Secondly, the moire image I moise of the verification data set is input into the network model as input data, and the input data is sequentially subjected to the encoder and the decoder to obtain the model output image I pred3 after moire removal processing, note that only the maximum image output by the model needs to be obtained for comparison with the real clear image during model verification, that is, I pred3 .
[0035] The beneficial effects of the present application are as follows:
[0036] The present application innovatively proposes a pyramid learnable band pass filter (P-LBF) to perform multi-scale frequency domain filtering in a learnable manner to effectively model and remove complex and coupled moire patterns in ultra-high definition images. The innovation proposes a cross-layer feature fusion module (CLF) to realize more effective feature information interaction. The present application achieves the best moire removal effect on the ultra-high definition image data set UHDM. BRIEF DESCRIPTION OF DRAWINGS
[0037] Figure 1 The structural diagram of the overall network architecture of the application and the CLF module.
[0038] Figure 2 The specific structure of the P-LBF module proposed in the application. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical scheme and advantages of the application more clear and understandable, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.
[0040] The application first makes the following definitions and explanations:
[0041] I moire : Moiré picture
[0042] I clear : Real clear picture
[0043] I pred : Output picture of the moiré removal network model
[0044] Implementation steps: the application proposes a super high definition image moiré removal method based on a pyramid learnable bandpass filter, and the basic steps are as follows:
[0045] Step 1: data set preparation;
[0046] Get the image data set needed for network training in step 2. Download the public data set UHDM data set, which contains 5000 groups of 4K images of different scenes. Each group of images contains two images, namely the moiré image I moire contaminated by moiré and the original clear image I clear . 5000 groups of images, 4500 groups of images are used as training data set, and the other 500 groups of images are used as verification data set. The moiré image I moire is used as the input image data in the network model training process, and the original clear image I clear is used as the reference image for comparison with the model predicted image I pred in the network model training process.
[0047] Step 2: build an image moiré removal network model based on a pyramid learnable bandpass filter.
[0048] The network model takes an encoder-decoder network as the main framework, and uses a cross-layer feature fusion module (CLF) to strengthen the connection between the encoder and the decoder.
[0049] Given a moire image, first, a pixel shuffle is used to reduce the computational cost, and then a convolution layer is used to extract shallow features. The shallow features are input into the encoder-decoder network.
[0050] The encoder and decoder modules of the encoder-decoder network, each layer of which adopts the same structure, are 3-layer structures for step-by-step removal of moire from the moire-polluted image under different scales of receptive fields. Bilinear interpolation down-sampling is used between every two encoder levels to halve the feature size to gradually increase the receptive field. Similarly, bilinear interpolation is also used between every two decoders to double the feature resolution and gradually change to the same size as the input image.
[0051] In each layer of the encoder and decoder modules, first, the dilated residual dense block (DRDB) is used to further extract the input features, and then the pyramid learnable band-pass filter module (P-LBF) is used to decouple the coupled moire. Finally, the output features F R The output features of the P-LBF are added to the output features of the DRDB to obtain the final output features of the current layer of the encoder / decoder module.
[0052] In order to realize more effective information interaction between the shallow features in the encoder and the deep features in the decoder, the cross-layer feature fusion module (CLF) is used to enable the network to better preserve image details while removing moire.
[0053] The specific structure of the P-LBF module is as follows: The P-LBF module is composed of a pyramid filter block (PF), a feature alignment block (FA), and a feature fusion block (FF). The PF is a pyramid structure composed of basic filter units (BU) blocks, which decouples the coupled moire at different scales. Since the features after frequency domain filtering at different scales do not have explicit domain consistency constraints, there may be domain bias, so we designed an FA block to align the features at different scales. In order to strengthen the correlation between the features after decoupling filtering at different scales, we designed an FF block to effectively fuse the aligned features.
[0054] The specific structure of the BU block is as follows: after the input features are subjected to an expansion convolution with a convolution kernel size of 3*3 and a ReLU activation function, the features are connected with the features before the convolution in the channel dimension, and then subjected to 4 times of the above repeated processing, and the expansion rates of the 5 expansion convolutions are 1, 2, 3, 2, and 1 in turn. After the 5 expansion convolutions and the ReLU activation function processing, the channel number is changed to 64 through a convolution with a convolution kernel of 3*3, that is, a 64-channel feature map is obtained, and then the learnable bandpass weight and the 64-channel feature map are input into the block frequency domain inverse transform (FDIT) learning bandpass weight, and then a 3*3 convolution and a feature scaling layer (FSL) are used. The output of the convolution layer is linearly constrained through the feature scaling layer to prevent the generation of a large gradient. The FSL includes a learnable parameter s initialized to 0.1, which will be updated together with other learnable parameters in the network during the training phase. The features after the FSL are added to the original input features of the BU block as the output features of the BU.
[0055] The specific structure of the PF block is as follows: the input features of the PF block are first subjected to 2 times down-sampling and 4 times down-sampling respectively to obtain three features F x1 , F x2 , and F x3 of different scales together with the original scale features. x1 , F x2 , and F x3 . Then the three features of different scales are respectively given to the BU block for decoupling filtering to remove the coupled moire. Finally, the bilinear up-sampling is used to restore the features after the down-sampling to the scale comparable to the original scale and connect them in the channel dimension to obtain the pyramid filtered features F c .
[0056] The specific structure of the FA block is as follows: the pyramid filtered features F c are used as the input features of the FA. First, a 1-dimensional vector v c is obtained in the channel dimension using global average pooling, and then a weight distribution in the channel dimension is learned using 3 fully connected layers and a sigmoid layer. Finally, the learned weight distribution is multiplied by the features F c to obtain the aligned features F c′ .
[0057] The specific structure of the FF block is as follows: since the DCT transform used in the BU block is separable in H and W, we calculate the spatial attention in H and W dimensions respectively to ensure that the calculated results match the filtering results of the BU block. We first use average pooling in H and W dimensions respectively to obtain one-dimensional features v h and v w , and then use a sigmoid layer to obtain the weight distribution in H and W dimensions. Then, F c′ is multiplied by the weights in different dimensions respectively to obtain the features F h and Fw Finally, we add F h , F w and F c′ and fuse them using a 1x1 convolution to get the output feature of FF.
[0058] Specific structure of CLF module: when processing super-high-definition moire images, the network generally needs to have a deep depth to process complex moire step by step. However, the increase of network depth will also cause the gradual loss of image details and unstable gradient transmission of the network. Therefore, we propose to gradually modulate the deep features by shallow features to solve this problem.
[0059] Specifically, CLF accepts two input features, which are the output features of the i-th (i = 1, 2) layer encoder F i E and the output features of the i+1-th (i = 1, 2) layer decoder after bilinear upsampling twice The features F i E of the i-th level encoder are input into a 1x1 convolution layer, an LReLU layer and a 1x1 convolution layer to generate a scaling factor γ, and the features F i E are input into a 1x1 convolution layer, an LReLU layer and a 1x1 convolution layer to generate a translation factor β. The dimensions of parameters γ and β are the same as the dimensions of the features , which are used to guide the transformation of the decoder features . Specifically, the features are scaled and translated by the following formula:
[0060]
[0061] to get the adjusted features In order to better preserve the image details, we perform channel-level connection between the output features F i E of the encoder and the modulated output features of the decoder . Finally, we use a 3x3 convolution to fuse the connected features and reduce the feature dimension, so as to get the output feature of CLF, which is used as the input feature of the i-th (i = 1, 2) layer decoder decoder for further processing.
[0062] Specific structure of the complete network: Given a moire image, first, a pixel shuffle is used to reduce the computational cost, and then a convolution layer is used to extract shallow features. Then, the shallow features are input into the encoder-decoder network. Our main network framework contains 3 encoders and 3 decoders. Bilinear interpolation down-sampling is used between each two encoder levels to halve the feature size to gradually increase the receptive field. Similarly, bilinear interpolation is also used between each two decoders to double the feature resolution and gradually become the same size as the input image.
[0063] Inside each encoder / decoder, we effectively handle complex and coupled moire through the P-LBF module. To achieve more effective information interaction between the shallow features in the encoder and the deep features in the decoder, we design a CLF module to enable the network to better preserve image details while removing moire.
[0064] Step 3: Train the constructed image demoising network model using the image dataset obtained in step 1;
[0065] The training method of the network model is to first input the moire image I moire prepared in step 1 into the network model, then obtain the required image I pred processed by the designed network model, and finally use the loss function to continuously optimize the image I pred processed by the model output, so that it gradually resembles the real clear image I clear in the dataset prepared in step 1.
[0066] During the training process, the loss function Loss uses the L1 function combined with the perceptual loss Lp as the loss function. The loss function is specifically represented as:
[0067] Loss = L1(I pred ,I clear ) + k * Lp(I pred ,I clear )
[0068] Where Lp(I pred ,I clear ) = || μ(I pred ) - μ(I clear ) ||, μ is the pre-trained VGG-16 network. k is a model hyperparameter used to balance the L1 loss and the perceptual loss. The training uses a multi-supervision strategy to supervise the step-by-step output of the decoder module, which is specifically represented as:
[0069] Loss_total = Loss(I pred1 ,I clear1 ) + Loss(I pred2 ,I clear2 ) + Loss(I pred3 ,I clear3 )
[0070] where I pred1 , I pred2 , I pred3 are the predicted images of the three decoder levels of the decoder module respectively, I clear1 , I clear2 , I clear3 are the real clear images of the corresponding size respectively.
[0071] Step 4: The trained image demoisture network model receives the verification dataset image in step 1 that needs to be demoisture processed, and outputs the image after completing the demoisture processing.
[0072] Load the image demoisture network model weight trained in step 3, update the parameters in the model. Secondly, the moire image I moire in step 1 is input into the network model as input data, and the input data is sequentially input into the encoder and the decoder to obtain the model output image I pred3 after the demoisture processing, and it is noted that only the maximum image output by the model needs to be obtained for comparison with the real clear image, that is, I pred3 .
[0073]
[0074] Table 1
[0075] We compared the method in the present application with other related work on the UHDM dataset, and the experimental results are shown in Table 1. From the data results in Table 1, it can be seen that the demoisture effect of the present application on the UHDM dataset is better than that of other existing methods, which shows the effectiveness of the present application.
[0076] The above is a further detailed description of the present application in combination with specific / preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, they can make several substitutions or modifications to the described embodiments without departing from the concept of the present application, and these substitutions or modifications should be regarded as belonging to the protection scope of the present application.
[0077] The part not described in detail in the present application belongs to the known technology of those skilled in the art.
Claims
1. A method for demoiring ultra-high-definition images based on a pyramid-learnable bandpass filter, characterized in that, The steps include the following: Step 1: Construct an image demoiring network model based on a pyramid learnable bandpass filter; the network model uses an encoder-decoder network as its main framework. Step 2: Given a moiré pattern image, first reduce computational cost by recombining pixels, and then extract shallow features through a convolutional layer; Step 3: Input the shallow features into the encoder-decoder network; The encoder-decoder network has a 3-layer structure for both the encoder and decoder modules, which are used to progressively remove moiré patterns from images contaminated with moiré patterns at different receptive fields. Bilinear interpolation downsampling is used between every two encoder layers to halve the feature size to gradually increase the receptive field. Similarly, bilinear interpolation is also used between every two decoder layers to double the feature resolution and gradually make it the same size as the input image. Step 4: In each layer of the encoder and decoder modules, the input features are first further extracted using the Dilated Residual Compact Block (DRDB). Then, the Pyramid Learnable Bandpass Filter (P-LBF) module is used to decouple the coupled moiré patterns. Finally, the output features of the DRDB are processed. The final output features of the encoder and decoder modules for the current layer are obtained by adding the output features of P-LBF; Step 5: Strengthen the connection between the encoder and decoder modules through the cross-layer feature fusion module CLF; The specific structure of the CLF module is as follows: CLF accepts two input features, namely the output features of the i-th encoder layer. The features obtained by bilinearly upsampling the output features of the (i+1)th decoder layer by a factor of two Where i = 1, 2; the features of the i-th encoder layer The input is fed into a 1×1 convolutional layer, an LReLU layer, and another 1×1 convolutional layer to generate a scaling factor γ, which then modifies the features. The input is fed into a 1×1 convolutional layer, an LReLU layer, and a 1×1 convolutional layer to generate a translation factor β; the dimensions of parameters γ and β are... The dimensions are the same, used to guide features. The transformation; specifically, the features are transformed using the following formula. Perform scaling and translation: Obtain the adjusted features To better preserve image details, the output features of the i-th encoder layer... and the output features of the modulated (i+1)th decoder layer Channel-level connections are performed between them; finally, 3×3 convolutions are used to fuse the connected features and reduce the feature dimension, thereby obtaining the output features of CLF. The output features of CLF are used as the input features of the i-th decoder layer for further processing. Step 6: Train the constructed image demoiring network model; Step 7: The trained image demoiring network model receives the validation dataset images that need to be demoired, and outputs the images after completing the demoiring process.
2. The method for demoiring ultra-high-definition images based on a pyramid-learnable bandpass filter according to claim 1, characterized in that, The specific structure of the pyramid learnable bandpass filter module P-LBF is as follows: The P-LBF module consists of three parts: a pyramid filtering block (PF), a feature alignment block (FA), and a feature fusion block (FF). The PF is a pyramid structure composed of basic filtering unit (BU) blocks, which decouples the coupled moiré patterns at different scales. The FA block aligns the features at multiple scales. The FF block effectively fuses the aligned features. The specific structure of the BU block is as follows: The input features are concatenated with the features before convolution (3x3 kernel) and ReLU activation function in the channel dimension. This process is repeated four times, with the dilation rates of the five dilated convolutions being 1, 2, 3, 2, and 1 respectively. After five dilated convolutions and ReLU activation function, a 3x3 kernel convolution is applied to increase the number of channels to 64, resulting in a 64-channel feature map. The learnable bandpass weights and the 64-channel feature map are then input into the block-based frequency domain inverse transform (FDIT) to learn the bandpass weights. This is followed by a 3x3 convolution and a feature scaling layer (FSL). The FSL linearly constrains the output of the convolutional layer to prevent excessive gradients. The FSL includes a learnable parameter s initialized to 0.1, which is updated along with other learnable parameters during training. The features after FSL are added to the original input features of the BU block to obtain the output features of the BU. The specific structure of the PF block is as follows: The input features of the PF block are first downsampled by 2x and 4x respectively, and then combined with the original scale features to obtain features at three different scales. , , Then, the features at three different scales are respectively fed into the BU block for decoupling filtering to remove coupling moiré patterns; finally, bilinear upsampling is used to restore the downsampled features to a scale comparable to the original scale and concatenate them along the channel dimension to obtain the pyramid-filtered features. ; The specific structure of the FA block is as follows: FA converts the features after pyramid filtering. As input features, firstly, global average pooling is applied along the channel dimension to obtain a 1D vector. Then, three fully connected layers and a sigmoid layer are used to learn the weight distribution along the channel dimension. Finally, the learned weight distribution is compared with the features. Multiplication yields aligned features ; The specific structure of the FF block: First, average pooling is used on both the H and W dimensions to obtain one-dimensional features. , Then, a sigmoid layer is used to obtain the weight distribution along the H and W dimensions, followed by... The features are obtained by multiplying them by the weights on different dimensions respectively. and Finally, , and The features are added together and fused using a 1x1 convolution to obtain the output features of FF.
3. The method for demoiring ultra-high-definition images based on a pyramid-learnable bandpass filter according to claim 1, characterized in that, Step 6 is explained in detail below: The network model is trained by first inputting mole images from the dataset. Then, the desired demoiré processed image is obtained through the designed network model. Finally, the loss function is used to continuously optimize the demoired image output by the model. This gradually makes it resemble real, clear images in the prepared dataset. ; During training, the loss function use Loss combined with perceived loss As the loss function; the loss function is specifically expressed as: +k* in ( )- ( )||, For a pre-trained VGG-16 network; To predict images for the model, The image is the original, clear image; k is a model hyperparameter used to balance L1 loss and perceptual loss; training employs a multi-supervision strategy to supervise the progressive output of the decoder module, specifically as follows: + + in , , These are the predicted images from the three decoder layers of the decoder module. These are real, clear images of the corresponding sizes.
4. The method for demoiring ultra-high-definition images based on a pyramid-learnable bandpass filter according to claim 3, characterized in that, The training dataset uses the publicly available UHDM dataset, which contains 5000 sets of 4K images of different scenes. Each set contains two images: one with moiré patterns and the other with moiré patterns. and original clear image Of the 5000 images, 4500 were used as the training dataset, and the remaining 500 were used as the validation dataset; among them, moiré patterns... As input image data during the training process of the network model, the original clear image As used in the network model training process to predict images Reference images for comparison.
5. The method for demoiring ultra-high-definition images based on a pyramid learnable bandpass filter according to claim 3 or 4, characterized in that, The specific method for step 7 is as follows: Load the weights of the image demoiring network model trained in step 6 and update the parameters in the model; next, load the moiré images from the validation dataset. The input data is fed into the network model, where it passes through the encoder and decoder modules to obtain the model output image after moiré removal. During model validation, the largest image output is selected. Compare with real, clear images.