A method, system, device and medium for removing moiré from high-resolution images

Through the U-shaped encoder-decoder architecture and pyramid feature extraction and attention feature fusion methods, the problem of large-scale molar patterns removal in high-resolution images is solved, and the effective removal of molar patterns artifacts and image quality improvement is achieved.

CN120259115BActive Publication Date: 2025-08-12NANCHANG UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510748305.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-06-06
Publication Date
2025-08-12
Estimated Expiration
2045-06-06

AI Technical Summary

Technical Problem

Existing methods are difficult to effectively remove large-scale molar patterns in high-resolution images under complex backgrounds, and molar artifacts often remain after removal, affecting image quality and subsequent computer vision tasks.

Method used

Using the U-type encoder-decoder architecture, combining pyramid feature extraction and attention feature fusion, the characteristics of different scales are dynamically fused through the hollow residual dense module and enhanced attention gate, and the joint loss function optimization model is constructed to train.

Benefits of technology

Effectively remove large-scale molar patterns in high-resolution images, improve the detailed information recovery and color restoration effect of molar pattern artifact areas, enhance training stability, and improve image quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120259115B_ABST
    Figure CN120259115B_ABST
Patent Text Reader

Abstract

The present application belongs to the field of image restoration in computer vision, and discloses a method, system, device and medium for removing moiré from high-resolution images. The method adopts a U-shaped multi-level encoder-decoder structure to address the problem of multi-band distribution of moiré; by constructing a feature pyramid at the same semantic level and integrating a hole residual dense module on each scale branch of the feature pyramid, multi-scale feature extraction and moiré texture removal at the same semantic level are achieved, combining multi-level residual dense connections and hole convolutions; by enhancing the attention gate, the extracted features of different scales at the same semantic level are dynamically fused, and feature fusion is enhanced by replacing ordinary convolution with grouped convolution and introducing an efficient channel attention module; by constructing a joint loss function of pixel loss, perceptual loss and color loss to optimize model training, and introducing a deep supervision strategy to enhance the stability of training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image restoration in computer vision, and in particular to a method, system, device and medium for removing moiré from high-resolution images based on pyramid feature extraction and attention feature fusion. Background Art

[0002] With the rapid development of information technology, images have become an important carrier of information and are widely used in education, industry, and other scenarios. However, when using a digital camera to capture the electronic screen of a display device, the resulting image often contains irregular colored stripes, which we call "moiré." This texture not only seriously affects the visual quality of the image, but also interferes with subsequent advanced computer vision tasks such as image classification and object detection. Moiré in an image is affected by multiple factors such as camera model, shooting distance, angle, and lighting conditions. The complexity and variability of texture patterns and the difficulty of wide distribution in multiple frequency bands pose a huge challenge to image de-moiré. Research related to image de-moiré not only helps to improve image quality and visual experience, but also provides more reliable data preprocessing support for subsequent advanced computer vision tasks, and has important application value and research significance.

[0003] Traditional methods for image moiré removal include filtering, interpolation, and image decomposition. Filtering methods typically treat moiré as noise and suppress high-frequency signals using low-pass or notch filters. This approach can easily lead to image detail loss and blurred edges. Interpolation methods reduce color fringes by reconstructing pixel information, but their effectiveness is limited in areas with complex textures and rich details. Image decomposition methods attempt to separate image content from moiré interference, but they rely heavily on prior knowledge and lack flexibility.

[0004] Compared to traditional image de-moiré methods, deep learning-based image de-moiré methods leverage the powerful feature learning capabilities of neural networks to more effectively identify and remove moiré, further improving de-moiré effectiveness. However, existing methods are primarily designed based on low-resolution moiré images. When processing moiré in high-resolution images, they still face challenges such as the difficulty of effectively removing large-scale moiré in complex backgrounds and the residual moiré artifacts after de-moiré texture removal. Further improving de-moiré effectiveness remains a research challenge that urgently needs to be overcome. Summary of the Invention

[0005] The present invention aims to solve the problems of large-scale moiré patterns being difficult to effectively remove in complex backgrounds and the residual moiré artifacts after removing the moiré texture in existing methods, and to provide a high-resolution image moiré removal method, system, device and medium based on pyramid feature extraction and attention feature fusion.

[0006] In a first aspect, the present invention provides a method for removing moiré from a high-resolution image, comprising the following steps:

[0007] Selecting an image dataset for a high-resolution image moiré removal task, wherein the image dataset comprises a set of moiré-patterned images and a set of real moiré-free images;

[0008] A high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion was constructed. This model uses a U-shaped encoder-decoder architecture and constructs feature pyramids at the same semantic level in each encoder and decoder for feature extraction. Moiré textures are removed using a dilated residual dense module, and features of different scales at the same semantic level are dynamically fused using enhanced attention gates.

[0009] Constructing a joint loss function for the model, the joint loss function including a pixel loss function, a perceptual loss function, and a color loss function;

[0010] Using the image dataset and the joint loss function to train the model, the model learns the conversion from the image with moiré to the real image without moiré;

[0011] The trained model is used to remove moiré patterns from high-resolution images.

[0012] As an optional implementation of the first aspect of the present application, the U-shaped encoder-decoder architecture includes multiple layers of encoders and decoders, the encoder includes a pixel rearrangement downsampling operation, the decoder includes a pixel rearrangement upsampling operation, and the decoder output of the intermediate layer is used to cooperate with the deep supervision strategy to optimize the model training.

[0013] As an optional implementation of the first aspect of the present application, the encoder and decoder structures include: a feature pyramid at the same semantic level for feature extraction; a hole residual dense module for removing moiré texture on different scale branches of the feature pyramid; an enhanced attention gate for dynamically fusing different scale features of the same semantic level extracted by the feature pyramid; and achieving semantic alignment of features of different scales through bilinear interpolation upsampling.

[0014] As an optional implementation of the first aspect of the present application, the dilated residual dense module combines multi-level residual dense connections and dilated convolutions, and the dilated convolution adopts a dilated convolution array of a dilated rate array.

[0015] As an optional implementation of the first aspect of the present application, the enhanced attention gate replaces the ordinary convolution of the attention gate with grouped convolution to extract multi-scale features, and suppresses irrelevant channels and enhances key channel features by introducing an efficient channel attention module, thereby achieving dynamic fusion of features of different scales.

[0016] As an optional implementation of the first aspect of the present application, the joint loss function includes: a pixel loss function, which adopts the L1 Charbonnier loss function; a perceptual loss function, which calculates the L2 distance between the real image and the model predicted image in the feature space based on a layer of extracted feature maps of a pre-trained VGG-19 network; a color loss function, which calculates the L1 norm of the real image and the model predicted image based on the U component and V component in the YUV color space.

[0017] As an optional implementation of the first aspect of the present application, the joint loss function is combined with a deep supervision strategy, and its calculation formula is: ,in, The model's Layer decoder output features, represents the corresponding real image, express and The L1Charbonnier loss between express and The perceptual loss between express and Color loss between A weighting factor representing color loss.

[0018] In a second aspect, an embodiment of the present application provides a high-resolution image moiré removal system, comprising:

[0019] A dataset selection module is used to select an image dataset for a high-resolution image demoiré task, wherein the image dataset includes a set of images with moiré patterns and a set of real images without moiré patterns;

[0020] A model building module for constructing a high-resolution image de-moiré model based on pyramid feature extraction and attention feature fusion. The model adopts a U-shaped encoder-decoder architecture and constructs feature pyramids at the same semantic level in each encoder and decoder for feature extraction. Moiré textures are removed using a dilated residual dense module, and features of different scales at the same semantic level are dynamically fused using enhanced attention gates.

[0021] A loss function construction module, configured to construct a joint loss function for the model, wherein the joint loss function includes a pixel loss function, a perceptual loss function, and a color loss function;

[0022] A model training module, configured to train the model using the image dataset and the joint loss function, so that the model learns the conversion from an image with moiré to a real image without moiré;

[0023] The moiré removal module is used to remove moiré patterns from high-resolution images using the trained model.

[0024] In a third aspect, an embodiment of the present application provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein when the program or instruction is executed by the processor, the steps of the method described in the first aspect are implemented.

[0025] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0026] Compared to existing technologies, this paper proposes a high-resolution image moiré removal method based on pyramid feature extraction and attention feature fusion for removing moiré from high-resolution images. This method utilizes a U-shaped multi-stage encoder-decoder structure to address the multi-band distribution of moiré. By constructing a feature pyramid at the same semantic level and using a dilated residual dense module on the three scale branches of the feature pyramid, this method achieves multi-scale feature extraction and moiré texture removal at the same semantic level, addressing the difficulty in effectively removing large-scale moiré in complex backgrounds. By dynamically fusing the extracted features at different scales at the same semantic level through an enhanced attention gate, this method effectively improves detail recovery and color restoration in moiré artifact regions, addressing the problem of residual moiré artifacts after moiré texture removal. A proposed joint loss function is used to optimize model training, thereby enhancing training stability. This method effectively addresses key issues currently encountered in high-resolution image moiré removal, including the difficulty in accurately distinguishing moiré in complex backgrounds, poor large-scale moiré removal, loss of detail in the de-moiréed image, and color distortion, demonstrating excellent moiré removal results. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of a method for removing moiré from a high-resolution image according to an embodiment of the present invention;

[0028] Figure 2 This is the structure diagram of the high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion;

[0029] Figure 3 is a structural diagram of an encoder / decoder according to an embodiment of the present invention;

[0030] Figure 4 is a structural diagram of a dilated residual dense block (DRDB) according to an embodiment of the present invention;

[0031] Figure 5 is a structural diagram of an enhanced attention gate (EAG) according to an embodiment of the present invention;

[0032] Figure 6 The figure is a schematic structural diagram of a high-resolution image moiré removal system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0034] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the application can be implemented in a sequence other than those illustrated or described here. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated before and after are in a kind of "or" relationship. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically limited.

[0035] Example 1

[0036] See also Figure 1 , is a flow chart of a high-resolution image moiré removal method provided by an embodiment of the present invention. The method may include the following steps:

[0037] S1: Select an image dataset for the high-resolution image moiré removal task. The image dataset contains a collection of moiré images and a collection of real moiré-free images.

[0038] It is understandable that a suitable moiré image dataset is of great significance for promoting the research of image de-moiré task. The current public datasets of high-resolution moiré images are FHDMi and UHDM datasets, which can be used for model training. For example, the FHDMi dataset contains 12,000 pairs of high-resolution images captured by real scenes, with resolutions of , covering scenes such as wallpapers, sports video frames, movie clips, documents, etc.; the UHDM dataset contains 5000 pairs of 4K high-definition images with a resolution of , covering scenes such as landscapes, sports, video frames and documents. Represents a collection of images containing moiré patterns, using represents a set of real images without moiré patterns, then each paired image data can be expressed as ,in , .

[0039] S2: Construct a high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion. The model adopts a U-shaped encoder-decoder architecture and constructs a feature pyramid at the same semantic level in each encoder and decoder for feature extraction. The moiré texture is removed through a hollow residual dense module, and different scale features at the same semantic level are dynamically fused through enhanced attention gates.

[0040] A high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion is constructed, with a convolutional neural network with a U-shaped encoder-decoder architecture as the backbone network. First, to address the problem of difficult to effectively remove large-scale moiré in complex backgrounds, this model constructs a feature pyramid at this semantic level in each encoder and decoder for feature extraction, and removes moiré textures using the DRDB module on each scale branch of the feature pyramid. Subsequently, to address the problem of moiré artifacts remaining after removing the moiré texture, this model dynamically fuses different scale features at the same semantic level through EAG to achieve information complementarity at the same semantic level, thereby improving the recovery of detail information and color restoration in the moiré artifact area.

[0041] Reference Figure 2 The overall structure of this high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion is U-shaped, consisting of three layers of encoders and decoders. The input image is first pixel-unshuffled and then input into encoder E1. The encoded result is not only passed to decoder D1 at the same level, but also bilinearly downsampled and input into encoder E2 at the next level. Encoder E2 is similar to E1, while the encoding result of encoder E3 is directly passed to decoder D3 at the same level. The decoding results of each decoder are output after pixel-shuffled upsampling. At the same time, the decoding results of decoders D3 and D2 are also bilinearly upsampled and passed to the upper decoder. The decoding results output by the intermediate decoder are used to introduce deep supervision strategies, providing multi-level supervision signals.

[0042] Reference Figure 3 The decoder structure in the model is the same as that of the encoder. The core modules include a feature pyramid at the same semantic level for feature extraction, a DRDB module for removing moiré textures, and an EAG module for removing moiré artifacts. First, feature extraction is performed by constructing a feature pyramid at the same semantic level. The DRDB module removes moiré textures at different scales of the feature pyramid. To ensure that features of different scales maintain semantic consistency in the subsequent fusion process, bilinear interpolation upsampling is used to achieve semantic alignment to facilitate subsequent feature fusion operations. EAG can dynamically fuse features of different scales by improving the attention gate (AG), thereby realizing information complementarity between features of different scales at the same semantic level.

[0043] Reference Figure 4 The DRDB module combines multi-level residual dense connections and dilated convolutions to enhance the model's ability to remove large-scale moiré textures. This module uses a dilated convolution array with a jagged dilation ratio to expand the receptive field of feature extraction without increasing computational effort, effectively capturing large-scale moiré features while avoiding the "grid effect" of dilated convolutions. Furthermore, the combination of dilated convolution arrays and residual dense connections not only mitigates the vanishing gradient problem associated with network deepening, but also preserves image details during deep information transfer.

[0044] Reference Figure 5 The EAG module is an improvement on AG. The main function of AG is to enhance the attention to important areas by adaptively focusing on the feature information of important areas. However, AG requires high semantic consistency between input features to accurately capture key features. When the correlation between input features is weak, AG will not be able to effectively extract important information. To solve the above problems, EAG replaces the ordinary convolution of AG with grouped convolution, thereby extracting richer multi-scale features without significantly increasing the computational burden. At the same time, the Efficient Channel Attention (ECA) module is introduced to suppress irrelevant channels and enhance key channel features, thereby weakening the negative impact when high-level and low-level features are weakly correlated. EAG can dynamically fuse features of different scales at the same semantic level through the improved attention gate mechanism, realizing information complementarity between features of different scales at the same semantic level.

[0045] S3: Construct a joint loss function for the model, which includes pixel loss function, perceptual loss function and color loss function.

[0046] The model proposed by this method needs to construct a joint loss function during training. The joint loss function includes pixel loss function, perceptual loss function and color loss function. Combining the above three loss functions, the final joint loss function is obtained. Represents a collection of images containing moiré patterns, using Represents a real image set, then the model training process can be expressed as: given paired image data and In the case of , the proposed model and joint loss function are used to learn the translation from moiré images to real moiré-free images.

[0047] The pixel loss function is usually used to represent the pixel-level difference between the real image and the de-moiré image output by the model. Here, the pixel-level L1 Charbonnier loss function is selected. The pixel loss function is:

[0048]

[0049] in, and Represent the real image and the de-moiré image output by the model, Is a smoothing term, taken during model training .

[0050] In order to make up for the shortcomings of pixel-level loss and make the de-moiré image output by the model closer to people's subjective perception, so as to better measure the similarity between the de-moiré image and the real image, a perceptual loss function is further introduced. The perceptual loss function is:

[0051]

[0052] in, and Represent the real image and the de-moiré image output by the model, Indicates that the pre-trained VGG-19 network is The feature map of the layer, during the model training process , represents the square of the L2 norm value, Indicates that the pre-trained network The number of channels of the feature map extracted by the layer, and Respectively represent the pre-trained VGG-19 network in the The height and width of the feature map extracted by the layer, Represents the perceptual loss between the real image and the de-moiréd image output by the model.

[0053] In the image de-moiré task, the color distortion problem of the image will further cause the loss of image details and color information, seriously affecting the quality and visual effect of the de-moiré image. In order to improve the color restoration effect of the model's de-moiré image, a color loss function is introduced. By constraining the model's color information in the YUV color space, the accuracy of color restoration is improved. The color loss function is:

[0054]

[0055] in, and Represents the de-moiré images output by the model and real images The U component in the YUV color space, and Represent the de-moiré images and real images The V component in the YUV color space, represents the L1 norm.

[0056] Combined with the deep supervision strategy, the joint loss function of the obtained model can be expressed as:

[0057]

[0058] in, The model's Layer decoder output features, represents the corresponding real image, express and The L1 Charbonnier loss between express and The perceptual loss between express and Color loss between A weighting factor representing color loss.

[0059] S4: Use the image dataset and the joint loss function to train the model so that the model learns the conversion from moiré images to moiré-free real images.

[0060] After selecting the required dataset and constructing the model and loss function, the model needs to be trained. Over multiple rounds of training, the model gradually learns to transform moiré-reduced images into moiré-free real-world images. After training, the model's moiré removal capabilities are tested. When fed a high-resolution image containing moiré, the model outputs a moiré-reduced high-resolution image.

[0061] S5: Use the trained model to remove moiré patterns from high-resolution images.

[0062] After the model is trained and tested, you can use the model's moiré removal capability, take the high-resolution moiré image as the model input, and use the trained model to perform the image moiré removal task.

[0063] In summary, in this embodiment, the present invention proposes a high-resolution image moiré removal method based on pyramid feature extraction and attention feature fusion. This method utilizes a U-shaped multi-stage encoder-decoder structure to address the multi-band distribution of moiré. By constructing a feature pyramid at the same semantic level in each encoder and decoder and applying a dilated residual dense block (DRDB) to the three scale branches of the pyramid, multi-scale feature extraction and moiré texture removal at the same semantic level are achieved. Furthermore, the extracted features at different scales at the same semantic level are dynamically fused using an enhanced attention gate (EAG) to improve detail recovery and color restoration in moiré artifact areas. This method optimizes model training through a proposed joint loss function (combining pixel, perceptual, and color losses, and incorporating a deep supervision strategy), thereby enhancing training stability and ultimately achieving high-resolution image moiré removal.

[0064] Example 2

[0065] See also Figure 6 , shown is a schematic structural diagram of a high-resolution image moiré removal system proposed in the second embodiment of the present application, the system including the following key modules:

[0066] A dataset selection module 100 is configured to select an image dataset for a high-resolution image demoiré task, wherein the image dataset comprises a set of images with moiré patterns and a set of real images without moiré patterns.

[0067] Model construction module 200, for constructing a high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion. The model adopts a U-shaped encoder-decoder architecture, constructs a feature pyramid at the same semantic level in each encoder and decoder for feature extraction, removes moiré textures using a dilated residual dense module, and dynamically fuses features of different scales at the same semantic level using an enhanced attention gate.

[0068] A loss function construction module 300 is used to construct a joint loss function for the model, wherein the joint loss function includes a pixel loss function, a perceptual loss function, and a color loss function;

[0069] A model training module 400 is configured to train the model using the image dataset and the joint loss function, so that the model learns the conversion from an image with moiré to a real image without moiré;

[0070] The moiré removal module 500 is configured to remove moiré patterns from high-resolution images using the trained model.

[0071] A high-resolution image moiré removal system in an embodiment of the present application may be a device, or a component, integrated circuit, or chip in a terminal. The device may be a mobile electronic device or a non-mobile electronic device. For example, the mobile electronic device may be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), and the non-mobile electronic device may be a server, network attached storage (NAS), personal computer (PC), etc., which are not specifically limited in the embodiments of the present application.

[0072] In the embodiments of the present application, a high-resolution image moiré removal system may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiments of the present application.

[0073] The high-resolution image moiré removal system provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, each process of implementing a high-resolution image moiré removal method in the method embodiment will not be described again here.

[0074] Optionally, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the various processes of the above-mentioned high-resolution image moiré removal method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be described here.

[0075] An embodiment of the present application also provides a readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the various processes of the embodiment of the above-mentioned high-resolution image moiré removal method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0076] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk.

[0077] It should be noted that, in this article, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element. In addition, it should be noted that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in the opposite order according to the functions involved. For example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0078] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of this application.

[0079] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A method for removing moiré from a high-resolution image, characterized in that: The following steps are involved: Selecting an image dataset for a high-resolution image moiré removal task, wherein the image dataset comprises a set of moiré-patterned images and a set of real moiré-free images; A high-resolution image moiré removal model based on pyramid feature extraction and attention feature fusion was constructed. This model uses a U-shaped encoder-decoder architecture and constructs feature pyramids at the same semantic level in each encoder and decoder for feature extraction. Moiré textures are removed using a dilated residual dense module, and features of different scales at the same semantic level are dynamically fused using enhanced attention gates. Constructing a joint loss function for the model, the joint loss function including a pixel loss function, a perceptual loss function, and a color loss function; Using the image dataset and the joint loss function to train the model, the model learns the conversion from the image with moiré to the real image without moiré; The trained model is used to remove moiré patterns from high-resolution images.

2. The method according to claim 1, characterized in that The U-shaped encoder-decoder architecture includes multiple layers of encoders and decoders, wherein the encoder includes a pixel rearrangement downsampling operation, and the decoder includes a pixel rearrangement upsampling operation. The decoder output of the middle layer is used to cooperate with the deep supervision strategy to optimize the model training.

3. The method according to claim 1, characterized in that The encoder and decoder structures include: Feature pyramid at the same semantic level is used for feature extraction; A dilated residual dense module for removing moiré textures on different scale branches of the feature pyramid; An enhanced attention gate is used to dynamically fuse features of different scales at the same semantic level extracted by the feature pyramid; And semantic alignment of features of different scales is achieved through bilinear interpolation upsampling.

4. The method according to claim 1 or 3, characterized in that The dilated residual dense module combines multi-level residual dense connections and dilated convolutions, and the dilated convolutions use a dilated convolution array with a sawtooth dilation rate array.

5. The method according to claim 1 or 3, characterized in that The enhanced attention gate extracts multi-scale features by replacing the ordinary convolution of the attention gate with grouped convolution, and suppresses irrelevant channels and enhances key channel features by introducing an efficient channel attention module, thereby achieving dynamic fusion of features of different scales.

6. The method according to claim 1, characterized in that The joint loss function includes: Pixel loss function, adopts L1 Charbonnier loss function; Perceptual loss function, based on the feature map extracted from a layer of the pre-trained VGG-19 network, calculates the L2 distance between the real image and the model's predicted image in the feature space; The color loss function calculates the L1 norm of the U and V components of the real image and the model-predicted image in the YUV color space.

7. The method according to claim 6, characterized in that The joint loss function is combined with the deep supervision strategy, and its calculation formula is: , in, The model's Layer decoder output features, represents the corresponding real image, express and The L1 Charbonnier loss between express and The perceptual loss between express and Color loss between A weighting factor representing color loss.

8. A high-resolution image moiré removal system, characterized in that: include: A dataset selection module is used to select an image dataset for a high-resolution image demoiré task, wherein the image dataset includes a set of images with moiré patterns and a set of real images without moiré patterns; A model building module for constructing a high-resolution image de-moiré model based on pyramid feature extraction and attention feature fusion. The model adopts a U-shaped encoder-decoder architecture and constructs feature pyramids at the same semantic level in each encoder and decoder for feature extraction. Moiré textures are removed using a dilated residual dense module, and features of different scales at the same semantic level are dynamically fused using enhanced attention gates. A loss function construction module, configured to construct a joint loss function for the model, wherein the joint loss function includes a pixel loss function, a perceptual loss function, and a color loss function; A model training module, configured to train the model using the image dataset and the joint loss function, so that the model learns the conversion from an image with moiré to a real image without moiré; The moiré removal module is used to remove moiré patterns from high-resolution images using the trained model.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of a high-resolution image moiré removal method as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the high-resolution image moiré removal method according to any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Low-light image enhancement method based on deep Retinex

    JP7493867B1

  • Method for image motion deblurring, apparatus, electronic device and medium therefor

    US20240404025A1