A dynamic noise-based invertible halftoning method and system

Abstract

The present application relates to the technical field of image hard copy reproduction, in particular to a reversible halftone method and system based on dynamic noise, comprising: constructing a reversible halftone model, including a halftone generation network and a recovery network; passing the original color image through the halftone generation network of the reversible halftone model, outputting a reversible halftone image; and passing the reversible halftone image through the recovery network, outputting a recovered color image. The present application improves the quality of the halftone image, improves the detail accuracy and color restoration degree of the reversible halftone image, ensures that the recovered color image reflects all details and color ranges of the original color image as much as possible, and improves the image processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of image hard copy technology, and in particular to a reversible halftone method and system based on dynamic noise. Background Technology

[0002] Halftone technology converts a continuous-tone image into an image containing only a limited number of colors (usually black and white) while preserving the visual grayscale effect. This is especially important in monochrome printing and display devices. By adjusting the size and distribution of black dots, halftone technology simulates different grayscale levels, enabling rich grayscale and detail to be displayed even in black and white printing. It is widely used in newspaper, magazine, book printing, and various artistic creations.

[0003] Existing halftone techniques are mainly divided into traditional methods and deep learning-based methods.

[0004] Traditional halftone algorithms mainly include ordered dithering, error diffusion, and search-based algorithms. Ordered dithering methods periodically divide a continuous-tone image into many small blocks, comparing each block with a pre-defined dithering array to generate the final halftone image. These algorithms offer high parallelism and are the fastest, but the resulting halftone images are of poor quality. Error diffusion algorithms maintain the principle of local hue invariance; in each processing step, only one pixel is quantized, and the resulting quantization error is diffused to adjacent unprocessed pixels through different weights. Although error diffusion produces unpleasant visual artifacts, it maintains a good balance between quality and efficiency, and is therefore widely used. Search-based methods treat halftone as an optimization problem. By designing a halftone evaluation metric and then optimizing using greedy or simulated annealing algorithms, good halftone results can be achieved; however, their search time is often unacceptable on real-time devices.

[0005] Deep learning algorithms based on neural networks represent a new direction for halftone technology. Choi et al. used two deep generative models to simulate the Direct Binary Search (DBS) algorithm, generating halftone images with less noise and free from artifacts associated with error diffusion, but this method was very time-consuming and unsuitable for real-time processing. Jiang et al. treated halftone as a reinforcement learning problem, cleverly solving the problem of gradient non-backpropagation after thresholding, and achieved lightweight CNN generation of visually quality-satisfying blue noise halftones by introducing a new suppression loss function. However, none of the above methods considered the invertibility of halftone in their design, a problem that needs to be addressed through inverse halftone techniques.

[0006] The purpose of inverse halftone technology is to convert halftone images back to their original or near-original continuous-tone images. This process is commonly used in image processing, editing, and analysis, especially when it is necessary to recover higher-quality image data from printed materials or processed images. However, recovering a continuous-tone image with accurate details from a halftone image that has lost information is challenging.

[0007] The most direct inverse halftone method is to use low-pass filtering to eliminate noise in high-frequency regions of the image. To improve the blurring of image edges and details caused by low-pass filtering, researchers have proposed many enhanced fast filtering algorithms. For example, Xiong et al. used wavelet transform to separate the background and edges, and performed noise reduction in the high-frequency regions of inverse halftone. Subsequently, Ste treated halftone restoration as an estimate for each pixel, using maximum a posteriori probability (MAP) to accurately reconstruct smooth regions of the image. Mese and Guo introduced an inverse halftone method using a lookup table (LUT) with prior information. To reduce complexity and improve efficiency, researchers optimized the LUT in subsequent studies. Son et al. proposed a method based on locally learned dictionary pairs, providing better detail and contour representations through feature clustering. In addition, deep convolutional neural networks have become the main method for solving various image reconstruction problems, achieving state-of-the-art performance on different datasets. Hou et al. developed the first deep learning-based inverse halftone and compression solution by training CNN as a nonlinear transformation function. Kim et al. proposed an automatic halftone image descrambling method that incorporates contextual hierarchical information. Xia et al. improved the restoration effect by merging global enhancement and detail enhancement through a progressive residual network. Li et al. further optimized the recovery effect and performance by using multi-scale input. However, halftone images lose a lot of information during their creation process, and even the most advanced inverse halftone methods can only recover an approximate grayscale version. Recovering color from gray halftone images remains a challenge.

[0008] Reversible halftone technology is a novel concept proposed by researchers in recent years. This technology generates a reversible halftone image that can be restored to the original version by embedding implicit information. This allows for accurate recovery of the original continuous-tone image from the reversible halftone image, ensuring image quality. This is particularly important for applications requiring high-quality image reconstruction, such as archival preservation, high-end printing, and professional photography. Xia et al. proposed a reversible deep halftone method that introduces dynamic Gaussian noise, enabling CNN networks to generate non-periodic reversible halftone patterns. While the reversible halftone images generated by their model have good recoverability, they do not achieve strong blue noise characteristics, and the proposed two-stage training scheme is complex and unstable. Lau et al., building on Xia's work, introduced an inverse halftone network as a brightness predictor, making the model focus more on the color restoration capability of the reversible halftone, generating a reversible halftone image with complete recoverability. However, the division of the restoration task leads to the loss of blue noise characteristics in the reversible halftone image. Furthermore, the large number of parameters results in high computational complexity and makes it difficult to apply to industrial equipment.

[0009] In summary, existing reversible halftone techniques produce halftone images with low quality and low restoration accuracy, and their high computational complexity leads to reduced image processing efficiency. Summary of the Invention

[0010] Therefore, the technical problem to be solved by the present invention is to overcome the problems of low quality and low restoration accuracy of halftone images generated in the prior art, and high computational complexity leading to reduced image processing efficiency.

[0011] To address the aforementioned technical problems, this invention provides a reversible halftone method based on dynamic noise, comprising:

[0012] Construct a reversible halftone model, including a halftone generation network and a recovery network;

[0013] The halftone generation network includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence.

[0014] Both the local feature extraction module and the high-level feature extraction module include several feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the feature extraction residual block. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The adaptive random noise is concatenated with the input features of the dynamic noise module and then passed through a single-layer convolution to obtain the output features of the dynamic noise module.

[0015] The recovery network comprises several lightweight residual blocks and flat convolutional layers;

[0016] The original color image is passed through a halftone generation network of a reversible halftone model to output a reversible halftone image; the reversible halftone image is then passed through a restoration network to output a restored color image.

[0017] Preferably, the local feature extraction module includes two feature extraction residual blocks; the advanced feature extraction module includes four feature extraction residual blocks.

[0018] Preferably, the formula for the dynamic noise module is expressed as f(I O +N), where I O The input features of the dynamic noise module are represented by N, where N represents adaptive random noise and f represents a single-layer convolution.

[0019] The dynamic noise module adaptively adjusts the intensity of random noise using the following formula:

[0020]

[0021] Where α(t) is the noise intensity at time step t, and α0 is the initial noise intensity. This is the hue loss value at time step t. γ is the initial hue loss value, and γ is a hyperparameter.

[0022] Preferably, the output module includes a standard residual block and a Gumbel-Softmax layer.

[0023] Preferably, the recovery network includes 7 lightweight residual blocks, each lightweight residual block includes two convolutional layers and a ReLU activation function, and the input features of the lightweight residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the lightweight residual block.

[0024] Preferably, the original color image is passed through a halftone generation network of a reversible halftone model to output a reversible halftone image; the reversible halftone image is then passed through a restoration network to output a restored color image, including:

[0025] The original color image is input into the input module of the halftone generation network to obtain input features; the input features are then processed by a local feature extraction module to obtain initial features; the initial features are then processed by two downsampling convolution modules to obtain downsampled features; the downsampled features are then processed by a high-level feature extraction module to further extract semantic information and compensate for noise to obtain high-level features; the high-level features are then processed by two upsampling convolution modules to obtain upsampled features; the upsampled features are then processed by an output module to obtain a reversible halftone image.

[0026] A reversible halftone image is input into a restoration network, passes through several lightweight residual blocks, and a restoration feature is obtained. The restoration feature is then passed through a flat convolutional layer to output a restored color image.

[0027] Preferably, when training the reversible halftone model, the total loss function includes the reversible halftone loss and the recovery loss;

[0028] The reversible halftone loss includes content loss, tone consistency loss, and spectral loss;

[0029] The reversible halftone image is processed by a pre-trained inverse halftone model to obtain a restored grayscale image. The restored grayscale image and the grayscale images of the original color image are input into the convolutional network VGG-19. The content loss is constructed based on the feature maps extracted from each layer of the convolutional network VGG-19 based on the grayscale images of the restored grayscale image and the original color image.

[0030] A tone consistency loss is constructed based on the mean square error calculated from the reversible halftone image and the original color image after passing through a Gaussian filter.

[0031] Construct a spectral loss based on the spectrum of a reversible halftone image and the ideal spectrum;

[0032] The recovery loss includes pixel L1 loss and spatial frequency L1 loss;

[0033] Pixel L1 loss and spatial frequency L1 loss are constructed based on the differences in pixels and spectrum between the restored color image and the original color image, respectively.

[0034] Preferably, in reversible halftone loss, the content loss is expressed as follows:

[0035]

[0036] Where i,j represent the spatial location of the feature map, k represents the channel index of the feature map, and F content F represents the grayscale image of the original color image. generated This represents the recovered grayscale image obtained from a reversible halftone image using a pre-trained inverse halftone model. and F content and F generated Feature map extracted from layer l of convolutional network VGG-19;

[0037] The formula for the loss of tone consistency is expressed as:

[0038] Among them, O h I represents a reversible halftone image. cLet G represent the original color image, and G{·} represent a Gaussian filter with a convolution kernel size of 11×11. This represents the average operator for all input images in the training set;

[0039] The formula for the spectral loss is expressed as:

[0040]

[0041] Among them, FFT generated FFT represents the spectrum of a reversible halftone image. idea Let f represent the ideal spectrum, and let f represent a halftone image with constant gray levels.

[0042] Preferably, in the recovery loss, the formula for the pixel L1 loss is expressed as:

[0043]

[0044] Among them, O c Indicates the restoration of a color image, I c This represents the original color image, where n represents the total number of pixels in the image, and u represents the pixel index.

[0045] The formula for the spatial frequency L1 loss is expressed as follows:

[0046]

[0047] Here, FFT(·) represents Fast Fourier Transform.

[0048] The present invention also provides a reversible halftone system based on dynamic noise, comprising:

[0049] The model building module is used to build reversible halftone models, including halftone generation and recovery networks;

[0050] The halftone generation network includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence.

[0051] Both the local feature extraction module and the high-level feature extraction module include several feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the feature extraction residual block. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The adaptive random noise is concatenated with the input features of the dynamic noise module and then passed through a single-layer convolution to obtain the output features of the dynamic noise module.

[0052] The recovery network comprises several lightweight residual blocks and flat convolutional layers;

[0053] The reversible halftone module is used to process the original color image through the halftone generation network of the reversible halftone model and output a reversible halftone image; the reversible halftone image is then processed by the restoration network to output a restored color image.

[0054] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0055] This invention discloses a reversible halftone method based on dynamic noise. The halftone generation network employs a U-shaped network structure. A downsampling convolutional module progressively reduces the image spatial resolution while extracting global information, while an upsampling module restores the image spatial resolution and reconstructs the detailed information of the original color image. This helps the model optimize feature representation at both global and local levels. The feature extraction residual blocks used in the local and high-level feature extraction modules are designed with lightweight features, solving the problems of insufficient network depth or excessive computational overhead in traditional methods. This reduces the number of model parameters and computational complexity while retaining sufficient expressive power. Furthermore, the feature extraction residual blocks utilize a dynamic noise module to introduce noise of varying intensities. By dynamically adjusting the noise intensity, the reversible halftone image is continuously perturbed during generation, avoiding the risk of the network getting trapped in local flat solutions. This ensures that the reversible halftone image output by the halftone generation network has better smoothness, improving the quality and restoration accuracy of the reversible halftone image.

[0056] Furthermore, this invention introduces spectral loss into the inverse halftone loss and spatial frequency L1 loss into the restoration loss. By mapping the image from the spatial domain to the frequency domain through Fourier transform, it pays more attention to the distribution of high-frequency components, provides global feature constraints, and compensates for the locality defects of pixel-level loss. This allows the generated reversible halftone image and restored color image to better preserve the details in the original color image. Moreover, the frequency domain loss and pixel-level loss complement each other. This invention optimizes the generation results in both the spatial and frequency domains simultaneously, further improving the restoration performance of the reversible halftone image.

[0057] Therefore, the reversible halftone image generated by this invention is visually closer to the input image, improving the quality of the halftone image, enhancing the accuracy of details and color reproduction in the reversible halftone image, and ensuring that the restored color image reflects as much detail and color range as possible from the original color image. Furthermore, this invention reduces the number of parameters in the reversible halftone model to accelerate image processing, reduce computational resources consumed during image processing, improve image processing efficiency, and make reversible halftone technology suitable for real-time processing and large-scale image data processing. Attached Figure Description

[0058] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0059] Figure 1 This is a structural diagram of the reversible halftone model of the present invention;

[0060] Figure 2 This is a structural diagram of the dynamic noise module;

[0061] Figure 3 This is a visualization comparison of the flatness of the halftone in a CNN network, where... Figure 3 (a) in the image is the input image. Figure 3 In the image, (b) and (c) represent the intermediate feature map and output halftone map of a traditional CNN, respectively. Figure 3 In the image, (d) and (e) represent the intermediate feature map and output halftone map of the CNN with NIB added, respectively. Figure 3 In the diagram, (f) and (g) represent the intermediate feature map and the output halftone map of the CNN with the dynamic noise module added, respectively.

[0062] Figure 4 This is a comparison of the visual quality of halftone images, in which... Figure 4 In the image, (a) represents the input continuous grayscale image. Figure 4 Image (b) in the image is the halftone image output by the Ostromouhkov method. Figure 4 (c) in the image is the halftone image output by the TRH method. Figure 4 In the image, (d) represents the halftone image output by the RVH method. Figure 4 (e) in the figure represents the halftone image output by the method of the present invention;

[0063] Figure 5 It is a visual contrast image that restores the color image, in which Figure 5 (a) in the image is the original color image and its details. Figure 5 In the image (b), the reversible halftone image and its magnified portion, as well as the restored color image and its magnified portion, are output by the RVH method. Figure 5 In the diagram, (c) represents the reversible halftone image and its magnified portion, the restored color image and its magnified portion, output by the method of this invention. Figure 5 In the diagram, (d) represents the restored color image output by the MPRL+IaCol method;

[0064] Figure 6 This is a comparison of reversible halftone images using NIB and those using a dynamic noise module. Figure 6 In the image, (a) represents the input grayscale image. Figure 6 Image (b) in the image is a reversible halftone image obtained using NIB. Figure 6(c) in the image is a reversible halftone image obtained using the dynamic noise module; Figure 6 (d) in the image is a magnified view of (a). Figure 6 (e) in the image is a magnified view of (b). Figure 6 (f) in the diagram is a magnified view of (c). Detailed Implementation

[0065] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0066] Example 1

[0067] Reference Figure 1 As shown, the present invention provides a reversible halftone method based on dynamic noise, which generates reversible halftone images and restores color images by constructing a reversible halftone model.

[0068] The reversible halftone model includes a halftone generation network and a recovery network.

[0069] The halftone generation network adopts a U-shaped network structure, which includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence.

[0070] The input module includes a convolutional layer and a ReLU activation function, used to receive the input raw color image and extract initial features. The convolutional layer has a kernel size of 3×3, a stride of 1, and padding of 1.

[0071] The local feature extraction module includes two feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to form the output features of the feature extraction residual block.

[0072] A typical convolutional neural network (CNN) consists of convolutional layers (with bias terms), pooling layers, activation functions, and fully connected layers. When the input is planar, it cannot introduce spatial variations into the output. This is due to the paradigm of spatially shared convolutional kernels in convolutional operations, which Xia et al. termed "flatness degradation." To address this issue, they proposed a Noisy Excitation Block (NIB), which can be used as a network-agnostic plugin for CNNs. While the NIB circumvents the flatness degradation problem, a more pressing drawback is the lack of noise perturbation, which often leads to overly regular textures guided by error diffusion. This perturbation is gradually reduced during the network's convolutional operations when noise is added only at the input stage.

[0073] Existing techniques such as NIB employ fixed-intensity noise, using the same noise distribution and intensity for all inputs throughout the training and generation process. This fails to adaptively adjust the noise level based on the characteristics of the input data or feature layers. Fixed-intensity noise may be insufficient when certain layers require stronger noise to overcome flatness issues. Furthermore, fixed-intensity noise may corrupt important features when certain layer features are fragile.

[0074] Therefore, this invention utilizes a dynamic noise module to introduce noise of varying intensities into each feature extraction residual block. By dynamically adjusting the noise intensity, the reversible halftone image is continuously perturbed during the generation process. The key idea of ​​the dynamic noise module is to introduce spatially varied feature representations without compromising the integrity of the original input information. Furthermore, the randomness of the noise is beneficial for jitter handling because it focuses on the overall pattern distribution rather than individual pixel values, thus providing the network with sufficient jitter dependency.

[0075] The structure of the dynamic noise module is referenced. Figure 2 As shown. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The adaptive random noise is concatenated with the input features of the dynamic noise module and then passed through a single-layer convolution to obtain the output features of the dynamic noise module, expressed by the formula f(I). O +N), where I O denoted by , where N represents the input features of the dynamic noise module, N represents adaptive random noise, and f represents a single-layer convolution.

[0076] Specifically, the dynamic noise module adaptively adjusts the intensity of random noise using the following formula:

[0077]

[0078] Where α(t) is the noise intensity at time step t, and α0 is the initial noise intensity. This is the hue loss value at time step t. γ is the initial hue loss value, and γ is a hyperparameter that controls the noise intensity response.

[0079] The adaptive design of noise diversity in the dynamic noise module avoids the risk of the network getting stuck in a local flat solution, solves the limitations of fixed noise design, ensures that the reversible halftone image output by the halftone generation network has better smoothness, and improves the quality and restoration accuracy of the reversible halftone image. Figure 3 A visualization comparison of the flatness of halftone in a CNN network, where... Figure 3 (a) in the image is the input image. Figure 3 In the image, (b) and (c) represent the intermediate feature map and output halftone map of a traditional CNN, respectively. Figure 3In the image, (d) and (e) represent the intermediate feature map and output halftone map of the CNN with NIB added, respectively. Figure 3 In the diagram, (f) and (g) represent the intermediate feature map and the output halftone map of the CNN with the dynamic noise module added, respectively.

[0080] The downsampling convolutional module comprises two convolutional layers connected in sequence and a ReLU activation function. It progressively reduces the resolution of the feature map while increasing the number of channels to extract more abstract features, typically containing global information. The downsampled feature map contains high-level semantic information of the image, but due to the reduced spatial resolution, its local details may be lost.

[0081] Preferably, the kernel size of each convolutional layer in the downsampling convolutional module is 3×3, the stride is 2, and the padding is 1.

[0082] The advanced feature extraction module includes four feature extraction residual blocks, which are used to further extract advanced features with semantic information from the image and compensate for noise.

[0083] The upsampling convolutional module comprises a sequentially connected upsampling layer, two convolutional layers, and a ReLU activation function. It is used to restore the feature map to the input size by progressively recovering the spatial resolution and to help reconstruct the image's detailed information. During this process, the upsampling convolutional module fuses information from corresponding layers of the downsampling convolutional module to obtain local detailed information.

[0084] Preferably, the upsampling layer uses bilinear interpolation or deconvolution to upsample the feature map. Each convolutional layer in the upsampling convolutional module has a kernel size of 3×3, a stride of 1, and padding of 1.

[0085] The output module includes a standard residual block and a Gumbel-Softmax layer. The Gumbel-Softmax layer is used to quantize the output of the halftone generation network into 0 or 1, resulting in a reversible halftone image.

[0086] The standard residual block comprises two structures consisting of convolutional layers and ReLU activation functions, as well as skip connections.

[0087] This invention utilizes the reparameter-reversal strategy of Gumbel-softmax to avoid interference with the model's gradient propagation in the output of the halftone generation network. It enables the network output to be directly quantized as 0 or 1, enhancing the correspondence between the halftone generation network and the grayscale values ​​of the real image.

[0088] The halftone generation network addresses the problem of insufficient multi-scale information extraction from the original color image by using downsampling and upsampling convolutional modules, helping the model optimize feature representation at both global and local levels. Simultaneously, the feature extraction residual blocks employed in the halftone generation network are based on a lightweight design, solving the problems of insufficient network depth or excessive computational overhead in traditional methods, reducing the number of model parameters and computational complexity while retaining sufficient expressive power. Furthermore, the halftone generation network constructed in this invention introduces noise of varying intensities into each feature extraction residual block, solving the problem of network flatness degradation, continuously perturbing the network during the learning process, thereby enhancing the reconstructive capability of the generated reversible halftone images.

[0089] The recovery network consists of 7 lightweight residual blocks and a flat convolutional layer.

[0090] Each lightweight residual block is used to extract reversible halftone features and reconstruct grayscale information. It consists of two convolutional layers and a ReLU activation function. The input features of the lightweight residual block and the output features of the ReLU activation function are added together through skip connections to form the output features of the lightweight residual block.

[0091] Flat convolutional layers are used to generate restored color images. They are single convolutional layers with a kernel size of 1×1, a stride of 1, and padding of 0.

[0092] The original color image is passed through a halftone generation network of a reversible halftone model to output a reversible halftone image; the reversible halftone image is then passed through a restoration network to output a restored color image, including:

[0093] The original color image is input into the input module of the halftone generation network to obtain input features; the input features are then processed by a local feature extraction module to obtain initial features; the initial features are then processed by two downsampling convolution modules to obtain downsampled features; the downsampled features are then processed by a high-level feature extraction module to further extract semantic information and compensate for noise to obtain high-level features; the high-level features are then processed by two upsampling convolution modules to obtain upsampled features; the upsampled features are then processed by an output module to obtain a reversible halftone image.

[0094] A reversible halftone image is input into a restoration network, passes through several lightweight residual blocks, and a restoration feature is obtained. The restoration feature is then passed through a flat convolutional layer to output a restored color image.

[0095] The total loss function during training of the reversible halftone model includes the reversible halftone loss and the recovery loss, as shown in the formula:

[0096] L = L h +L r

[0097] Where L represents the total loss function, L h L represents reversible halftone loss. r This indicates the recovery of losses.

[0098] The reversible halftone loss includes content loss, tone consistency loss, and spectral loss, as expressed in the formula:

[0099] L h =ω1L content +ω2L tone +ω3L freq

[0100] Among them, L content Indicates content loss, L tone L represents the loss of tonal consistency. freq ω1 represents the spectral loss; ω2 and v3 are the loss weights for content loss, hue consistency loss, and spectral loss, respectively. In this embodiment, the values ​​are ω1 = 1, ω2 = 0.4, and v3 = 0.1.

[0101] Content loss is used to measure the similarity between the generated halftone image and the content image. Specifically, this invention evaluates content loss by calculating the feature differences between the generated reversible halftone image and the content image at certain intermediate layers (usually convolutional layers). That is, a pre-trained inverse halftone model is used to recover the grayscale image from the reversible halftone image, and the content error between the grayscale image and the original color image is calculated.

[0102] In this embodiment, the reversible halftone image is processed by a pre-trained inverse halftone model to obtain a restored grayscale image. The restored grayscale image and the grayscale images of the original color image are input into the VGG-19 convolutional network. Based on the feature maps extracted from each layer of the VGG-19 convolutional network from the restored grayscale image and the grayscale images of the original color image, a content loss is constructed, expressed by the formula:

[0103]

[0104] Where i,j represent the spatial location of the feature map, k represents the channel index of the feature map, and F content F represents the grayscale image of the original color image. generated This represents the recovered grayscale image obtained from a reversible halftone image using a pre-trained inverse halftone model. and F content and F generated Feature map extracted from layer l of convolutional network VGG-19.

[0105] Preferably, the pre-trained inverse halftoning model adopts the model from Deep inverse halftoning via progressive residual learning.

[0106] Tone consistency loss is used to ensure that the generated reversible halftone image remains visually similar to the input original color image, measured separately for the original color image I. c and reversible halftone image O h The mean square error between the Gaussian filters is calculated using the following formula:

[0107]

[0108] Among them, O h I represents a reversible halftone image. c G represents the original color image, and G(·) represents a Gaussian filter with a convolution kernel size of 11×11. This represents the average operator for all input images in the training set.

[0109] Existing methods mostly rely on pixel-level losses, such as L1 or L2 losses, which are difficult to effectively capture the global frequency domain features of an image. The difference in visual quality between halftone and grayscale images often lies in the mismatch between the distribution of high-frequency and low-frequency components. Reversible halftone images generated by existing methods may exhibit abrupt high-frequency noise in transition regions, affecting visual quality. Since variance measures the degree of data dispersion, it can explicitly characterize the distribution characteristics and fluctuations of halftone points. Based on these analyses, this invention designs a spectral loss to constrain the consistency between the generated halftone image and the target image in the frequency domain, particularly focusing on high-frequency components (the distribution of edges and details). The spectral loss uses the mean square error (MSE) between the spectrum of the model-generated reversible halftone image and the ideal spectrum as the loss function to accurately evaluate the model's ability to preserve the blue noise characteristics of the image. By reducing unnecessary high-frequency oscillations in the frequency domain, it improves the smoothness and visual perception quality of the halftone image.

[0110] Specifically, the spectral loss is constructed based on the spectrum of the reversible halftone image and the ideal spectrum, with the following formula:

[0111]

[0112] Among them, FFT generated FFT represents the spectrum of a reversible halftone image. idea Let f represent the ideal spectrum, and let f represent a halftone image with constant gray levels.

[0113] Since spectral analysis is only applicable to halftone images with constant gray levels, this invention optimizes the loss function by using additional mini-batch images with constant gray levels when training the model.

[0114] Spectral loss maps an image from the spatial domain to the frequency domain via Fourier transform, providing global feature constraints and compensating for the locality limitations of pixel-level loss. It offers better constraints on the overall distribution of low-frequency and smooth regions, improving the overall consistency of the generated reversible halftone image. Spectral loss focuses on the distribution of high-frequency components, enabling the generated reversible halftone image and the restored color image to better preserve or reproduce details.

[0115] The spectral constraints in reversible halftone loss effectively suppress pseudo-high-frequency noise in halftone images, improving the smoothness of the generated image. Secondly, human vision is more sensitive to frequency domain features, especially the mid- and high-frequency components; introducing spectral loss can better optimize the perceptual quality of the generated image. Furthermore, spectral loss can complement pixel-level losses such as L1 / L2, simultaneously optimizing the generation results in both the spatial and frequency domains, thus improving the restoration performance of reversible halftone and grayscale images.

[0116] The recovery loss includes pixel L1 loss and spatial frequency L1 loss, as shown in the formula:

[0117] L r =L pixel +L FFT

[0118] Among them, L pixel Indicates pixel L1 loss, L FFT This represents the spatial frequency L1 loss.

[0119] The pixel L1 loss is constructed based on the pixel differences between the restored color image and the original color image, and the formula is as follows:

[0120]

[0121] Among them, O c Indicates the restoration of a color image, I c This represents the original color image, where n represents the total number of pixels in the image, and u represents the pixel index.

[0122] The spatial frequency L1 loss is constructed based on the difference in the spectrum between the restored color image and the original color image, and the formula is as follows:

[0123]

[0124] FFT(·) represents Fast Fourier Transform, which is used to convert image signals from the spatial domain to the frequency domain.

[0125] Since grayscale image restoration requires reconstructing image details (high-frequency information) and global smoothness (low-frequency information), and traditional restoration loss functions' constraints on local pixels may not fully describe the differences in frequency components, the restoration loss employed in this invention constrains the restored grayscale image to closely approximate the original image in the frequency domain, ensuring that both high-frequency details and low-frequency smoothness can be accurately restored. This enhances the ability to restore texture and details while avoiding over-smoothing.

[0126] In summary, the reversible halftone method based on dynamic noise described in this invention employs a U-shaped network structure in the halftone generation network. The downsampling convolutional module progressively reduces the image spatial resolution while extracting global information, while the upsampling module restores the image spatial resolution and reconstructs the detailed information of the original color image, helping the model optimize feature representation at both global and local levels. The feature extraction residual blocks used in the local and high-level feature extraction modules are based on a lightweight design, solving the problems of insufficient network depth or excessive computational overhead in traditional methods. This reduces the number of model parameters and computational complexity while retaining sufficient expressive power. Furthermore, the feature extraction residual blocks utilize a dynamic noise module to introduce noise of varying intensities. By dynamically adjusting the noise intensity, the reversible halftone image is continuously subjected to perturbations during generation, avoiding the risk of the network getting trapped in locally flat solutions. This ensures that the reversible halftone image output by the halftone generation network has better smoothness, improving the quality and restoration accuracy of the reversible halftone image.

[0127] Furthermore, this invention introduces spectral loss into the inverse halftone loss and spatial frequency L1 loss into the restoration loss. By mapping the image from the spatial domain to the frequency domain through Fourier transform, it pays more attention to the distribution of high-frequency components, provides global feature constraints, and compensates for the locality defects of pixel-level loss. This allows the generated reversible halftone image and restored color image to better preserve the details in the original color image. Moreover, the frequency domain loss and pixel-level loss complement each other. This invention optimizes the generation results in both the spatial and frequency domains simultaneously, further improving the restoration performance of the reversible halftone image.

[0128] Therefore, the reversible halftone image generated by this invention is visually closer to the input image, improving the quality of the halftone image, enhancing the accuracy of details and color reproduction in the reversible halftone image, and ensuring that the restored color image reflects as much detail and color range as possible from the original color image. Furthermore, this invention reduces the number of parameters in the reversible halftone model to accelerate image processing, reduce computational resources consumed during image processing, improve image processing efficiency, and make reversible halftone technology suitable for real-time processing and large-scale image data processing.

[0129] Example 2

[0130] This embodiment verifies the effectiveness of the reversible halftone method based on dynamic noise described in Embodiment 1 through experiments.

[0131] Regarding halftone, the visual quality comparison results of the halftone images are referenced. Figure 4 As shown, where Figure 4 In the image, (a) represents the input continuous grayscale image. Figure 4 (b) in the image is the halftone image output by the Ostromouhkov method. Figure 4 (c) in the image is the halftone image output by the TRH method. Figure 4 In the image, (d) represents the halftone image output by the RVH method. Figure 4 In the diagram, (e) represents the halftone image output by the method of this invention. From... Figure 4 As can be seen, the snail image demonstrates the advantage of the proposed method in halftone detail. The reversible halftone proposed in this invention preserves the recognizable structure of the original image. In contrast, classic methods completely destroy the vein network because they do not consider the structural details of the image. Even DBS only continuously optimizes the halftone image pixel by pixel using the HVS model. Compared to RVH, the proposed method achieves almost identical visual results and has a more efficient runtime.

[0132] The quantitative evaluation results of the halftone algorithm are shown in Table 1. It can be seen that the method of this invention achieves optimal performance on SSIM and is also competitive on PSNR.

[0133] Table 1. Quantitative Evaluation Results of Halftone Algorithm

[0134]

[0135] Regarding the recovery effect of inverse halftone, in order to evaluate the ability of the method of the present invention to reconstruct continuous tones and subtle details, the best current inverse halftone method, MPRL, was selected for grayscale image recovery and combined with the most advanced automatic colorization method, IA-COL, to compare the recovered color image output by the present invention. Furthermore, this embodiment also compares the method of the present invention with the RVH method to confirm that the method of the present invention is superior to RVH in terms of recovery capability.

[0136] The quantitative evaluation results of the inverse halftone algorithm are shown in Table 2. This comparison may not be entirely appropriate because the input to the inverse halftone model is a halftone image obtained using error diffusion, while the network input of this invention is a reversible halftone image with embedded information. However, by comparing it with these state-of-the-art methods, the effectiveness of the method in image restoration can be reflected to some extent. As can be seen from Table 2, the method of this invention outperforms RVH in both PSNR and SSIM.

[0137] Table 2. Quantitative evaluation results of the inverse halftone algorithm

[0138]

[0139]

[0140] Figure 5 To restore the visual contrast of a color image, where Figure 5 (a) in the image is the original color image and its details. Figure 5 In the image (b), the reversible halftone image and its magnified portion, as well as the restored color image and its magnified portion, are output by the RVH method. Figure 5 In the diagram, (c) represents the reversible halftone image and its magnified portion, the restored color image and its magnified portion, output by the method of this invention. Figure 5 In the diagram, (d) represents the restored color image output by the MPRL+IaCol method. It can be seen that the method of this invention restores finer details and more realistic colors than the RVH and MPRL+IaCol methods. In fact, inverse halftone relies on inferring original information from halftone images, while automatic colorization extends seemingly reasonable colors to grayscale images. Compared to the RVH method, the method of this invention introduces noise at each layer of the network and embeds more hidden information during the generation of reversible halftone.

[0141] Regarding the recovery effect of reversible halftone, NIB is compared with the dynamic noise module proposed in this invention. Figure 6 A comparison of reversible halftone images using NIB and using the dynamic noise module, where Figure 6 In the image, (a) represents the input grayscale image. Figure 6 Image (b) in the image is a reversible halftone image obtained using NIB. Figure 6 (c) in the image is a reversible halftone image obtained using the dynamic noise module; Figure 6 (d) in the image is a magnified view of (a). Figure 6 (e) in the image is a magnified view of (b). Figure 6 In the diagram, (f) is a magnified view of (c). Compared with existing reversible halftone methods, the method of this invention can produce better textures, such as... Figure 6 As shown in (c) above. Other reversible halftone methods exhibit noticeable artificial textures, such as... Figure 6 As shown in (b) of the image. In the criteria for evaluating halftone images, the uniformity and discreteness of the point distribution are particularly important, as these represent better blue noise characteristics. Figure 6The paper demonstrates samples generated by the model components Dynamic Noise Module (DNRMM) and Noise Induced Enhancement (NIB), which allows for easy selection of the optimal halftone image. Therefore, the proposed DNRMM guides accurate decisions at each layer of the model, something that previous reversible halftone methods using NIB and NCB could not achieve.

[0142] Regarding runtime, this embodiment evaluates the runtime of each algorithm on a 512*512 "Lenna" test case, and the results are shown in Table 3.

[0143] Table 3. Comparison of running time and model parameter count

[0144]

[0145] As shown in Table 3, in the reversible halftone part, compared with RVH and TRH, the model of this invention reduces the number of parameters by 96.77% and the processing time by 57.28%, with a runtime comparable to that of traditional error diffusion. In the image restoration part, compared with RVH and TRH, the model of this invention not only improves processing speed but also achieves a certain improvement in image quality. The method of this invention demonstrates that lightweight CNNs have the ability to generate better and faster reversible halftone images.

[0146] Example 3

[0147] Based on the reversible halftone method based on dynamic noise described in Embodiment 1, this embodiment proposes a reversible halftone system based on dynamic noise, including:

[0148] The model building module is used to build reversible halftone models, including halftone generation and recovery networks;

[0149] The halftone generation network includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence.

[0150] Both the local feature extraction module and the high-level feature extraction module include several feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the feature extraction residual block. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The adaptive random noise is concatenated with the input features of the dynamic noise module and then passed through a single-layer convolution to obtain the output features of the dynamic noise module.

[0151] The recovery network comprises several lightweight residual blocks and flat convolutional layers;

[0152] The reversible halftone module is used to process the original color image through the halftone generation network of the reversible halftone model and output a reversible halftone image; the reversible halftone image is then processed by the restoration network to output a restored color image.

[0153] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0155] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0157] Obviously, the above embodiments are merely illustrative examples for clear explanation and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations here. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A reversible halftone method based on dynamic noise, characterized in that, include: Construct a reversible halftone model, including a halftone generation network and a recovery network; The halftone generation network includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence. Both the local feature extraction module and the high-level feature extraction module include several feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the feature extraction residual block. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The formula for the dynamic noise module is expressed as follows: ,in This represents the input characteristics of the dynamic noise module. Indicates adaptive random noise, This represents a single-layer convolution; The dynamic noise module adaptively adjusts the intensity of random noise using the following formula: ， in, It is the noise intensity at time step t. It is the initial noise intensity. This is the hue loss value at time step t. This is the initial hue loss value. It's a hyperparameter; The adaptive random noise and the input features of the dynamic noise module are concatenated and then passed through a single-layer convolution to obtain the output features of the dynamic noise module. The recovery network comprises several lightweight residual blocks and flat convolutional layers; The original color image is passed through a halftone generation network of a reversible halftone model to output a reversible halftone image; the reversible halftone image is then passed through a restoration network to output a restored color image.

2. The reversible halftone method based on dynamic noise according to claim 1, characterized in that, The local feature extraction module includes two feature extraction residual blocks; the advanced feature extraction module includes four feature extraction residual blocks.

3. The reversible halftone method based on dynamic noise according to claim 1, characterized in that, The output module includes a standard residual block and a Gumbel-Softmax layer.

4. The reversible halftone method based on dynamic noise according to claim 1, characterized in that, The recovery network consists of 7 lightweight residual blocks. Each lightweight residual block includes two convolutional layers and a ReLU activation function. The input features of the lightweight residual block and the output features of the ReLU activation function are added together through skip connections to form the output features of the lightweight residual block.

5. The reversible halftone method based on dynamic noise according to claim 1, characterized in that, The original color image is passed through a reversible halftone model halftone generation network to output a reversible halftone image. The reversible halftone image is then passed through a restoration network to output a restored color image, including: The original color image is input into the input module of the halftone generation network to obtain input features; the input features are then processed by a local feature extraction module to obtain initial features; the initial features are then processed by two downsampling convolution modules to obtain downsampled features; the downsampled features are then processed by a high-level feature extraction module to further extract semantic information and compensate for noise to obtain high-level features; the high-level features are then processed by two upsampling convolution modules to obtain upsampled features; the upsampled features are then processed by an output module to obtain a reversible halftone image. The reversible halftone image is input into the restoration network and passed through several lightweight residual blocks to obtain the restored features; The restored features are passed through a flat convolutional layer to output a restored color image.

6. The reversible halftone method based on dynamic noise according to claim 1, characterized in that, When training the reversible halftone model, the total loss function includes the reversible halftone loss and the recovery loss; The reversible halftone loss includes content loss, tone consistency loss, and spectral loss; The reversible halftone image is processed by a pre-trained inverse halftone model to obtain a restored grayscale image. The restored grayscale image and the grayscale images of the original color image are input into the convolutional network VGG-19. The content loss is constructed based on the feature maps extracted from each layer of the convolutional network VGG-19 based on the grayscale images of the restored grayscale image and the original color image. A tone consistency loss is constructed based on the mean square error calculated from the reversible halftone image and the original color image after passing through a Gaussian filter. Construct a spectral loss based on the spectrum of a reversible halftone image and the ideal spectrum; The recovery loss includes pixel L1 loss and spatial frequency L1 loss; Pixel L1 loss and spatial frequency L1 loss are constructed based on the differences in pixels and spectrum between the restored color image and the original color image, respectively.

7. The reversible halftone method based on dynamic noise according to claim 6, characterized in that, In reversible halftone loss, the content loss is expressed as: ， in, Indicates the spatial location of the feature map. The channel index of the feature map. A grayscale representation of the original color image. This represents the recovered grayscale image obtained from a reversible halftone image using a pre-trained inverse halftone model. and They represent and Feature map extracted from layer l of convolutional network VGG-19; The formula for the loss of tone consistency is expressed as: ， in, This represents a reversible halftone image. Represents the original color image. This represents a Gaussian filter with a convolution kernel size of 11×11. This represents the average operator for all input images in the training set; The formula for the spectral loss is expressed as: ， in, Represents the spectrum of a reversible halftone image. Represents the ideal spectrum. A halftone image representing a constant gray level.

8. The reversible halftone method based on dynamic noise according to claim 6, characterized in that, In the recovery loss, the formula for the pixel L1 loss is expressed as: ， in, Indicates the restoration of a color image. This represents the original color image, where n represents the total number of pixels in the image, and u represents the pixel index. The formula for the spatial frequency L1 loss is expressed as follows: ， in, This represents the Fast Fourier Transform.

9. A reversible halftone system based on dynamic noise, characterized in that, include: The model building module is used to build reversible halftone models, including halftone generation and recovery networks; The halftone generation network includes an input module, a local feature extraction module, two downsampling convolutional modules, a high-level feature extraction module, two upsampling convolutional modules, and an output module connected in sequence. Both the local feature extraction module and the high-level feature extraction module include several feature extraction residual blocks. Each feature extraction residual block includes a dynamic noise module, a ReLU activation function, and a convolutional layer connected in sequence. The input features of the feature extraction residual block and the output features of the ReLU activation function are added together through skip connections to obtain the output features of the feature extraction residual block. The dynamic noise module generates random noise and adaptively adjusts the intensity of the random noise to obtain adaptive random noise. The formula for the dynamic noise module is expressed as follows: ,in This represents the input characteristics of the dynamic noise module. Indicates adaptive random noise, This represents a single-layer convolution; The dynamic noise module adaptively adjusts the intensity of random noise using the following formula: ， in, It is the noise intensity at time step t. It is the initial noise intensity. This is the hue loss value at time step t. This is the initial hue loss value. It's a hyperparameter; The adaptive random noise and the input features of the dynamic noise module are concatenated and then passed through a single-layer convolution to obtain the output features of the dynamic noise module. The recovery network comprises several lightweight residual blocks and flat convolutional layers; The reversible halftone module is used to process the original color image through the halftone generation network of the reversible halftone model and output a reversible halftone image; the reversible halftone image is then processed by the restoration network to output a restored color image.

Images