Differential halftone method and device based on deep self-supervised learning

Abstract

The invention relates to the technical field of halftoning, in particular to a differentiable halftoning method and device based on deep self-supervised learning, solves the problem of non-differentiating caused by halftoning discrete selection by introducing a Gumb l-Softmax module, and realizes gradient unbiased estimation in a model back propagation process. In order to further strengthen the effect of the halftone image, a novel blue noise loss function is introduced, the distribution of halftone dots is optimized, meanwhile, a region confidence aggregation mechanism is put forward, and the network pays more attention to the interaction information between pixels in the training process by combining the spatial correlation of the pixels. According to the method, the high-quality halftone image rich in details can be generated, and the method has remarkable advantages in the aspect of operation efficiency and has good expandability.

Description

Technical Field

[0001] The present invention relates to the technical field of halftone technology, and in particular to a differentiable halftone method and device based on deep self-supervised learning. Background Art

[0002] Halftone is a technology that converts continuous tone images into their approximate discrete versions and is widely used in the printing field. Halftone processing creates different grayscale levels through different arrangements and pixel densities of pixels. Due to the low-pass filtering characteristics of the human visual system, the naked eye perceives halftone images as smooth continuous tone images at a certain observation distance. Halftone algorithms are classified according to the periodicity of pixel arrangement and whether they are clustered or scattered points. The halftone images generated by non-periodic scattered point halftone algorithms are usually composed of evenly distributed and discrete black and white points, and often have blue noise characteristics, which conform to the human eye perception characteristics. Therefore, halftone patterns that match the blue noise properties are the key to generating high-quality halftone images. The implementation of color halftones usually decomposes color images into multiple color channels, and halftones the color intensity information of each channel separately. During the printing or display process, these halftone-processed channels are superimposed in a specific way, and the interaction between the dots will reproduce the color and brightness of the original image. Therefore, the halftone processing method of grayscale images is the basis for expanding to color halftones. Understanding and optimizing grayscale halftone processing is crucial to achieving high-quality color halftones.

[0003] In the research of halftone technology, image quality and processing efficiency are the main research focuses. Common halftone algorithms can be divided into three categories: ordered dithering algorithm, error diffusion algorithm and search-based algorithm. The ordered dithering algorithm systematically divides the continuous tone image into uniform small areas and compares them with the pre-designed or generated dithering matrix. This method provides an effective way for fast image halftone processing with its efficient parallel processing capability and low computational complexity; however, this speed advantage often comes at the expense of the quality of the halftone image. The error diffusion algorithm has made significant progress in visual quality through fine pixel-level processing and error propagation mechanism, but the serial nature of the algorithm and the potential visual artifact problem have become obstacles to further optimization. Search-based methods regard halftone processing as a complex optimization problem. They first define a halftone quality evaluation index that comprehensively considers the characteristics of the human visual system (HVS), and then use heuristic algorithms such as simulated annealing or direct binary search to accurately optimize these evaluation indicators. Since this method directly adjusts specially designed indicators (such as pixel-level mean square error), it can generate halftone images with better quality. However, its high computational cost has become the main obstacle to the widespread application of this technology.

[0004] Deep learning models, especially convolutional neural networks, have attracted much attention due to their outstanding performance in image recognition, generation, and conversion tasks. Supervised learning, unsupervised learning, and reinforcement learning are three common paradigms in deep learning. In halftone processing, convolutional neural networks learn the mapping relationship between paired image data in order to achieve efficient image dithering through a single forward propagation. However, this learning process faces two major challenges:

[0005] (1) Preparing “real” halftone images as training labels is costly, and searching for the best halftone images requires huge computing resources. Existing search-based methods can either only optimize specific metrics or rely on meta-heuristic search algorithms that require parameter tuning for each instance.

[0006] (2) Halftone processing is essentially a one-to-many mapping problem, that is, the same continuous image may correspond to multiple valid halftone representations. If the pixel-level loss function (such as cross entropy) is directly used for optimization, the model may only learn the average representation of the input image, which cannot meet the discrete requirements of halftone processing.

[0007] Therefore, how to achieve halftoning by relying on self-supervision without a large-scale labeled dataset is an important part of halftoning research.

[0008] Another core problem in the application of deep learning to halftone processing stems from the strong binary characteristics of halftone images, which directly leads to the non-differentiability of discrete choices (such as deterministic selection of the best pixel state) during training, thus hindering the back propagation of gradients. Existing research uses generative adversarial networks to train from pre-prepared halftone label datasets, aiming to minimize the Pearson error between the output and the label. Divergence, but because it requires the construction of a large number of labeled data sets, its practicality is limited to a certain extent. On the other hand, the research mainly solves the problem of gradient backpropagation, and one strategy is the gradient pass-through estimator. This method performs actual discrete operations in the forward propagation, and in the backpropagation process, the gradient is directly passed to the input of the discrete variable, ignoring the discontinuity caused by discretization. This method weakens the characteristics of binarization, so it is necessary to introduce a discretization penalty loss to push the output value to the nearest discrete end. However, this greedy binarization rule may damage the optimization from a global perspective. Although the gradient pass-through estimator is simple and easy to use, the inaccuracy of its gradient estimation may affect the optimization quality of the model. The existing technology also uses policy gradient-based methods such as REINFORCE. Since halftone image processing requires sampling of all pixels, its gradient estimation is often accompanied by a high variance through accumulation, resulting in low training efficiency and difficulty in model convergence.

[0009] In summary, the existing half-toning algorithms have the problems of slow processing speed and poor half-toning effect. Summary of the invention

[0010] Therefore, the technical problem to be solved by the present invention is to overcome the problems of slow processing speed of halftone algorithm and poor halftone effect in the prior art.

[0011] To solve the above technical problems, the present invention provides a differentiable halftone method based on deep self-supervised learning, comprising:

[0012] Build a deep differentiable halftone model, including residual network and Gumbel-Softmax module;

[0013] When training the deep differentiable halftone model, a continuously-toned grayscale image and a constant grayscale image are input into the deeply differentiable halftone model, random Gaussian noise is introduced into the continuously-toned grayscale image and the constant grayscale image respectively, a feature map is extracted using a residual network, the feature map output by the residual network is reparameterized using a Gumbel-Softmax module, and halftone images corresponding to the continuously-toned grayscale image and the constant grayscale image are output respectively; the distance between the continuously-toned grayscale image, the constant grayscale image and the halftone image is calculated using a loss function, and the parameters of the residual network are iteratively optimized;

[0014] After the deep differentiable halftone model is trained, the Gumbel-Softmax module is replaced with the argmax function to obtain the target deep differentiable halftone model.

[0015] Halftoning of continuous tone images using a target deeply differentiable halftoning model.

[0016] Preferably, the re-parameterizing the feature map output by the residual network using Gumbel-Softmax comprises:

[0017] Given a discrete random variable, assume it has K categories, and generate an independent Gumbel noise for each category;

[0018] The Softmax function is used to re-parameterize the output feature map of the residual network, discretize it, and output a halftone image.

[0019] Preferably, each pixel in the output feature map generates an independent Gumbel noise under each category, and the formula is:

[0020] G k =-log(-log(U k ))

[0021] Among them, U krepresents the probability distribution of the kth category and obeys the uniform distribution U~Uniform(0,1); G k represents the Gumbel noise of the kth category, k = 1, 2, ..., K;

[0022] The Softmax function is used to reparameterize the output feature map of the residual network and discretize it. The formula is:

[0023]

[0024] Among them, x k is the output value of the kth category, τ is the temperature parameter, π k and π l Represent the probabilities of the kth category and the lth category respectively, G l represents the Gumbel noise of the lth category;

[0025] The maximum output value of all categories is used as the halftone output value of the current pixel.

[0026] Preferably, the loss function of the deeply differentiable halftoning model includes blue noise loss, mean square error loss and structural similarity index loss.

[0027] Preferably, the formula for the blue noise loss is:

[0028] L N =MSE(V c (f ρ ),V h (f ρ ))

[0029] Among them, L N represents the blue noise loss, V c (f ρ ) represents the ideal blue noise spectrum variance, V h (f ρ ) represents the blue noise spectrum variance of the halftone image, and MSE represents the mean square error;

[0030] The formula for the blue noise spectrum variance of the halftone image is:

[0031]

[0032] Among them, V h (f ρ ) represents the blue noise spectrum variance of the halftone image, is the power spectrum of the halftone image, P(f ρ ) represents the radial average power spectral density of the halftone image, f represents the frequency, and f ρ is the radial frequency with a width of ρ, r(fρ ) is f ρ The width of the ring is Δρ = 1, n(r(f ρ )) is r(f ρ ) The number of discrete frequency samples in ;

[0033] The calculation formula of the radial average power spectral density of the halftone image is:

[0034]

[0035] The calculation formula of the power spectrum of the halftone image is:

[0036]

[0037] Where h represents the halftone image output by the Gumbel-Softmax module, N represents the total number of pixels in the image, and DFT represents Fourier transform.

[0038] Preferably, the formula for the mean square error loss is:

[0039] : C =HVS(HVS(h),HVS(c))

[0040] Among them, HVS represents human visual system, h represents halftone image, and c represents continuous grayscale image.

[0041] Preferably, optimizing the mean square error loss includes:

[0042] When calculating the mean square error loss, a regional confidence aggregation mechanism is introduced. A confidence feature map for pixel-by-pixel matching is constructed based on the mean square error of each pixel in the halftone image and the continuous grayscale image. Then, a sampling sub-region is extracted from the confidence feature map to calculate the local descriptor of each pixel.

[0043] The mean square error between the halftone image and the continuous grayscale image is calculated based on the local descriptor of each pixel to obtain the mean square error loss.

[0044] Preferably, the formula of the local descriptor is expressed as:

[0045]

[0046] Among them, D i,j represents the local descriptor of the pixel with coordinates (i, j), N(i, j) represents the sampling sub-region centered on the pixel with coordinates (i, j), (u, v) represents the pixel in the uth row and vth column of the sampling sub-region, and w u,v represents the convolution filter weight of the pixel in the uth row and vth column of the sampling sub-region, F i+u,j+vRepresents the mean square error of the pixel with coordinates (i+u,j+v).

[0047] Preferably, the sampling sub-regions of the region confidence aggregation mechanism are consistent with the filter window of the human visual system.

[0048] The present invention also provides a differentiable halftone device based on deep self-supervised learning, comprising:

[0049] Model building module for building a deep differentiable halftone model, including residual network and Gumbel-Softmax module;

[0050] A training module, which is used to train the deep differentiable halftone model, input a continuous grayscale image and a constant grayscale image into the deep differentiable halftone model, introduce random Gaussian noise into the continuous grayscale image and the constant grayscale image respectively, extract a feature map using a residual network, re-parameterize the feature map output by the residual network using a Gumbel-Softmax module, and output halftone images corresponding to the continuous grayscale image and the constant grayscale image respectively; calculate the distance between the continuous grayscale image, the constant grayscale image and the halftone image using a loss function, and iteratively optimize the parameters of the residual network;

[0051] A halftone module is used to replace the Gumbel-Softmax module with the argmax function after the deep differentiable halftone model is trained to obtain the target deep differentiable halftone model; the continuous tone image is halftoned using the target deep differentiable halftone model.

[0052] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0053] The invention discloses a differentiable halftone method based on deep self-supervised learning. In the training process of the deep differentiable halftone model, a Gumbel-Softmax module is introduced. The reparameterization strategy of Gumbel-Softmax is used to simulate the sampling process of discrete distribution through a smooth differentiable approximation method, thereby solving the non-differentiable problem caused by the discrete selection of halftones. In the training process, effective unbiased gradient estimation is achieved, and the dependence on label data is eliminated. The halftone evaluation metric is directly optimized. The trained target deep differentiable halftone model can not only generate high-quality halftone images rich in details, but also has significant advantages in computational efficiency and good scalability.

[0054] Furthermore, the present invention optimizes the blue noise loss in the loss function of the deep differentiable halftone model, and significantly suppresses the anisotropy of the constant grayscale image during the training phase by quantifying and comparing the mean square error of the blue noise spectrum variance of the halftone image output by the deep differentiable halftone model and the ideal blue noise spectrum variance, thereby promoting the natural generation of blue noise characteristics. In addition, the present invention introduces a regional confidence aggregation mechanism in the mean square error loss, and effectively simulates the perceptual characteristics of the human visual system by comprehensively considering the interdependence between pixels, thereby maintaining the local structural information and texture details of the image, enhancing the model's information extraction of local area details and global structure, and improving the quality of the halftone image. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0056] Figure 1 is a structural diagram of the depth differentiable halftone model of the present invention;

[0057] Figure 2 is the halftone effect of each method on the snail image, where Figure 2 (a) is a continuous grayscale image of the snail. Figure 2 (b) in the figure is the halftone image output by the VAC method. Figure 2 (c) in the figure is the halftone image output by the OVED method. Figure 2 (d) in the figure is the halftone image output by the DBS method. Figure 2 (e) in the figure is the halftone image output by the RVH method. Figure 2 (f) in the figure is the halftone image output by the TRH method. Figure 2 (g) in the figure is a halftone image output by the method of the present invention without using the regional confidence aggregation mechanism. Figure 2 (h) is a halftone image output by the method of the present invention;

[0058] Figure 3 is the halftone effect of each method on the real butterfly image, where Figure 3 (a) is a continuous grayscale image of a real butterfly image. Figure 3 (b) in the figure is the halftone image output by the VAC method. Figure 3 (c) in the figure is the halftone image output by the OVED method. Figure 3 (d) in the figure is the halftone image output by the DBS method. Figure 3 (e) in the figure is the halftone image output by the RVH method. Figure 3 (f) in the figure is the halftone image output by the TRH method. Figure 3(g) in the figure is a halftone image output by the method of the present invention without using the regional confidence aggregation mechanism. Figure 3 (h) is a halftone image output by the method of the present invention;

[0059] Figure 4 is a comparison of the halftoning results of different constant grayscale images and the radial average power spectral density and anisotropy measure, where Figure 4 The (a) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 20. Figure 4 The (b) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 80. Figure 4 The (c) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 100. Figure 4 Column (d) in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 127;

[0060] Figure 5 is the Fourier amplitude spectrum of an image with a constant grayscale level of 127 without and with blue noise loss, where Figure 5 (a) is the Fourier amplitude spectrum without using blue noise loss. Figure 5 (b) in the figure is the Fourier amplitude spectrum using blue noise loss;

[0061] Figure 6 It is the MSE loss curve of various deep learning methods;

[0062] Figure 7 It is the optimization result diagram under different weight parameter values ​​of structural similarity index loss;

[0063] Figure 8 These are two five-level halftone example results. Figure 8 The (a) column in is a continuous grayscale image. Figure 8 The (b) column in the figure is the five-level halftone result image. Figure 8 Column (c) in is the output line chart. DETAILED DESCRIPTION

[0064] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0065] Embodiment 1

[0066] Reference Figure 1As shown, the present invention proposes a differentiable halftone method based on deep self-supervised learning, comprising:

[0067] Build a deep differentiable halftone model, including residual network and Gumbel-Softmax module;

[0068] When training the deep differentiable halftone model, a continuously-toned grayscale image and a constant grayscale image are input into the deeply differentiable halftone model, random Gaussian noise is introduced into the continuously-toned grayscale image and the constant grayscale image respectively, a feature map is extracted using a residual network, the feature map output by the residual network is reparameterized using a Gumbel-Softmax module, and halftone images corresponding to the continuously-toned grayscale image and the constant grayscale image are output respectively; the distance between the continuously-toned grayscale image, the constant grayscale image and the halftone image is calculated using a loss function, and the parameters of the residual network are iteratively optimized;

[0069] After the deep differentiable halftone model is trained, the Gumbel-Softmax module is replaced with the argmax function to obtain the target deep differentiable halftone model.

[0070] Halftoning of continuous tone images using a target deeply differentiable halftoning model.

[0071] The following is a detailed introduction to the training process of the deep differentiable halftone model.

[0072] Since convolutional neural networks have a convolutional paradigm with spatially shared kernels, the inductive bias they bring about is prone to flatness degradation. Specifically, the convolution of a constant signal s(x)≡b with an arbitrary kernel function k(x) is still a constant signal Where μ(k(x)) represents the mean of the kernel function.

[0073] Therefore, given a flat input X, no matter what the parameters of the convolutional neural network are, the operation of the convolutional neural network will degenerate into a scaling operation Y = αX. Therefore, the present invention uses the Gaussian noise map as a spatial variation and introduces it into the feature space to provide sufficient jitter dependency for the model without destroying the integrity of the original input information.

[0074] The input of the deeply differentiable halftoning model is defined by the following formula:

[0075] I c =Concatenate(c,z)

[0076]

[0077] Among them, c is a continuous grayscale image, c gis a constant grayscale image, z is a Gaussian noise image; Concatenate is a concatenation operation; I c is a noisy continuous grayscale image, is a noise constant grayscale image. The random noise introduced by the Gaussian noise image helps the deep differentiable halftone model focus on the statistical distribution characteristics of the overall pattern rather than a single pixel value.

[0078] The noisy continuous grayscale image and the noisy constant grayscale image are input into the residual network, and the feature map is extracted using the residual network. The residual network contains multiple residual blocks, each of which processes the input features and adds the processed feature map to the original input. These residual blocks help to effectively transfer information in the deep network, thereby avoiding the gradient vanishing problem, so that the network maintains performance stability when learning deep features.

[0079] The Gumbel-Softmax module is used to reparameterize the feature map output by the residual network to achieve unbiased estimation of the gradient, and output the halftone image h corresponding to the continuous grayscale image and the constant grayscale image respectively.

[0080] The Gumbel-Softmax module aims to solve the problem that the reparameterization technique cannot be directly applied to discrete data. This strategy is based on two insights: the Gumbel distribution can be used to parameterize discrete distributions; the argmax function itself is not continuous, and by using the Softmax function controlled by the temperature parameter, a continuous and differentiable approximation can be provided for the discrete selection process.

[0081] Specifically, in order to discretize the output feature map of the residual network, it is necessary to give a discrete random variable X, assuming that it has K categories, and the probabilities of these K categories are π1, π2, ..., π K , the unnormalized log probability of each class is expressed as logπ k .

[0082] Each pixel in the output feature map generates an independent Gumbel noise under each category, the formula is:

[0083] G k =-log(-log(U k ))

[0084] Among them, U k represents the probability distribution of the kth category and obeys the uniform distribution U~Uniform(0,1); G k represents the Gumbel noise of the kth category, k = 1, 2, …, K.

[0085] The traditional halftone method is to add Gumbel noise to the unstandardized logarithmic probability of each category, and then characterize the discrete random variable X through the argmax function. The formula is:

[0086]

[0087] In the processing of discrete variables in the traditional method, the discrete result based on the argmax function is discontinuous. In order to facilitate the gradient calculation during the training process, the present invention adopts the Softmax function based on the temperature parameter to re-parameterize the output feature map of the residual network, discretize it, and output the halftone image. The formula is:

[0088]

[0089] Among them, x k is the output value of the kth category, τ is the temperature parameter, π k and π l Represent the probabilities of the kth category and the lth category respectively, G l represents the Gumbel noise of the lth category.

[0090] The maximum output value of all categories is used as the halftone output value of the current pixel.

[0091] The present invention converts the probability π1, π2, ..., π K The dependency is transferred from the non-differentiable random sampling function to the differentiable function composed of Softmax and log operations, and its internal mechanism cleverly continues the noise benefit introduced in the initial Concatenate operation. That is, by introducing noise based on Gumbel distribution, it not only effectively simulates and extends the random influence of the initial noise, but also further enhances the model's ability to process discrete outputs.

[0092] Due to the discreteness of halftones, the image after filtering by the human visual system (HVS) can effectively simulate the perception characteristics of the human eye. The present invention further performs end-to-end optimization on this basis.

[0093] In order to comprehensively evaluate and optimize the quality of the halftone images generated by the model, the loss function of the deep differentiable halftone model includes blue noise loss, mean square error loss and structural similarity index loss, and the formula is:

[0094] L total =L C +w1·L N +w2·L S

[0095] Among them, L total is the loss function of the deeply differentiable halftone model, LC is the mean square error loss, L N is the blue noise loss, w1 is the weight parameter of the blue noise loss, L S is the structural similarity index loss, and w2 is the weight parameter of the structural similarity index loss.

[0096] In this embodiment, w1=0.02, w2=0.01, which are empirically set values.

[0097] The loss function of the deep differentiable halftoning model is introduced in detail below.

[0098] In halftone processing, the blue noise texture pattern has non-periodic characteristics while avoiding low-frequency graininess, which is consistent with human eye perception. Halftones that conform to blue noise should maintain the following spectral characteristics: (1) contain fewer low-frequency components; (2) the energy distribution in the high-frequency area is relatively flat; (3) the anisotropy is very low at all frequencies. Previous studies have shown that when processing constant grayscale images, simply relying on reducing the mean square error loss and structural similarity loss of the image after filtering through the human visual system is not enough to ensure good blue noise quality. In addition, the parallel processing characteristics of convolutional neural networks are prone to cause globally consistent checkerboard artifacts, which further leads to the degradation of halftone image quality. Existing studies have penalized the low-frequency components of halftone images through discrete cosine transform. Although these components have been minimized to a certain extent, the problem of excessive anisotropy has not been significantly solved. This excessive anisotropy may visually lead to unwelcome texture bias.

[0099] In order to optimize the blue noise characteristics of the generated halftone image and evaluate its quality and avoid the model's tendency to extreme output, the present invention introduces a power spectrum variance-based measurement method to construct a blue noise loss during the model training process.

[0100] First, the power spectrum of the halftone image output by the Gumbel-Softmax module is calculated by discrete Fourier transform (DFT), and the formula is:

[0101]

[0102] Where h represents the halftone image output by the Gumbel-Softmax module, N represents the total number of pixels in the image, and DFT represents Fourier transform. is the power spectrum of the halftone image.

[0103] Calculate the radial average power spectral density of the halftone image using the formula:

[0104]

[0105] Among them, P(f ρ) represents the radial average power spectral density of the halftone image, f ρ is the radial frequency with a width of ρ, r(f ρ ) is f ρ The width of the ring is Δρ = 1, n(r(f ρ )) is r(f ρ ), where f represents the frequency.

[0106] Calculate the blue noise spectrum variance, the formula is:

[0107]

[0108] Among them, V h (f ρ ) represents the blue noise spectrum variance of the halftone image.

[0109] Since the variance is used to measure the degree of data dispersion, it can explicitly characterize the distribution characteristics and fluctuation degree of halftone dots. The present invention designs a blue noise loss, and the formula is:

[0110] L N =MSE(V c (f ρ ),V h (f ρ ))

[0111] Among them, V c (f ρ ) represents the ideal blue noise spectrum variance, V h (f ρ ) represents the blue noise spectrum variance of the halftone image, and MSE represents the mean square error.

[0112] The blue noise loss is a quantitative comparison of the mean square error between the blue noise spectrum variance of the halftone image output by the deep differentiable halftone model and the ideal blue noise spectrum variance, which can accurately evaluate the effect of the model in maintaining the blue noise properties of the image. Since spectral analysis is only meaningful for halftone images with constant grayscale, the present invention optimizes the loss function on additional small batches of constant grayscale images.

[0113] The mean square error loss is used to calculate the mean square error between the halftone image and the continuous grayscale image after filtering based on the human visual system. The formula is:

[0114] L C =MSE(HVS(h),HVS(c))

[0115] Wherein, HVS represents human visual system, h represents halftone image, and c represents continuous grayscale image. The filter window size of the human visual system is 11×11.

[0116] The structural similarity index loss is used to evaluate the similarity between the halftone image and the continuous grayscale image, and the formula is:

[0117] L S =1-SSIM(h,c)

[0118] Among them, SSIM represents the structural similarity index, h represents the halftone image, and c represents the continuous grayscale image.

[0119] During the training process, the loss function L total As the final optimization goal of the model, the model is guided to optimize the visual similarity, structural similarity and blue noise characteristics of the image at the same time during the learning process. In this way, not only can a halftone image that is visually similar to the original image be generated, but it can also ensure that it has ideal blue noise characteristics, thus achieving the optimal balance in visual effects.

[0120] In the halftone image quality assessment, existing metrics such as mean square error and structural similarity index can quantify the differences between images, but they are often limited by the one-sided pursuit of pixel-level accuracy and ignore the impact of pixel-to-pixel interactions on the overall perceived quality. These metrics preprocess the target image and the reference image through the human visual system model to simulate the sensitivity of the human eye to image details, and then generate an error map reflecting the image difference and average it to obtain a scalar metric result. The performance of a pixel depends not only on its own value, but also on the influence of other pixels in its local window, and this range of influence is determined by the window size limited by the HVS filter. However, only by directly optimizing the average accuracy through generalized mean pooling, the gradient of each network parameter is simplified to the average value of the gradient at all pixels. This processing method treats the confidence matching costs of all involved pixel pairs equally, and fails to fully consider the key impact of the pixel arrangement pattern in the halftone image on the quality of the generated image.

[0121] In view of the above limitations, the present invention introduces a regional confidence aggregation mechanism when calculating the mean square error loss, which is used to abandon the practice of examining pixels in isolation and instead take a more global perspective. The confidence of local pixels is aggregated into a unified clustering center for processing. The purpose is to more comprehensively optimize the texture and structural details of the image by comprehensively considering the quality evaluation of the local area where the pixels are located.

[0122] Specifically, the regional confidence aggregation mechanism constructs a confidence feature map for pixel-by-pixel matching based on the mean square error of each pixel in the halftone image and the continuous grayscale image, and then extracts the sampling sub-region from the confidence feature map to calculate the local descriptor of each pixel. Each descriptor not only contains the information of the current pixel, but also integrates the confidence values ​​of all pixels in its sampling sub-region. At the same time, the regional confidence aggregation mechanism introduces learnable convolution filter weights to adjust the contribution of different pixels in the aggregation process, ensuring that the model can adaptively capture the local features of the image.

[0123] The formula of the local descriptor is expressed as:

[0124]

[0125] Among them, D i,j represents the local descriptor of the pixel with coordinates (i, j), N(i, j) represents the sampling sub-region centered on the pixel with coordinates (i, j), (u, v) represents the pixel in the uth row and vth column of the sampling sub-region, and w u,v represents the convolution filter weight of the pixel in the uth row and vth column of the sampling sub-region, F i+u,j+v Represents the mean square error of the pixel with coordinates (i+u,j+v).

[0126] Preferably, the sampling sub-region of the regional confidence aggregation mechanism is consistent with the HVS filter window, so that the model can be within the same perception range as the HVS, thereby more effectively aggregating the confidence information in the neighborhood. In this embodiment, the size of the sampling sub-region is 11x11.

[0127] At the same time, the convolution filter weights can be regarded as part of the convolution layer in the model and can be learned together with the network parameters. In order to reach the convergence target faster, the initial convolution filter weight value is set to a Gaussian distribution with a standard deviation of 2. Under this mechanism, the decision output of the model is not only based on the current pixel point, but also better captures the contextual information of the image by marking the correlation between pixels, thereby enhancing the spatial perception ability of the model.

[0128] The mean squared error loss calculates the mean squared error between the halftone image and the continuous-tone grayscale image based on the local descriptor of each pixel calculated by the region confidence aggregation mechanism.

[0129] In summary, the present invention introduces the Gumbel-Softmax module in the training process of the deep differentiable halftone model, utilizes the reparameterization strategy of Gumbel-Softmax, and simulates the sampling process of discrete distribution through a smooth differentiable approximation method, thereby solving the non-differentiable problem caused by discrete selection of halftones, and realizes effective unbiased gradient estimation in the training process, breaking away from the dependence on label data, and directly optimizing the halftone evaluation metric. The trained deep differentiable halftone model can not only generate high-quality halftone images rich in details, but also has significant advantages in computational efficiency, can be applied to scenes such as multi-level halftones, has good scalability, and provides an effective solution for image halftones.

[0130] Furthermore, the present invention optimizes the blue noise loss in the loss function of the deep differentiable halftone model, and significantly suppresses the anisotropy of the constant grayscale image during the training phase by quantifying and comparing the mean square error of the blue noise spectrum variance of the halftone image output by the deep differentiable halftone model and the ideal blue noise spectrum variance, thereby promoting the natural generation of blue noise characteristics. In addition, the present invention introduces a regional confidence aggregation mechanism in the mean square error loss, and effectively simulates the perceptual characteristics of the human visual system by comprehensively considering the interdependence between pixels, thereby maintaining the local structural information and texture details of the image, enhancing the model's information extraction of local area details and global structure, and improving the quality of the halftone image.

[0131] Embodiment 2

[0132] In order to verify the effectiveness of the method proposed in the present invention, this embodiment conducts relevant ablation experiments by comparing the performance differences between the method of the present invention and the existing halftone method, and explores its scalability in different application scenarios.

[0133] 1. Experimental setup

[0134] This embodiment uses the VOC2012 dataset for training and evaluation, randomly selects 13758 images as the training set, and retains 1684 and 1683 images as the validation set and test set for model performance evaluation. The present invention is based on self-supervised learning. By designing a specific loss function, the model does not need to rely on labeled data during training, so that a large number of unlabeled grayscale images can be used for learning. In order to ensure the consistency and comparability of the experiment, all images are converted to grayscale images before processing, and the proposed method is comprehensively compared with other existing methods. In order to further prove the effectiveness and generalization ability of the model, it is tested on multiple public datasets, including Set5, Set14, B100, B200, General100, Manga109, Urban100, and DIV2K datasets.

[0135] The present invention adopts the residual network as the backbone network of halftone. In the training, the batch size is set to 64, and the training image is randomly cropped to 64x64. During the training process, the loss function L is minimized. total To train the network, the hyperparameters w1 = 0.02, w2 = 0.01, and the initial temperature coefficient τ of the reparameterization is set to 0.5. The entire model is trained using the ADAM optimizer, and the learning rate α is adjusted from 3e-4 to 1e-5 through the cosine annealing schedule. The entire training process is performed on an NVIDIA RTX 2080Ti GPU for 400 rounds.

[0136] 2. Halftone quality evaluation

[0137] In order to comprehensively evaluate the performance of the proposed method in the halftone image generation task, this embodiment performs a systematic quantitative evaluation and visual quality analysis, and compares a variety of existing halftone methods, including Void-and-cluster method (VAC), Ostromoukhov's method (OVED), optimization-based search method (DBS), and deep neural network-based methods (RVH and TRH), where the present invention / foundation represents the method of the present invention that does not adopt the regional confidence aggregation mechanism.

[0138] The quantitative results of all methods on the testing dataset are detailed in Table 1. The tone consistency is measured by the peak signal-to-noise ratio (PSNR) between the HVS-filtered halftone and the input continuous tone, and the structural similarity index (SSIM) is used to evaluate the structure and texture information of the halftone image.

[0139] Figure 2 is the halftone effect of each method on the snail image, where Figure 2 (a) is a continuous grayscale image of the snail. Figure 2 (b) in the figure is the halftone image output by the VAC method. Figure 2 (c) in the figure is the halftone image output by the OVED method. Figure 2 (d) in the figure is the halftone image output by the DBS method. Figure 2 (e) in the figure is the halftone image output by the RVH method. Figure 2 (f) in the figure is the halftone image output by the TRH method. Figure 2 (g) in the figure is a halftone image output by the method of the present invention without using the regional confidence aggregation mechanism. Figure 2 (h) in FIG. 1 is a halftone image output by the method of the present invention.

[0140] Table 1 Quantitative comparison of halftone methods

[0141]

[0142] Experimental results show that the proposed method achieves competitive scores in PSNR and achieves the best performance in SSIM. It is worth noting that although the DBS method achieves the highest PSNR value by optimizing the search strategy, a very high PSNR does not mean a better halftone effect visually, and the extreme pursuit of PSNR index will sacrifice image structure details. Figure 2 (d) shows that the result generated by the DBS method visually loses local details. The method of the present invention pays more attention to the balance between image structure texture and tone consistency.

[0143] Table 1 also compares the number of parameters of different methods and the running time on the 512x512 pixel "Lenna" test image. The results show that although the VAC method has the shortest running time, the halftone image it generates is of relatively low quality. On the other hand, the search-based method (DBS) has a long running time due to computationally intensive operations, which limits its feasibility in practical applications. In contrast, the deep neural network-based method can generate halftone images with richer details and higher quality. At the same time, the method of the present invention is superior to the RVH and TRH methods in reducing the number of parameters and shortening the running time, showing higher computational efficiency and lower time overhead. Overall, the method proposed in the present invention is in a leading position in image quality assessment and computational cost.

[0144] from Figure 2 It can be seen that VAC, OVED and DBS fail to effectively preserve the vein structure details of the image because they ignore the structural similarity of the image when generating the halftone image, which causes these methods to perform poorly when processing edge details, making the halftone image over-blurred and losing details. Although these methods can suppress artifacts to a certain extent, this is usually at the expense of subtle textures and edge information. In contrast, the method of the present invention effectively balances the relationship between artifact suppression and detail preservation, and more accurately reflects the structural characteristics of the original image. Not only does it perform well in suppressing artifacts, but it also avoids excessive blurring and successfully preserves the key features of the image, which makes the method of the present invention significantly superior to other methods in the generation quality of halftone images.

[0145] Figure 3 is the halftone effect of each method on the real butterfly image, where Figure 3 (a) is a continuous grayscale image of a real butterfly image. Figure 3 (b) in the figure is the halftone image output by the VAC method. Figure 3 (c) in the figure is the halftone image output by the OVED method. Figure 3(d) in the figure is the halftone image output by the DBS method. Figure 3 (e) in the figure is the halftone image output by the RVH method. Figure 3 (f) in the figure is the halftone image output by the TRH method. Figure 3 (g) in the figure is a halftone image output by the method of the present invention without using the regional confidence aggregation mechanism. Figure 3 (h) in the figure is the halftone image output by the method of the present invention. It can be observed that obvious checkerboard grid artifacts appear in the halftone images generated by the OVED, RVH and TRH methods, and OVED has relatively serious oblique texture features. Although this pattern visually presents relatively low low-frequency features, it does not fully meet the ideal characteristics of blue noise. The RVH and TRH methods have point aggregation phenomena, that is, pixel points tend to gather in certain areas. Such point distribution does not belong to the ideal random uniform distribution, which further affects the overall visual effect of the halftone image. While optimizing the low-frequency features, these methods fail to effectively deal with the blue noise distribution and point dispersion problems. In contrast, the halftone image generated by the method of the present invention not only effectively avoids the generation of checkerboard grid patterns, but also significantly reduces the point aggregation phenomenon, and can generate a halftone image with a more natural texture.

[0146] In order to verify the effectiveness of the proposed method, this embodiment conducts comparative experiments on the PSNR and SSIM indicators of different methods on multiple public test sets. The quantitative indicator comparison is shown in Table 2.

[0147] Table 2 Test results of public datasets (SSIM / PSNR)

[0148]

[0149]

[0150] As can be seen from Table 2, the method of the present invention shows the best SSIM index and good PSNR index on each test set, which verifies that the method of the present invention has obvious advantages in maintaining image details and structural integrity, and also shows that it has good generalization.

[0151] 3. Ablation experiment

[0152] In order to achieve the differentiability of the network framework during the binarization process, the RVH and TRH methods use a straight-through estimator to implement the gradient backpropagation during the binarization process and propose a binarization loss. Although this method promotes discrete selection to some extent, the existence of the binarization loss limits the optimization of the overall image quality. In contrast, the present invention adopts a strategy based on Gumbel-Softmax reparameterization, which allows the model to explore a continuous decision space during training while retaining the discrete selection features. Not only does it achieve the differentiability of the discrete selection process, it also enhances the training stability and efficiency of the model, thereby significantly improving the optimization effect of image quality evaluation.

[0153] In the halftoning process, blue noise characteristics are the key factors affecting the quality of halftoning images. Figure 4 is a comparison of the halftoning results of different constant grayscale images and the radial average power spectral density and anisotropy measure, where Figure 4 The (a) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 20. Figure 4 The (b) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 80. Figure 4 The (c) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant gray level of 100. Figure 4 The (d) column in the figure is the halftone result map, radial average power spectrum density map and anisotropy map of the image with a constant grayscale level of 127. In the image halftone result map, the left, middle and right are the halftone effects generated by disabling blue noise loss, using blue noise loss and DBS method respectively. It can be clearly seen that the halftone image without blue noise loss has more prominent stripe artifacts, while the use of this loss can significantly solve the stripe artifact problem. At the same time, the optimized halftone image also presents ideal blue noise characteristics in the radial frequency power spectrum, that is, fewer low-frequency components and higher and smoothly transitioned mid-high frequency components. In addition, the anisotropy curve of the halftone spectrum with blue noise characteristics is also relatively flat, avoiding sharp frequency peaks or abrupt changes. This smooth frequency response not only improves the visual quality of the image, but also effectively suppresses possible visual artifacts.

[0154] Figure 5 is the Fourier amplitude spectrum of an image with a constant grayscale level of 127 without and with blue noise loss, where Figure 5 (a) is the Fourier amplitude spectrum without using blue noise loss. Figure 5 (b) in the figure is the Fourier amplitude spectrum using blue noise loss. Figure 5It can be clearly observed in (a) that the halftone dot distribution has a clear directional preference when blue noise loss is not used, while after using blue noise loss, this preference is significantly reduced, showing a more ideal blue noise characteristic.

[0155] The method of the present invention can more comprehensively capture and utilize the confidence information in the local area by introducing the regional confidence aggregation mechanism. Table 1 shows that the SSIM and PSNR scores of the mechanism on the test dataset are improved.

[0156] Figure 6 It is the MSE loss curve diagram of various deep learning methods, which further shows that by comprehensively considering the confidence information between pixels, the method of the present invention can converge more quickly, thereby improving the training efficiency.

[0157] In addition, to further analyze the impact of model hyperparameters on performance, this embodiment performs a sensitivity analysis on the hyperparameter w2 in the loss function. Figure 7 It is the optimization result diagram under different weight parameter values ​​of structural similarity index loss. In this embodiment, w2=0.01 is selected, and the generated halftone image performs well between structural clarity and consistency. According to different preferences for image quality, a higher PSNR score can be obtained by reducing w2, but this may be at the expense of structural and texture details.

[0158] 4. Multi-level halftone

[0159] Although the method of the present invention is based on the basic two-level halftone problem, its flexibility and extensibility enable it to adapt to more complex image processing scenarios, especially the expansion of multi-level halftone. Multi-level halftone technology provides more dot options for each area of ​​the image by increasing the grayscale level, thereby visually achieving a smoother and continuous tone simulation.

[0160] To illustrate this point, this embodiment provides a specific example of multi-level halftone. By adjusting the number of output channels of the model and using the Gumbel-Softmax technique, the one-hot vector output by it is weighted summed with the predefined multi-level halftone level weights. Since the proposed blue noise loss is only optimized for binary discretization output, the blue noise solution is not used, and the desired discrete halftone result is finally obtained. The experiment shows that the SSIM and PSNR scores of the five-level halftone on the VOC test set are 0.3216 / 42.825 respectively. Figure 8 These are two five-level halftone example results. Figure 8 The (a) column in is a continuous grayscale image. Figure 8 The (b) column in the figure is the five-level halftone result image. Figure 8Column (c) in is the output line graph. It can be observed in the output line graph that the output of the model is still concentrated in the discrete grayscale center.

[0161] Although the extended solution proposed in this invention shows feasibility in the implementation of multi-level halftone algorithm, in order to further improve the image quality, several additional complex dimensions must be taken into account, including the selection of a suitable blue noise model to optimize visual quality and effectively mitigate stripe artifacts. In order to achieve these goals, these considerations need to be integrated into the existing framework in order to achieve deeper optimization and improvement. In view of the intertwined complexity of multi-dimensional problems, more sophisticated theoretical analysis and experimental verification are required, and these in-depth research works are regarded as the direction of future research.

[0162] In summary, the present invention aims at the limitations of the current digital halftone algorithm, such as slow processing speed and poor halftone effect. By introducing the reparameterization strategy of the Gumbel-Softmax module, the non-differentiable problem caused by the discrete selection of halftones is solved, and the unbiased estimation of gradients in the network back propagation process is realized. In order to further enhance the effect of the halftone image, a new blue noise loss function is designed to optimize the distribution of halftone dots. At the same time, a regional confidence aggregation module is proposed, which combines the spatial correlation of pixels to make the model pay more attention to the interactive information between pixels during the training process. Based on the above strategy, a self-supervised deep differentiable halftone model without label guidance is constructed by optimizing the expected value of halftone quality evaluation. The experimental results show that the method proposed by the present invention does not require image labels, and can generate high-quality halftone images while maintaining high processing speed and low parameter complexity, and effectively retain the local structural information and texture details of the image. In addition, the model can be flexibly extended to multi-level halftone processing to meet the needs of multi-level print heads.

[0163] Embodiment 3

[0164] Based on the differentiable halftoning method based on deep self-supervised learning described in Embodiment 1, this embodiment provides a differentiable halftoning device based on deep self-supervised learning, including:

[0165] Model building module for building a deep differentiable halftone model, including residual network and Gumbel-Softmax module;

[0166] A training module, which is used to train the deep differentiable halftone model, input a continuous grayscale image and a constant grayscale image into the deep differentiable halftone model, introduce random Gaussian noise into the continuous grayscale image and the constant grayscale image respectively, extract a feature map using a residual network, re-parameterize the feature map output by the residual network using a Gumbel-Softmax module, and output halftone images corresponding to the continuous grayscale image and the constant grayscale image respectively; calculate the distance between the continuous grayscale image, the constant grayscale image and the halftone image using a loss function, and iteratively optimize the parameters of the residual network;

[0167] A halftone module is used to replace the Gumbel-Softmax module with the argmax function after the deep differentiable halftone model is trained to obtain the target deep differentiable halftone model; the continuous tone image is halftoned using the target deep differentiable halftone model.

[0168] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. A differentiable halftoning method based on deep self-supervised learning, characterized in that include: Build a deep differentiable halftone model, including residual network and Gumbel-Softmax module; When training the deep differentiable halftone model, a continuously-toned grayscale image and a constant grayscale image are input into the deeply differentiable halftone model, random Gaussian noise is introduced into the continuously-toned grayscale image and the constant grayscale image respectively, a feature map is extracted using a residual network, the feature map output by the residual network is reparameterized using a Gumbel-Softmax module, and halftone images corresponding to the continuously-toned grayscale image and the constant grayscale image are output respectively; the distance between the continuously-toned grayscale image, the constant grayscale image and the halftone image is calculated using a loss function, and the parameters of the residual network are iteratively optimized; After the deep differentiable halftone model is trained, the Gumbel-Softmax module is replaced with the argmax function to obtain the target deep differentiable halftone model. Halftoning of continuous tone images using a target deeply differentiable halftoning model.

2. A differentiable halftoning method based on deep self-supervised learning according to claim 1, characterized in that: The method of re-parameterizing the feature map output by the residual network using Gumbel-Softmax includes: Given a discrete random variable, assume it has K categories, and generate an independent Gumbel noise for each category; The Softmax function is used to re-parameterize the output feature map of the residual network, discretize it, and output a halftone image.

3. A differentiable halftoning method based on deep self-supervised learning according to claim 2, characterized in that: Each pixel in the output feature map generates an independent Gumbel noise for each category, the formula is: G k =-log(-log(U k )) Among them, U k represents the probability distribution of the kth category and obeys the uniform distribution U~Uniform(0,1); G k represents the Gumbel noise of the kth category, k = 1, 2, ..., K; The Softmax function is used to reparameterize the output feature map of the residual network and discretize it. The formula is: Among them, x k is the output value of the kth category, τ is the temperature parameter, π k and π l Represent the probabilities of the kth category and the Ith category respectively, G l represents the Gumbel noise of the Ith category; The maximum output value of all categories is used as the halftone output value of the current pixel.

4. A differentiable halftoning method based on deep self-supervised learning according to claim 1, characterized in that: The loss functions of the deeply differentiable halftoning model include blue noise loss, mean square error loss, and structural similarity index loss.

5. A differentiable halftoning method based on deep self-supervised learning according to claim 4, characterized in that: The formula for the blue noise loss is: L N =MSE(V c (f ρ ),V h (f ρ )) Among them, L N represents the blue noise loss, V c (f ρ ) represents the ideal blue noise spectrum variance, V h (f ρ ) represents the blue noise spectrum variance of the halftone image, and MSE represents the mean square error; The formula for the blue noise spectrum variance of the halftone image is: Among them, V h (f ρ ) represents the blue noise spectrum variance of the halftone image, is the power spectrum of the halftone image, P(f ρ ) represents the radial average power spectral density of the halftone image, f represents the frequency, and f ρ is the radial frequency with a width of ρ, r(f ρ ) is f ρ The width of the ring is Δρ = 1, n(r(f ρ )) is r(f ρ ) The number of discrete frequency samples in ; The calculation formula of the radial average power spectral density of the halftone image is: The calculation formula of the power spectrum of the halftone image is: Where h represents the halftone image output by the Gumbel-Softmax module, N represents the total number of pixels in the image, and DFT represents Fourier transform.

6. A differentiable halftoning method based on deep self-supervised learning according to claim 4, characterized in that: The formula for the mean square error loss is: L C =MSE(HVS(h),HVS(c)) Among them, HVS represents human visual system, h represents halftone image, and c represents continuous grayscale image.

7. A differentiable halftoning method based on deep self-supervised learning according to claim 6, characterized in that: Optimizing the mean square error loss includes: When calculating the mean square error loss, a regional confidence aggregation mechanism is introduced. A confidence feature map for pixel-by-pixel matching is constructed based on the mean square error of each pixel in the halftone image and the continuous grayscale image. Then, a sampling sub-region is extracted from the confidence feature map to calculate the local descriptor of each pixel. The mean square error between the halftone image and the continuous grayscale image is calculated based on the local descriptor of each pixel to obtain the mean square error loss.

8. A differentiable halftoning method based on deep self-supervised learning according to claim 7, characterized in that: The formula of the local descriptor is expressed as: Among them, D i，j represents the local descriptor of the pixel with coordinates (i, j), N(i, j) represents the sampling sub-region centered on the pixel with coordinates (i, j), (u, v) represents the pixel in the uth row and vth column of the sampling sub-region, and w u，v represents the convolution filter weight of the pixel in the uth row and vth column of the sampling sub-region, F i+u，j+v Represents the mean square error of the pixel with coordinates (i+u, j+v).

9. The differentiable halftoning method based on deep self-supervised learning according to claim 7, characterized in that: The sampling sub-regions of the region confidence aggregation mechanism are consistent with the filter windows of the human visual system.

10. A differentiable halftoning device based on deep self-supervised learning, characterized in that: include: Model building module for building a deep differentiable halftone model, including residual network and Gumbel-Softmax module; A training module, which is used to train the deep differentiable halftone model, input a continuous grayscale image and a constant grayscale image into the deep differentiable halftone model, introduce random Gaussian noise into the continuous grayscale image and the constant grayscale image respectively, extract a feature map using a residual network, re-parameterize the feature map output by the residual network using a Gumbel-Softmax module, and output halftone images corresponding to the continuous grayscale image and the constant grayscale image respectively; calculate the distance between the continuous grayscale image, the constant grayscale image and the halftone image using a loss function, and iteratively optimize the parameters of the residual network; A halftone module is used to replace the Gumbel-Softmax module with the argmax function after the deep differentiable halftone model is trained to obtain the target deep differentiable halftone model; the continuous tone image is halftoned using the target deep differentiable halftone model.

Images