Inverse halftoning method and device based on conditional diffusion network
By combining the blue noise characteristics and Laplace priors, a diverse halftone image is generated, which solves the generalization problem of the U-net method in different jitter modes, and improves the repair quality and robustness of the inverse halftone.
Patent Information
- Application Number
- CN202211551244.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2022-10-13
- Filing Date
- 2022-12-05
- Publication Date
- 2025-08-15
- Estimated Expiration
- 2042-12-05
AI Technical Summary
The existing U-net-based inverse halftone method has poor generalization ability under different jitter modes, resulting in artifacts in restored continuous tone images.
Using a method based on conditional diffusion network, reverse jitter diffusion is performed by calculating image grayscale level and Laplace prior, blue noise characteristics are introduced, half-tone images in a variety of jitter form are generated, and inverse half-tone process learning is performed in the U-net network.
Improves the quality of print repair, reduces artifacts, and enhances the generalization ability and robustness of the inverse halftone model.
Smart Images
Figure CN115861105B_ABST
Abstract
Description
Technical Field
[0001] The invention designs an inverse halftone method and device based on a conditional diffusion network, and belongs to the field of computer vision. Technical Background
[0002] Halftoning is the process of simulating the brightness variations of a continuous-tone image by varying the size or frequency of halftone dots (e.g., ink dots). Over the past century, halftoning technology has been widely used in the printing industry to store valuable image data in media such as old newspapers, books, and photographs. Simultaneously, inverse halftoning technology has emerged to restore stored continuous-tone images from vintage materials. The goal of inverse halftoning is to minimize information loss during the restoration process, ensuring the highest possible visual quality in the restored print.
[0003] Traditional inverse halftoning methods are mainly based on image filtering (e.g., wavelet domain filtering, edge-preserving filtering, SUSAN filtering, bilateral filtering) and statistical learning methods (e.g., minimum mean square filtering, maximum a posteriori, lookup tables, dictionary learning, etc.). With the resurgence of deep learning technology, inverse halftoning methods based on deep neural networks have made great progress and received increasing attention. The most representative method is based on the U-net architecture, which has an encoder to learn the hidden code of the halftone print, followed by a decoder to reconstruct the image.
[0004] However, the supervised learning paradigm with element-wise regression commonly adopted in U-net based methods has poor generalization ability in practical applications. Specifically, when there is a large gap between the dithering patterns of the training and test halftones, obvious artifacts often appear in the restored continuous tone image. This is an important problem in practical applications because the algorithms used to generate halftones continue to evolve over time. Even for the same algorithm, different parameter selections will lead to different halftone dithering patterns. Taking frequency modulated (FM) halftone as an example, the present invention selects 9 classic error diffusion processes with different dithering patterns, namely Floyd-Steinberg Dithering, Jarvis-Judice-Ninke Dithering, Stucki Dithering, Atkinson Dithering, Burkes Dithering, Sierra Dithering and several of their variants. The present invention trains the U-net model on halftones generated by 5 algorithms and tests it on halftones obtained by the remaining 4 algorithms. The experimental results are shown in the attached figure. Figure 1 The artifacts can be clearly observed in the restored continuous-tone image. Summary of the Invention
[0005] In order to solve the above problems, the present invention designs an inverse halftoning method and device based on a conditional diffusion network. The method generates halftone images with diverse dithering forms through a conditional diffusion network, and then learns the inverse halftoning process based on the generated halftone images, thereby improving the restoration quality of printed products.
[0006] The technical contents of the present invention include:
[0007] An inverse halftoning method based on a conditional diffusion network, the method comprising:
[0008] Calculate image gray levels and Laplace priors for halftone images;
[0009] Under the image grayscale and the Laplace prior, for at least one initial state Perform reverse dither diffusion for T time steps to generate a halftone distribution; wherein the initial state Select from a random Gaussian distribution;
[0010] Under the halftone distribution, for at least one initial state Perform inverse halftone diffusion for T′ time steps to obtain an inverse halftone result of the halftone image; wherein the initial state Select from a random Gaussian distribution.
[0011] Furthermore, under the image grayscale and the Laplace prior, for each initial state Perform inverse dither diffusion for T time steps to generate a halftone distribution consisting of:
[0012] Constructing a training set; the training set includes a plurality of halftone image samples;
[0013] Compute the sample state vector for each halftone image sample Sample image grayscale and sample Laplace prior;
[0014] For the sample state vector Perform jitter diffusion for T time steps to obtain the sample state vector
[0015] Select several sample initial states from random Gaussian distribution
[0016] Injecting blue noise dither characteristics into the sample state vector Get the eigenvector o;
[0017] The eigenvector o is updated after being cascaded with the image grayscale and Laplace prior along the channel dimension;
[0018] The updated feature vector o is input into the U-net network to obtain the reverse dithering diffusion noise predicted at the tth time step;
[0019] Construct the objective function based on the noise that obeys the standard normal distribution and the reverse dithering diffusion noise
[0020] Based on the objective function Train an inverse jitter diffusion model to make the inverse jitter diffusion model approach the conditional probability
[0021] For the initial state The state vector corresponding to time step t The state vector Updated after concatenation with the image grayscale and Laplace prior along the channel dimension;
[0022] The updated state vector Input the trained reverse jitter diffusion model to obtain the state vector of time step t-1
[0023] Based on the initial state The corresponding state vector Make up halftone distribution.
[0024] Furthermore, the blue noise jitter characteristic is injected into the state vector Get the eigenvector o, including:
[0025] Use the pre-trained VGG network to extract the state vector Hidden features of
[0026] Generate k halftone vectors m i , to construct the meta-halftone set
[0027] The hidden features are learned through an affine learning layer and the meta halftone set The affine relationship between them, and then get the affine factor
[0028] Using the affine factor About the Meta Halftone Set Perform deep aggregation to obtain a refined representation of the meta-halftone
[0029] Refinement of the meta halftone representation and the state vector Perform spatial aggregation to obtain the meta-halftone guidance vector
[0030] Guided by the Meta Halftone and the jitter diffusion state vector Determine the eigenvector o.
[0031] Furthermore, the pair of the feature vectors Perform jitter diffusion for T time steps to obtain the state vector include:
[0032] Get a beta t Random Gaussian noise with controlled variance and noise ∈ following standard normal distribution;
[0033] Calculate the variable α that controls the noise intensity at time step t during the jitter diffusion process t =1-β t , and the noise intensity variable multiplied at time step t
[0034] Calculate the state vector
[0035] Furthermore, the generation of k element halftone vectors m i , to construct the meta-halftone set include:
[0036] Construct k images I with a large range of low-frequency areas i ;
[0037] Using a halftoning algorithm to pass the image I i Obtain the corresponding halftone H i ;
[0038] From the halftone H i Construct halftone sample h i , and from image I i Get image grayscale and Laplace prior
[0039] Based on the halftone sample h i , the image grayscale and Laplace prior Perform reverse jitter diffusion and predict the mean of the parameterized model Get the predicted noise
[0040] Based on the predicted noise Calculate the element halftone vector m i .
[0041] Construct a halftone vector m containing k elements i Meta Halftone Set
[0042] Furthermore, the objective function Among them, ∈ represents noise, I represents the identity matrix, represents the conditional probability.
[0043] Furthermore, under the halftone distribution, the initial state Performing inverse halftone diffusion for T′ time steps to obtain an inverse halftone result of the halftone image, comprising:
[0044] Constructing a training set; the training set includes a plurality of halftone image samples;
[0045] Compute the sample feature vector for each halftone image sample
[0046] For the sample feature vector Perform T ′ The jitter diffusion of time steps is used to obtain the sample state vector
[0047] Select several initial states from a random Gaussian distribution
[0048] The feature vector Update after concatenating with the halftone distribution along the channel dimension;
[0049] The updated feature vector Input U-net network and get the tth ′ The inverse halftone diffusion noise predicted by the time step; the objective function is constructed based on the noise obeying the standard normal distribution and each inverse halftone diffusion noise
[0050] Based on the objective function Train an inverse halftone diffusion model so that the inverse halftone diffusion model approximates the conditional probability Wherein, h represents the halftone distribution;
[0051] For the initial state The corresponding time step t ′ The state vector The state vector Update after concatenating with the halftone distribution along the channel dimension;
[0052] The updated state vector Input the trained inverse halftone diffusion model Get time step t ′ -1 state vector
[0053] Based on the state vector An inverse halftone result of the halftone image is obtained.
[0054] An inverse halftoning device based on a conditional diffusion network, the device comprising:
[0055] Preprocessing module for calculating image grayscale and Laplace prior of halftone image;
[0056] The reverse dither diffusion module is used to adjust at least one initial state under the image grayscale and the Laplace prior. Perform reverse dither diffusion for T time steps to generate a halftone distribution; wherein the initial state Select from a random Gaussian distribution;
[0057] The reverse halftone diffusion module is used to adjust at least one initial state under the halftone distribution. Perform T ′ time steps of inverse halftone diffusion to obtain the inverse halftone result of the halftone image; wherein the initial state Select from a random Gaussian distribution.
[0058] An electronic device, comprising:
[0059] processor;
[0060] a memory for storing instructions executable by the processor;
[0061] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the above-mentioned inverse halftoning method based on the conditional diffusion network.
[0062] A computer-readable storage medium stores computer program instructions, wherein the program instructions are executed by a processor to implement the steps of the above-mentioned inverse halftoning method based on a conditional diffusion network.
[0063] Compared with the prior art, the present invention has at least the following advantages:
[0064] 1. This is the first invention to propose a generative halftone method. This invention treats the black pixels in the halftone as physically moving particles. Through the reverse diffusion process, the randomly distributed particles are guided to move, thereby obtaining the desired halftone dither pattern.
[0065] 2. In order to avoid introducing redundant patterns and unwanted artifacts in the halftone generation process, this paper proposes a meta-halftone guidance network to integrate blue noise guidance into the halftone diffusion process.
[0066] 3. To achieve better generalization capability, the present invention uses the x0 state of halftone dither diffusion as the condition for inverse halftone diffusion, so that the inverse halftone diffusion model benefits from a wider range of dither patterns and learns a more robust mapping. BRIEF DESCRIPTION OF THE DRAWINGS
[0067] Figure 1 Illustration of artifacts in the inverse halftoning result produced by a traditional inverse halftoning algorithm.
[0068] Figure 2 FIG. 4 is a flow chart of the inverse halftoning method of the present invention.
[0069] Figure 3 Schematic diagram of the proposed meta-halftone guidance network.
[0070] Figure 4 This is a schematic diagram of the conditional diffusion model for inverse halftoning proposed by the present invention. DETAILED DESCRIPTION
[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the embodiments described are only specific embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0072] Traditional halftone dithering methods are mainly based on amplitude modulation and frequency modulation techniques. With the development of deep learning, some modern methods try to use convolutional neural networks to generate halftone images. However, these methods are limited to specific dithering patterns and cannot achieve flexible halftone generation. In order to solve this problem, considering that halftone prints are composed of only black and white pixels, the present invention regards the black pixels in the halftone as moving particles, so that these moving particles move from a random Gaussian distribution to a halftone distribution under a reverse dithering diffusion process, thereby realizing a halftone generation method. It should be noted that in order to more clearly illustrate the technical solution of the present invention, the following content uses They represent the state vector of the reverse halftone dither diffusion process and the state vector of the reverse inverse halftone diffusion process during the test, respectively. They represent the state vector of the dither diffusion process during training, the state vector of the inverse halftone diffusion process, the state vector of the reverse dither diffusion process, and the state vector of the reverse inverse halftone diffusion process respectively.
[0073] Specifically, the method flow chart of the present invention is as follows: Figure 2 As shown, the following steps are included.
[0074] Step 1: Calculate the image grayscale and Laplace prior of the halftone image.
[0075] For a halftone image requiring inverse halftoning, the present invention uses the halftone image's grayscale level r and Laplace prior l as conditions guiding the reverse diffusion process, given that particles tend to rely on the image's gradient information when forming a halftone pattern. Therefore, the present invention requires first obtaining the halftone image's grayscale level and Laplace prior.
[0076] Step 2: Under the image grayscale and Laplace prior, for at least one initial state Perform inverse dither diffusion for T time steps to generate the halftone distribution.
[0077] The present invention first selects the initial state from the random Gaussian distribution Then, with the help of the trained reverse jitter diffusion model, reverse diffusion is performed for T time steps to obtain the state vector Finally, based on all initial states The generated state vector Get halftone distribution. Different from the existing technology, the state vector of the t-th time step is The U-net network is directly input. In order to generate high-quality halftones, the present invention also introduces blue noise characteristics into the state vector of the U-net network.
[0078] Specifically, the generation of halftone distribution includes the following steps:
[0079] Step 2.1: Dither diffusion of halftone image samples.
[0080] The present invention first constructs a plurality of halftone image samples, and then calculates the sample state vector of each halftone image sample. The sample image grayscale and sample Laplace prior, then the sample state vector Perform jitter diffusion for T time steps to obtain the sample state vector Where s1 represents the halftone dither diffusion stage.
[0081] Dither diffusion is to perform halftone dither diffusion on halftone image samples. Black pixel distribution Diffusion process is a Markov noise process, which gradually moves towards Add noise to obtain a noisy image for T time steps
[0082] The specific implementation of the dither diffusion can be achieved by adding β in the diffusion step at each time step t. t Random Gaussian noise with controlled variance is used to obtain the sample state vector The process of adding noise is as follows:
[0083]
[0084]
[0085] Among them, β t ∈(0,1),t=1,…,T,I is the identity matrix.
[0086] In addition, the jitter diffusion can also use the reparameterization technique to obtain the To take a sample:
[0087]
[0088]
[0089] The noise intensity variable α at time step t t =1-β t , the noise intensity variable is multiplied at time step t In this way, the present invention can be directly Middle pair Sampling is performed without the need to repeatedly employ a Markov process and calculation
[0090] Step 2.2: Sample initial state Reverse jitter diffusion.
[0091] In the halftone dithering scene, the present invention hopes to use the reverse diffusion process to From a random Gaussian distribution (i.e. state ) to sample halftones Considering that particles tend to rely on the gradient information of the image when forming halftone patterns, the present invention uses the image grayscale r and Laplace prior l of the halftone image sample as the conditions to guide the reverse diffusion process. Formally, the reverse halftone diffusion process is defined in the distribution This is a Markov chain with the initial state The inverse jitter diffusion model can be defined as:
[0092]
[0093]
[0094] in, represents the mean of the model predictions, represents the variance of the model predictions.
[0095] Unlike previous methods that use class labels or shape latent representations as diffusion model conditions, this paper not only uses pixel-level image priors as conditions, which are implemented in a simple manner, but also introduces a blue noise feature. This blue noise feature avoids obvious low-frequency visual artifacts in the generated halftone by forcing random pixel dithering, which is crucial for generating high-quality halftones.
[0096] In summary, in one embodiment of the present invention, the reverse jitter diffusion may include the following sub-steps.
[0097] Step 2.2.1: Select several sample initial states from a random Gaussian distribution
[0098] Assume a random Gaussian distribution The present invention considers the simplest random Gaussian distribution here, that is, the mean of the Gaussian distribution is 0 and the variance is 1, then the Gaussian distribution can be expressed as Random sampling is performed from the Gaussian distribution, and the sample obtained is recorded as It can be written as
[0099] Step 2.2.2: Inject the blue noise dithering characteristic into the sample state vector Get the eigenvector o.
[0100] Previous research works have proposed some traditional methods to derive blue noise dither patterns, such as simulated annealing-based methods, void-and-cluster techniques, power spectrum operation algorithms, and dither pattern sorting methods. However, most of these methods are based on statistical methods or manually designed blue noise dither matrices, which are difficult to directly apply to neural frameworks based on learning and optimization. Recently, some researchers have tried to add blue noise properties to the L1 norm as a model learning target, but since the calculation of the L1 norm requires known halftone generation results, it cannot be used to generate models. So far, how to incorporate blue noise dither characteristics into halftone generation models remains an unexplored problem. In order to solve this problem, the present invention proposes a meta-halftone guided network (such as Figure 3 As shown in Figure 2), it introduces blue noise into the halftone dither diffusion process to avoid the dither result containing artifacts or redundant patterns. Formally, for step t of the halftone dither diffusion process, the present invention considers the state vector It is used as the input of the meta-halftone guidance network to obtain the state vector o containing the blue noise dithering characteristics.
[0101] The meta-halftone guidance network of the present invention comprises: a hidden feature extraction module, a halftone set construction module, an affine learning module, and a depth and space aggregation module.
[0102] 1) Hidden feature extraction module.
[0103] The present invention first Input feature extraction network E to obtain the extracted hidden features Right now In the experiment, the present invention uses the pre-trained VGG network as E. Other networks such as InceptionNet or ResNet can also be used. The purpose of using pre-training is to save computing resources and training time.
[0104] 2) Halftone set building blocks.
[0105] The present invention defines a meta-halftone set It consists of a set of halftone vectors m i Composition (1≤i≤k):
[0106]
[0107] Wherein, k represents the number of elementary halftones. The present invention obtains m by dithering and diffusing the halftone state of a group of low-frequency images. i Specifically, the present invention constructs k images I1, I2, I3, ..., I with a large range of low-frequency areas. k (such as an image with a constant grayscale), and use a traditional halftone algorithm (such as the Floyd-Steinberg algorithm) to obtain the corresponding halftones H1, H2, H3, ..., H k The present invention first trains several steps using traditional diffusion models alone, where the common U-net architecture is used without adding the proposed meta-halftone guidance network. The prediction result can be expressed as:
[0108]
[0109] The superscript h i Indicates that from H i ∈{H1,H2,H3,…,H k} constructed halftone sample, and Can be from I i ∈{I1,I2,I3,…,I k} is obtained accordingly. For the sake of brevity, The superscript s1 is omitted here. The present invention can predict the mean value through the parameterized model Get model prediction noise
[0110]
[0111] The present invention uses m i To represent the model prediction And construct the meta-halftone set accordingly
[0112]
[0113] 3) Affine learning module.
[0114] The model learns hidden features through an affine learning layer and element halftone set The affine relationship between them, and then get the affine factor
[0115]
[0116] where w g←i and b g←i denote the learnable weights and biases respectively.
[0117] 4) Depth and spatial aggregation module.
[0118] Use the calculated The present invention is to set the element halftone Perform deep aggregation to learn refined representations on depth channels:
[0119]
[0120] In addition to depth information, spatial information is also important for meta-halftone guidance. Meta-halftone can provide guidance for the generation of new dither patterns in local areas. In view of this, the present invention will refine the representation of meta-halftone. and the jitter diffusion state vector Perform spatial aggregation and express the results as
[0121]
[0122] in Represents the element-wise concatenation in the spatial dimension, * represents the spatial convolution operation, and d represents the convolution kernel. The output o of the meta-halftone guided network is guided by the meta-halftone. and the jitter diffusion state vector The feature vector o is then fed into the U-net for model prediction as in previous work.
[0123] Step 2.2.3: Update the feature vector o by concatenating it with the image grayscale and Laplace prior of the halftone image sample along the channel dimension.
[0124] The present invention concatenates the feature vector o with r and l along the channel dimension at the diffusion step t to provide pixel-level guidance:
[0125]
[0126] in represents the connection operation. Due to the characteristics of halftone, local dithering patterns are more important than global semantic information for generating high-quality halftone.
[0127] Step 2.2.4: Input the updated feature vector o into the U-net network to obtain the reverse dithering diffusion noise predicted at the tth time step; construct the objective function based on a noise that obeys a standard normal distribution and each reverse dithering diffusion noise Based on the objective function Train an inverse jitter diffusion model to make the inverse jitter diffusion model approach the conditional probability
[0128] The present invention intends to reverse the jitter diffusion model Approximating conditional probability According to Bayes' theorem, we have
[0129]
[0130] can be represented as a Gaussian distribution:
[0131]
[0132]
[0133]
[0134] A simplified objective function is used to train the model in the halftone dither diffusion process:
[0135]
[0136] Step 2.3: Initial State Reverse jitter diffusion.
[0137] Similar to the above steps 2.1 and 2.2, during the test, the present invention first selects the initial state from the random Gaussian distribution Then, for the initial state The state vector corresponding to time step t The state vector After concatenation with the image grayscale and Laplace prior along the channel dimension, the updated state vector Input the trained reverse jitter diffusion model to obtain the state vector of time step t-1 Finally, it is approximately assumed that the initial states The corresponding state vector A halftone distribution h.
[0138] Step 3: Under the halftone distribution, for at least one initial state Perform T ′ The inverse halftone diffusion is performed for time steps to obtain an inverse halftone result of the halftone image.
[0139] The goal of the inverse halftone process is to learn the mapping from halftone distribution to continuous tone distribution and reduce the information loss in the process. It is similar to the halftone dither diffusion process, but the diffusion conditions are different from the inverse halftone diffusion process. From continuous tone distribution Continuous-tone image Adding noise, and the process Gradually denoise from Gaussian noise to obtain the desired continuous tone image.
[0140] Specifically, the present invention also needs to first train the inverse halftone diffusion model by constructing a training set.
[0141]
[0142]
[0143] Unlike the inverse dither diffusion model, the blue noise dither feature is not injected during training, and the inverse halftone diffusion model is used to estimate Therefore, the loss function used by the inverse halftone diffusion model for:
[0144]
[0145] Next, the initial state is selected from a random Gaussian distribution Afterwards, the corresponding time step t ′ The state vector After being updated by concatenation with the halftone distribution h along the channel dimension, the trained inverse halftone diffusion model is input. Get time step t ′ -1 state vector
[0146] Finally, based on the state vector Get the inverse halftone result of the halftone image
[0147] In summary, for the halftone dithering process, the present invention trains a halftone dithering conditional diffusion model to generate halftones with different dithering patterns under image grayscale and Laplace prior conditions. By changing the initial state of the diffusion model, it can simulate different dithering processes to generate diverse halftone images. To avoid introducing redundant patterns and unwanted artifacts in the halftone generation process, the present invention proposes a meta-halftone guidance network that incorporates blue noise guidance into the diffusion process. For the inverse halftone process, the present invention trains an inverse halftone diffusion model to learn the mapping function from the halftone distribution to the continuous tone distribution. In this way, halftones with more diverse distributions are input into the inverse halftone model, which helps the model learn a more robust mapping and improves its generalization ability for unseen samples.
[0148] The inverse halftoning device of the present invention includes a conditional diffusion model of inverse halftoning, such as Figure 4 As shown, the present invention consists of a halftone dithering diffusion process and an inverse halftone diffusion process. This method treats black pixels in the halftone as physically moving particles. Through the inverse diffusion process, these randomly distributed particles are guided to move, thereby achieving the desired halftone distribution. By inputting generated halftones with diverse dithering patterns into the inverse halftone diffusion process, the robustness and generalization capabilities of the inverse halftone diffusion module are enhanced.
[0149] Although the specific implementation process and example drawings of the present invention are disclosed for illustrative purposes, their purpose is to facilitate understanding of the content of the present invention and to facilitate implementation accordingly, those skilled in the art will appreciate that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the illustrated implementation process and example drawings.
Claims
1. An inverse halftoning method based on a conditional diffusion network, characterized in that: The method comprises: Calculate image gray levels and Laplace priors for halftone images; Under the image grayscale and the Laplace prior, for at least one initial state Perform reverse dither diffusion for T time steps to generate a halftone distribution; wherein the initial state Select from a random Gaussian distribution; Under the halftone distribution, for at least one initial state Perform T ′ time steps of inverse halftone diffusion to obtain the inverse halftone result of the halftone image; wherein the initial state Select from a random Gaussian distribution.
2. The method according to claim 1, wherein Under the image grayscale and the Laplace prior, for each initial state Perform inverse dither diffusion for T time steps to generate a halftone distribution consisting of: Constructing a training set; the training set includes a plurality of halftone image samples; Compute the sample state vector for each halftone image sample Sample image grayscale and sample Laplace prior; For the sample state vector Perform jitter diffusion for T time steps to obtain the sample state vector Select several sample initial states from random Gaussian distribution Injecting blue noise dither characteristics into the sample state vector Get the eigenvector o; The eigenvector o is updated after being cascaded with the image grayscale and Laplace prior along the channel dimension; The updated feature vector o is input into the U-net network to obtain the reverse dithering diffusion noise predicted at the tth time step; Construct the objective function based on the noise that obeys the standard normal distribution and the reverse dithering diffusion noise Based on the objective function Train an inverse jitter diffusion model to make the inverse jitter diffusion model approach the conditional probability For the initial state The state vector corresponding to time step t The state vector Updated after concatenation with the image grayscale and Laplace prior along the channel dimension; The updated state vector Input the trained reverse jitter diffusion model to obtain the state vector of time step t-1 Based on the initial state The corresponding state vector Make up halftone distribution.
3. The method according to claim 2, wherein The blue noise dither characteristic is injected into the state vector Get the eigenvector o, including: Use the pre-trained VGG network to extract the state vector Hidden features of Generate k halftone vectors m i , to construct the meta-halftone set The hidden features are learned through an affine learning layer and the meta halftone set The affine relationship between them, and then get the affine factor Using the affine factor About the Meta Halftone Set Perform deep aggregation to obtain a refined representation of the meta-halftone Refinement of the meta halftone representation and the state vector Perform spatial aggregation to obtain the meta-halftone guidance vector Guided by the Meta Halftone and the jitter diffusion state vector Determine the eigenvector o.
4. The method according to claim 3, wherein The pair of the feature vectors Perform jitter diffusion for T time steps to obtain the state vector include: Get a beta t Random Gaussian noise with controlled variance and noise ∈ following standard normal distribution; Calculate the variable α that controls the noise intensity at time step t during the jitter diffusion process t =1-β t , and the noise intensity variable multiplied at time step t Calculate the state vector 5. The method according to claim 4, wherein The k-element halftone vector m is generated i , to construct the meta-halftone set include: Construct k low-frequency region images I i ; Using a halftoning algorithm to pass the image I i Obtain the corresponding halftone H i ; From the halftone H i Construct halftone sample h i , and from image I i Get image grayscale and Laplace prior Based on the halftone sample h i , the image grayscale and Laplace prior Perform reverse jitter diffusion and predict the mean of the parameterized model Get the predicted noise Based on the predicted noise Calculate the element halftone vector m i; Construct a halftone vector m containing k elements i Meta Halftone Set 6. The method according to claim 5, wherein The objective function Among them, ∈ represents noise, I represents the identity matrix, represents the conditional probability.
7. The method according to claim 1, wherein Under the halftone distribution, the initial state Perform T ′ time steps of inverse halftone diffusion to obtain an inverse halftone result of the halftone image, comprising: Constructing a training set; the training set includes a plurality of halftone image samples; Compute the sample feature vector for each halftone image sample For the sample feature vector Perform T ′ The jitter diffusion of time steps is used to obtain the sample state vector Select several initial states from a random Gaussian distribution The feature vector Update after concatenating with the halftone distribution along the channel dimension; The updated feature vector Input U-net network and get the tth ′ The inverse halftone diffusion noise predicted by the time step; the objective function is constructed based on the noise obeying the standard normal distribution and each inverse halftone diffusion noise Based on the objective function Train an inverse halftone diffusion model so that the inverse halftone diffusion model approximates the conditional probability For the initial state The corresponding time step t ′ The state vector The state vector Update after concatenating with the halftone distribution along the channel dimension; The updated state vector Input the trained inverse halftone diffusion model to get the time step t ′ -1 state vector Based on the state vector An inverse halftone result of the halftone image is obtained.
8. An inverse halftoning device based on a conditional diffusion network, characterized in that: The device comprises: Preprocessing module for calculating image grayscale and Laplace prior of halftone image; The reverse dither diffusion module is used to adjust at least one initial state under the image grayscale and the Laplace prior. Perform reverse dither diffusion for T time steps to generate a halftone distribution; wherein the initial state Select from a random Gaussian distribution; The reverse halftone diffusion module is used to adjust at least one initial state under the halftone distribution. Perform T ′ time steps of inverse halftone diffusion to obtain the inverse halftone result of the halftone image; wherein the initial state Select from a random Gaussian distribution.
9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the inverse halftoning method based on the conditional diffusion network as described in any one of claims 1 to 7.
10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the inverse halftoning method based on a conditional diffusion network described in any one of claims 1 to 7 are implemented.