Printing-shooting process image degradation simulation method, device and equipment based on image-to-image diffusion model

Abstract

The invention discloses a printing-shooting process image degradation simulation method, device and equipment based on a graph-to-graph diffusion model, and aims to solve the problems that an existing degradation simulation method is greatly different from a real physical process and cannot be accurately controlled. The method comprises the following steps: firstly, constructing a printing-shooting data set containing a plurality of printing parameters and shooting parameters; then coding the physical parameters into conditional control vectors, and injecting the conditional control vectors into a Unet network of a graph-to-graph diffusion model for training; when an image is generated, an innovative double-flow noise layer structure is adopted, the structure combines a first branch adopting a traditional digital simulation method and a second branch adopting the pre-training diffusion model in parallel, and one of the first branch and the second branch is selected to be output according to a preset probability. According to the method, a highly vivid degraded image can be generated, so that the robustness and accuracy of downstream tasks (such as deep watermarking and image recognition) in a real printing-shooting scene are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and particularly to a method, device and equipment for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model. Background Art

[0002] The printing - shooting process is a common link in image applications. For example, scenarios such as document digitization, image dissemination, and reuse all involve this process. In the printing stage, due to factors such as printer nozzle characteristics, ink or toner diffusion, and paper material texture, problems such as detail loss, color deviation, or blurring may occur in the image. In the shooting stage, optical system aberrations of the camera, lens distortion, changes in shooting environment lighting, and sensor noise further exacerbate the image degradation. These degraded images often lead to a significant decline in the effect in subsequent processing (such as image recognition or watermark extraction).

[0003] Taking the deep watermark technology as an example, the watermark embedded in the image is often damaged during the image degradation process, resulting in the failure of watermark extraction, which seriously affects the robustness and security of the deep watermark algorithm. Currently, for the simulation methods of image degradation in the printing - shooting process, most are based on simple mathematical simulations and empirical formulas. Such methods are difficult to comprehensively and accurately depict the multi - source and variable degradation mechanisms in the actual scenario. The simulated degraded images are quite different from the images in the real scenario and cannot provide high - quality and realistic sample data for the training of related image processing technologies such as deep watermarking or digital recognition, thus limiting the performance improvement of related technologies in practical applications.

[0004] In recent years, as a deep - learning model based on probabilistic generation, the diffusion model can model complex data distributions by gradually adding noise in the data space and learning the denoising process. The graph - to - graph diffusion model is an extended application in the field of image conversion, aiming to achieve content - style mapping and migration between different image feature spaces. Essentially, the image degradation caused by the printing - shooting process can also be regarded as a complex image feature conversion process. Therefore, the graph - to - graph diffusion model with conditional control ability, relying on its ability to model complex data and image feature conversion ability, provides an ideal technical path for simulating printing - shooting image degradation. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a method, device and equipment for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model, which can simulate the complex image degradation introduced by the printing - shooting process in a high - fidelity and controllable manner.

[0006] In the first aspect, the present application provides a method for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model, including:

[0007] Construct a printing - shooting dataset covering multiple scenarios, where the dataset includes original images and degraded images corresponding to the original images, and each group of degraded images is associated with the printing parameters and shooting parameters that cause their degradation;

[0008] Construct a conditional control vector based on the printing parameters and shooting parameters to parametrically control the degradation process;

[0009] Train a graph - to - graph diffusion model based on the dataset. The training process includes injecting the conditional control vector into the Unet network module in the model so that the model learns the mapping relationship from the original image to the degraded image specified by the conditional control vector;

[0010] Generate a final degraded image from an input image using a two - stream noise layer. The two - stream noise layer includes: a first branch that processes the input image using traditional digital simulation methods to generate a first degraded image; a second branch that processes the input image using the trained graph - to - graph diffusion model to generate a second degraded image; and selecting one of the first degraded image and the second degraded image as the final degraded image according to a preset probability.

[0011] Optionally, the printing parameters include DPI, ink amount, and paper type; the shooting parameters include illuminance, shooting angle, and shooting distance.

[0012] Optionally, the step of constructing the conditional control vector includes: performing normalization processing on continuous physical parameters; mapping categorical physical parameters to dense vectors through a learnable linear layer; and concatenating all processed parameter vectors to form the final conditional control vector.

[0013] Optionally, the step of injecting the conditional control vector into the Unet network module is specifically: in each network layer of the Unet network module, up - sample the conditional control vector to the same spatial size as the input feature map of this network layer; and fuse the up - sampled conditional control vector into the input feature map through channel - level addition.

[0014] Optionally, the traditional digital simulation method in the first branch sequentially includes performing image compression, geometric transformation, blurring operation, and color adjustment on the image.

[0015] Optionally, the step of training the graph - to - graph diffusion model further includes: optimizing the model using a hybrid loss function that combines mean squared error loss, perceptual loss based on the VGG network, and adversarial loss.

[0016] In a second aspect, the present application provides a print - photograph process image degradation simulation device based on a graph - to - graph diffusion model, including:

[0017] A dataset construction module, configured to construct a print - photograph dataset covering multiple scenarios, the dataset including original images and corresponding degraded images, where each group of degraded images is associated with specific print and photograph parameters that cause their degradation;

[0018] A conditional control vector construction module, based on the print parameters and photograph parameters, constructs a conditional control vector for parametric control;

[0019] A model training module, configured to train a graph - to - graph diffusion model based on the dataset, which injects the conditional control vector into the Unet network module in the model, so that the model learns the mapping relationship from the original image to the degraded image specified by the conditional control vector;

[0020] An image generation module, configured to generate a final degraded image by using a two - stream noise layer including a trained graph - to - graph diffusion model and a traditional digital simulation. The two - stream noise layer includes: a first branch, which processes the input image by using a traditional digital simulation method to generate a first degraded image; a second branch, which processes the input image by using the trained graph - to - graph diffusion model to generate a second degraded image; and selects one of the first degraded image and the second degraded image as the final degraded image according to a preset probability.

[0021] In a third aspect, the present application provides a print - photograph process image degradation simulation device based on a graph - to - graph diffusion model, including: a memory, configured to store a computer program; a processor, configured to execute the computer program to implement the foregoing print - photograph process image degradation simulation method based on a graph - to - graph diffusion model.

[0022] As can be seen from the above, the present application has the following beneficial effects:

[0023] Achieved controllable degradation simulation: By encoding print and photograph physical parameters as conditional control vectors and injecting them into the diffusion model, this method can dynamically and accurately generate corresponding degradation effects according to specified parameter combinations, and has the generalization ability across devices and scenarios.

[0024] Generated high - fidelity degraded images: Utilizing the powerful distribution learning ability of the graph - to - graph diffusion model, it can capture complex noise and degradation patterns in the real world. The generated images are highly realistic in terms of details and global structure, providing high - quality training data for downstream tasks (such as deep watermarking, image recognition).

[0025] Improved the robustness of downstream tasks: Innovatively designed a two-stream noise layer, which integrates the physical interpretability of traditional digital simulation and the data-driven complexity of diffusion models. The generated degraded samples are both realistic and diverse, thus significantly enhancing the robustness and accuracy of related algorithms in actual application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flowchart of the printing-photographing degradation simulation method disclosed in this application.

[0027] Figure 2 It is an overall framework diagram of the printing-photographing degradation simulation method disclosed in this application.

[0028] Figure 3 It is a schematic diagram of the injection of the conditional control vector into the Unet network disclosed in this application.

[0029] Figure 4 It is a schematic diagram of the structure of the two-stream noise layer disclosed in this application.

[0030] Figure 5 It is a schematic diagram of the structure of the traditional digital simulation layer disclosed in this application.

[0031] Figure 6 It is a module diagram of the printing-photographing process image degradation simulation device disclosed in this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0033] Refer to Figure 1 , the embodiments of the present invention disclose a printing-photographing process image degradation simulation method based on a graph-to-graph diffusion model, which is implemented in the python language in this example, and the model is built on the Pytorch deep learning framework. The overall conceptual framework of this method is as shown in Figure 2 , which describes the complete process from the real physical process to the dataset construction and then to the model simulation output.

[0034] Step S1: Construct a printing - shooting dataset; In this embodiment, first, construct a printing - shooting dataset covering multiple scenarios. This dataset contains 2000 pairs of original - degraded image pairs, comprehensively covering key degradation factors such as lighting, shooting distance, angle, and different printing device parameters. Specifically, the printing parameters include: DPI, with optional values {300, 600, 1200}; ink volume, in the range [70%, 130%]; paper type, with optional values {glossy, matte, newsprint}. The shooting parameters include: illuminance, with optional values {200, 500, 1000} lux; shooting angle, in the range [60°, 120°]; shooting distance, in the range [5 cm, 30 cm], and the formula is:

[0035]

[0036] Step S2: Construct a conditional control vector based on printing and shooting parameters; Based on the physical parameters collected in Step S1, construct a conditional control vector c. This process includes three steps: normalization, embedding encoding, and splicing fusion:

[0037] 1) Perform normalization processing on continuous parameters such as ink volume, angle, and distance, linearly map them to the interval [-1, 1], and eliminate the dimension difference:

[0038]

[0039]

[0040]

[0041] 2) For categorical parameters such as DPI, paper type, and illuminance, map them to a 128 - dimensional dense vector of = 128 through a learnable linear layer:

[0042]

[0043]

[0044]

[0045] 3) Concatenate all processed parameter vectors to form the final conditional control vector c:

[0046]

[0047] Step S3: Based on the dataset and the conditional control vector, train the graph-to-graph diffusion model; in this embodiment, the core of the graph-to-graph diffusion model is a Unet network structure. To achieve conditional control, in each layer of the Unet network, the conditional control vector c is explicitly injected. Specifically, as shown in Appendix Figure 3 As shown, let F be the input feature map of a certain network layer. The conditional control vector c is first projected through an upsampling operation to generate a projection vector with the same spatial size as F, that is, . Subsequently, the projection vector is fused with the input feature map F through a channel-level addition operation to obtain the output feature map :

[0048]

[0049] In the model training stage, the dataset is divided into a training set (1600 pairs), a validation set (200 pairs), and a test set (200 pairs) in the ratio of 8:1:1. The training adopts a phased strategy and uses the Adam optimizer (beta1 = 0.9, beta2 = 0.999). The loss function L is a mixed loss function, and its composition is: . Among them, is the mean square error loss between the generated image and the real degraded image, is the perceptual loss extracted based on the VGG16 network, is the adversarial loss, which is used to improve the fidelity of the generated image.

[0050] Step S4: Use a two-stream noise layer containing the trained model and traditional digital simulation to generate the final degraded image; after the model training is completed, in this embodiment, a two-stream noise layer is constructed to generate the final degraded image, and its overall structure and probability selection mechanism are as shown in Figure 4 As shown. This layer consists of two parallel branches: the first branch is the traditional digital simulation branch. This branch sequentially performs JPEG compression (quality Q is between 70 and 100), perspective transformation, random kernel (Gaussian or linear) blur, brightness and contrast adjustment, and saturation adjustment on the input image to simulate a classic, physics-based degradation process, and its detailed process is as shown in Figure 5As shown. The second branch is a data-driven diffusion model branch. This branch utilizes the graph-to-graph diffusion model trained in step S2, inputs the original image and the specified conditional control vector, and generates a highly realistic and detailed degraded image. Finally, the output of one of the branches is selected as the final result through a preset probability. In this embodiment, the probability of selecting the diffusion model branch is set to 0.6, and the probability of selecting the traditional digital simulation branch is set to 0.4. This fusion strategy makes the generated degraded image have both the interpretability of physical laws and the complexity of data-driven.

[0051] Correspondingly, the present invention also provides a print-shoot process image degradation simulation device for implementing the above method. Referring to Figure 6 the device may specifically include the following modules:

[0052] A dataset construction module for performing step S1 to construct a print-shoot dataset covering multiple scenarios; a conditional control vector construction module for performing step S2 to construct a conditional control vector based on printing and shooting parameters; a model training module for performing step S3 to train a graph-to-graph diffusion model based on the dataset and the conditional control vector; and an image generation module for performing step S4 to generate the final degraded image using a two-stream noise layer. These modules work together to implement the complete technical process of the present invention from data preparation to model training and then to the final image simulation generation.

[0053] To verify the effectiveness of the degraded images generated by this method, the proposed two-stream noise layer is applied to the training data augmentation of deep watermarking and image recognition tasks. The experimental results show that after training with the degraded data generated by this method, the accuracy of deep watermark extraction is improved by 12.5%, and the Top-1 accuracy of image recognition is improved by 8.3%, proving that this method can significantly improve the robustness and accuracy of downstream tasks in complex print-shoot scenarios.

[0054] In summary, the present invention constructs a controlled print-shoot dataset, trains a conditional graph-to-graph diffusion model, and innovatively proposes a two-stream noise layer combining traditional simulation and data-driven, thereby realizing high-fidelity and controllable image degradation simulation, which has important value for improving the performance of related image processing technologies in practical applications.

[0055] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model, characterized in that, The described image degradation simulation method includes the following steps: Step S1: Construct a printing - shooting dataset covering multiple scenarios, where the dataset includes original images and corresponding degraded images, and each group of degraded images is associated with the printing parameters and shooting parameters that cause its degradation; Step S2: Construct a conditional control vector based on the printing parameters and shooting parameters to parametrically control the degradation process; Step S3: Train a graph - to - graph diffusion model based on the dataset. The training process includes injecting the conditional control vector into the Unet network module in the model, so that the model learns the mapping relationship from the original image to the degraded image specified by the conditional control vector; Step S4: Generate the final degraded image using a two - stream noise layer that includes a trained graph - to - graph diffusion model and a traditional digital simulation; The two - stream noise layer includes: a first branch that processes the input image using traditional digital simulation methods to generate a first degraded image; a second branch that processes the input image using the trained graph - to - graph diffusion model to generate a second degraded image; and selects one of the first degraded image and the second degraded image as the final degraded image according to a preset probability.

2. The method for simulating the degradation of printing - shooting process images based on a graph - to - graph diffusion model according to claim 1, wherein, In the method for constructing the dataset in step S1, the printing parameters include DPI, ink amount, and paper type, and the shooting parameters include illuminance, angle, and distance.

3. A method for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model according to claim 1, characterized in that, The process of constructing the conditional control vector in step S2 includes: 1) performing normalization on continuous physical parameters; 2) mapping categorical physical parameters to dense vectors through a learnable linear layer; 3) concatenating the processed parameter vectors to form the final conditional control vector.

4. A method for simulating image degradation in the printing - shooting process based on a graph - to - graph diffusion model according to claim 1, characterized in that, The process of injecting the conditional control vector into the Unet network module in the diffusion model is specifically: in each layer structure of the Unet network module, the conditional control vector is upsampled to the same spatial size as the input feature map and fused into the input feature through channel - level addition.

5. A method for simulating the degradation of printing - shooting process images based on a graph - to - graph diffusion model according to claim 1, characterized in that, The structure of the two - stream noise layer includes: the first branch uses traditional digital simulation methods to sequentially perform image compression, geometric transformation, blur operation, and color adjustment; the second branch uses a noise distortion network simulated by the trained graph - to - graph diffusion model.

6. An image degradation simulation device for the printing - shooting process based on a graph - to - graph diffusion model, characterized in that, Includes: A dataset construction module for constructing a printing - shooting dataset covering multiple scenarios, where the dataset includes original images and corresponding degraded images, and each group of degraded images is associated with the printing parameters and shooting parameters that cause its degradation; A conditional control vector construction module for constructing a conditional control vector based on the printing parameters and shooting parameters to parametrically control the degradation process; A model training module for training a graph - to - graph diffusion model based on the dataset. The training process includes injecting the conditional control vector into the Unet network module in the model, so that the model learns the mapping relationship from the original image to the degraded image specified by the conditional control vector; An image generation module for generating a final degraded image by using a dual-stream noise layer that includes a trained graph-to-graph diffusion model and a traditional digital simulation. The dual-stream noise layer includes: a first branch that processes an input image by using a traditional digital simulation method to generate a first degraded image; a second branch that processes the input image by using the trained graph-to-graph diffusion model to generate a second degraded image; and selecting one of the first degraded image and the second degraded image as the final degraded image according to a preset probability.

7. An image degradation simulation device for the printing-shooting process based on a graph-to-graph diffusion model, characterized in that, Comprising: A memory for storing a computer program; A processor for executing the computer program to implement the print-photograph process image degradation simulation method based on a graph-to-graph diffusion model according to any one of claims 1 to 5.

Images