Image degradation simulation method, device and equipment for printing-shooting process based on image-to-image diffusion model

Abstract

The present invention discloses a method, device and equipment for simulating image degradation in a printing-shooting process based on an image-to-image diffusion model, aiming to solve the problem that existing degradation simulation methods differ greatly from real physical processes and cannot be precisely controlled. The method first constructs a printing-shooting dataset covering a variety of printing parameters and shooting parameters; then encodes these physical parameters into conditional control vectors and injects them into a Unet network of an image-to-image diffusion model for training; when generating images, the present invention adopts an innovative dual-stream noise layer structure, which combines a first branch using a traditional digital simulation method and a second branch using the pre-trained diffusion model in parallel, and selects one of them for output according to a preset probability. The present invention can generate highly realistic degraded images, thereby significantly improving the robustness and accuracy of downstream tasks (such as deep watermarking and image recognition) in real printing-shooting scenarios.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, device and equipment for simulating image degradation in a printing-shooting process based on an image-to-image diffusion model. Background Art

[0002] The print-to-shoot process is a common step in image applications, encompassing scenarios such as document digitization, image dissemination, and reuse. During the printing phase, images can suffer from loss of detail, color deviation, or blurring due to factors such as printer nozzle characteristics, ink or toner diffusion, and paper texture. During the shooting phase, image degradation is further exacerbated by camera optical aberrations, lens distortion, ambient lighting variations, and sensor noise. These degraded images often significantly degrade the quality of subsequent processing, such as image recognition or watermark extraction.

[0003] Taking deep watermarking technology as an example, watermarks embedded in images are often destroyed during image degradation, causing watermark extraction failure and severely impacting the robustness and security of deep watermarking algorithms. Currently, methods for simulating image degradation during the printing-shooting process are mostly based on simple mathematical simulations and empirical formulas. These methods struggle to comprehensively and accurately characterize the multi-source and variable degradation mechanisms in real-world scenarios. The simulated degraded images differ significantly from their real-world counterparts, making them incapable of providing high-quality, realistic sample data for training related image processing techniques such as deep watermarking or digital recognition. This, in turn, limits the performance of these technologies in practical applications.

[0004] In recent years, diffusion models, as a probabilistic generation-based deep learning model, have been able to model complex data distributions by gradually adding noise to the data space and learning a denoising process. The image-to-image diffusion model is an extension of this approach to image conversion, aiming to achieve content and style mapping and migration between different image feature spaces. Essentially, the image degradation caused by the print-to-photograph process can also be viewed as a complex image feature conversion process. Therefore, the image-to-image diffusion model, with its ability to model complex data and transform image features, offers an ideal technical approach for simulating print-to-photograph image degradation. Summary of the Invention

[0005] In view of this, the object of the present invention is to provide a method, device and equipment for simulating image degradation in the printing-shooting process based on an image-to-image diffusion model, which can simulate the complex image degradation introduced by the printing-shooting process in a high-fidelity and controllable manner.

[0006] In a first aspect, the present application provides a method for simulating image degradation during a printing-shooting process based on an image-to-image diffusion model, comprising:

[0007] Constructing a printing-shooting dataset covering multiple scenes, the dataset including original images and degraded images corresponding to the original images, wherein each set of degraded images is associated with printing parameters and shooting parameters that cause the degradation of the images;

[0008] constructing a conditional control vector based on the printing parameters and the shooting parameters to perform parameterized control on the degradation process;

[0009] Training an image-to-image diffusion model based on the dataset, wherein the training process includes injecting the conditional control vector into a Unet network module within the model so that the model learns a mapping relationship from the original image to a degraded image specified by the conditional control vector;

[0010] A final degraded image is generated from an input image using a dual-stream noise layer. The dual-stream noise layer includes: a first branch that processes the input image using a traditional digital simulation method to generate a first degraded image; a second branch that processes the input image using the trained image-to-image diffusion model to generate a second degraded image; and selecting one of the first degraded image and the second degraded image as the final degraded image based on a preset probability.

[0011] Optionally, the printing parameters include DPI, ink volume and paper type; the shooting parameters include illumination, shooting angle and shooting distance.

[0012] Optionally, the step of constructing the conditional control vector includes: performing normalization processing on continuous physical parameters; mapping categorical physical parameters into dense vectors through a learnable linear layer; and splicing all processed parameter vectors to form a final conditional control vector.

[0013] Optionally, the step of injecting the conditional control vector into the Unet network module is specifically: in each network layer of the Unet network module, upsampling the conditional control vector to the same spatial size as the input feature map of the network layer; and fusing the upsampled conditional control vector into the input feature map through channel-level addition.

[0014] Optionally, the traditional digital simulation method in the first branch includes performing image compression, geometric transformation, blur operation and color adjustment on the image in sequence.

[0015] Optionally, the step of training the image-to-image diffusion model further includes: optimizing the model using a hybrid loss function that combines mean square error loss, perceptual loss based on a VGG network, and adversarial loss.

[0016] In a second aspect, the present application provides a device for simulating image degradation during a printing-shooting process based on an image-to-image diffusion model, comprising:

[0017] A dataset construction module is used to construct a print-shoot dataset covering multiple scenes. The dataset contains original images and their corresponding degraded images, where each set of degraded images is associated with the specific printing and shooting parameters that cause its degradation;

[0018] A conditional control vector construction module, which constructs a conditional control vector for parameterized control based on the printing parameters and the shooting parameters;

[0019] a model training module, configured to train an image-to-image diffusion model based on the dataset, by injecting the conditional control vector into a Unet network module within the model, so that the model learns a mapping relationship from an original image to a degraded image specified by the conditional control vector;

[0020] An image generation module is configured to generate a final degraded image using a dual-stream noise layer comprising a trained image-to-image diffusion model and traditional digital simulation, wherein the dual-stream noise layer comprises: a first branch that processes an input image using a traditional digital simulation method to generate a first degraded image; a second branch that processes the input image using the trained image-to-image diffusion model to generate a second degraded image; and select one of the first degraded image and the second degraded image as the final degraded image based on a preset probability.

[0021] In a third aspect, the present application provides a device for simulating image degradation during the printing-shooting process based on an image-to-image diffusion model, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the aforementioned method for simulating image degradation during the printing-shooting process based on the image-to-image diffusion model.

[0022] As can be seen from the above, this application has the following beneficial effects:

[0023] Controllable degradation simulation is achieved: by encoding the physical parameters of printing and shooting into conditional control vectors and injecting them into the diffusion model, this method can dynamically and accurately generate corresponding degradation effects based on the specified parameter combination, and has the ability to generalize across devices and scenarios.

[0024] Generates high-fidelity degraded images: Leveraging the powerful distributed learning capabilities of the image-to-image diffusion model, it is able to capture complex noise and degradation patterns in the real world. The generated images are highly realistic in details and global structures, providing high-quality training data for downstream tasks (such as deep watermarking and image recognition).

[0025] Improved robustness of downstream tasks: The innovative design of the dual-stream noise layer combines the physical interpretability of traditional digital simulations with the data-driven complexity of diffusion models. The generated degraded samples are both realistic and diverse, significantly enhancing the robustness and accuracy of related algorithms in practical application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of the printing-shooting degradation simulation method disclosed in this application.

[0027] Figure 2 This is the overall framework diagram of the printing-shooting degradation simulation method disclosed in this application.

[0028] Figure 3 Schematic diagram of the injection of the conditional control vector disclosed in this application into the Unet network.

[0029] Figure 4 This is a schematic diagram of the dual-stream noise layer structure disclosed in this application.

[0030] Figure 5 This is a schematic diagram of the traditional digital analog layer structure disclosed in this application.

[0031] Figure 6 This is a module diagram of the image degradation simulation device for the printing-shooting process disclosed in this application. DETAILED DESCRIPTION

[0032] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0033] Reference Figure 1 The embodiment of the present invention discloses a method for simulating image degradation in the printing-shooting process based on an image-to-image diffusion model. In this example, the method is implemented in Python and the model is built on the Pytorch deep learning framework. The overall conceptual framework of the method is as follows: Figure 2 As shown in Figure 2, it describes the complete process from real physical process to dataset construction and then to model simulation output.

[0034] Step S1: Construct a print-shooting dataset; in this embodiment, a print-shooting dataset covering multiple scenes is first constructed. The dataset contains 2000 pairs of original-degraded image pairs, comprehensively covering key degradation factors such as lighting, shooting distance, angle, and different printing device parameters. Specifically, the printing parameters include: DPI, with optional values ​​of {300, 600, 1200}; ink volume, ranging from [70%, 130%]; paper type, with optional values ​​of {glossy, matte, newsprint}. The shooting parameters include: illumination, with optional values ​​of {200, 500, 1000} lux; shooting angle, ranging from [60°, 120°]; shooting distance, ranging from [5cm, 30cm], and the formula is:

[0035]

[0036] Step S2: Construct a conditional control vector based on the printing and shooting parameters; construct a conditional control vector c based on the physical parameters collected in step S1. This process includes three steps: standardization, embedded coding, and splicing fusion:

[0037] 1) Perform normalization on continuous parameters such as ink volume, angle, and distance, and linearly map them to the [-1, 1] interval to eliminate dimensional differences:

[0038]

[0039]

[0040]

[0041] 2) For categorical parameters such as DPI, paper type, and illumination, a learnable linear layer is used to map them to d e =128 128-dimensional dense vector:

[0042]

[0043]

[0044]

[0045] 3) All processed parameter vectors are concatenated to form the final conditional control vector c:

[0046]

[0047] Step S3: Based on the data set and the conditional control vector, train the graph-to-graph diffusion model. In this embodiment, the core of the graph-to-graph diffusion model is a Unet network structure. In order to achieve conditional control, the conditional control vector c is explicitly injected into each layer of the Unet network. Specifically, as shown in the attached figure, Figure 3 As shown, let F be the input feature map of a certain network layer, and the conditional control vector c is first projected by upsampling operation to generate a projection vector c with the same spatial size as F. proj , that is, c proj =UP_Sampling(c). Then, the projection vector c is added by channel-level addition operation. proj Fuse with the input feature map F to obtain the output feature map F out :

[0048] F out =F+c proj

[0049] During the model training phase, the dataset was split into a training set (1600 pairs), a validation set (200 pairs), and a test set (200 pairs) at an 8:1:1 ratio. Training was performed in stages using the Adam optimizer (beta1 = 0.9, beta2 = 0.999). The loss function L was a hybrid loss function consisting of: L = 0.8*L mse +0.8*L vgg +0.2*l gan Among them, L mse is the mean square error loss between the generated image and the real degraded image, L vgg is the perceptual loss extracted based on the VGG16 network, L gan It is an adversarial loss used to improve the realism of generated images.

[0050] Step S4: Generate the final degraded image using a dual-stream noise layer containing the trained model and traditional digital simulation; After the model training is completed, this embodiment constructs a dual-stream noise layer for generating the final degraded image, and its overall structure and probability selection mechanism are as follows Figure 4 As shown in Figure 1. This layer consists of two parallel branches: the first branch is the traditional digital simulation branch. This branch sequentially performs JPEG compression (quality Q between 70-100), perspective transformation, random kernel (Gaussian or linear) blur, brightness and contrast adjustment, and saturation adjustment on the input image to simulate the classic, physical model-based degradation process. The detailed process is shown in Figure 1. Figure 5As shown. The second branch is a diffusion model branch based on data-driven. This branch uses the image-to-image diffusion model trained in step S2, inputs the original image and the specified conditional control vector, and generates a highly realistic and detailed degraded image. Finally, a preset probability is used to select the output of one of the branches as the final result. In this embodiment, the probability of selecting the diffusion model branch is set to 0.6, and the probability of selecting the traditional digital simulation branch is set to 0.4. This fusion strategy makes the generated degraded image both interpretable by physical laws and with data-driven complexity.

[0051] Accordingly, the present invention also provides a device for simulating image degradation during printing and shooting to implement the above method. Figure 6 , the device may specifically include the following modules:

[0052] A dataset construction module, configured to execute step S1 to construct a printing-shooting dataset covering multiple scenarios; a conditional control vector construction module, configured to execute step S2 to construct a conditional control vector based on printing and shooting parameters; a model training module, configured to execute step S3 to train an image-to-image diffusion model based on the dataset and the conditional control vector; and an image generation module, configured to execute step S4 to generate the final degraded image using a dual-stream noise layer. These modules work together to implement the complete technical process described in this invention, from data preparation to model training to final image simulation generation.

[0053] To validate the effectiveness of the degraded images generated by this method, the proposed two-stream noise layer was applied to training data augmentation for deep watermarking and image recognition tasks. Experimental results show that training with the degraded data generated by this method improves the accuracy of deep watermark extraction by 12.5% ​​and the top-1 accuracy of image recognition by 8.3%, demonstrating that this method can significantly improve the robustness and accuracy of downstream tasks in complex printing and shooting scenarios.

[0054] In summary, the present invention constructs a controlled print-shoot dataset, trains a conditional image-to-image diffusion model, and innovatively proposes a dual-stream noise layer that combines traditional simulation and data-driven methods, thereby achieving high-fidelity and controllable image degradation simulation, which is of great value for improving the performance of related image processing technologies in practical applications.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for simulating image degradation during printing and shooting based on an image-to-image diffusion model, characterized in that: The image degradation simulation method comprises the following steps: Step S1: constructing a printing-shooting dataset covering multiple scenes, wherein the dataset includes original images and degraded images corresponding to the original images, wherein each set of degraded images is associated with printing parameters and shooting parameters that cause the degradation of the images; Step S2: constructing a condition control vector based on the printing parameters and the shooting parameters to perform parameterized control on the degradation process; Step S3: training an image-to-image diffusion model based on the dataset, wherein the training process includes injecting the conditional control vector into a Unet network module within the model so that the model learns a mapping relationship from the original image to the degraded image specified by the conditional control vector; Step S4: Generate the final degraded image using a two-stream noise layer consisting of the trained image-to-image diffusion model and traditional digital simulation; The dual-stream noise layer includes: a first branch, which uses a traditional digital simulation method to process the input image to generate a first degraded image; a second branch, which uses the trained image-to-image diffusion model to process the input image to generate a second degraded image; and selects one of the first degraded image and the second degraded image as the final degraded image according to a preset probability.

2. The method for simulating image degradation during printing and shooting based on an image-to-image diffusion model according to claim 1, characterized in that: In the method for constructing the data set in step S1, the printing parameters include DPI, ink volume, and paper type, and the shooting parameters include illumination, angle, and distance.

3. The method for simulating image degradation during printing and shooting based on an image-to-image diffusion model according to claim 1, characterized in that: The process of constructing the conditional control vector in step S2 includes: 1) performing normalization processing on continuous physical parameters; 2) mapping categorical physical parameters into dense vectors through a learnable linear layer; 3) concatenating the parameter vectors after splicing to form the final conditional control vector.

4. The method for simulating image degradation during printing and shooting based on an image-to-image diffusion model according to claim 1, characterized in that: The conditional control vector injection process of the Unet network module in the diffusion model is specifically as follows: in each layer structure of the Unet network module, the conditional control vector is upsampled to the same spatial size as the input feature map and fused to the input feature through channel-level addition.

5. The method for simulating image degradation during printing and shooting based on an image-to-image diffusion model according to claim 1, characterized in that: The dual-stream noise layer structure includes: the first branch uses traditional digital simulation methods to perform image compression, geometric transformation, blur operation and color adjustment in sequence; the second branch uses a noise distortion network simulated by a trained image-to-image diffusion model.

6. A device for simulating image degradation during printing and shooting based on an image-to-image diffusion model, characterized in that: include: a dataset construction module, configured to construct a printing-shooting dataset covering multiple scenes, the dataset comprising original images and degraded images corresponding to the original images, wherein each set of degraded images is associated with printing parameters and shooting parameters that cause the degradation; a conditional control vector construction module, configured to construct a conditional control vector based on the printing parameters and the shooting parameters, so as to perform parameterized control on the degradation process; a model training module, configured to train an image-to-image diffusion model based on the dataset, wherein the training process includes injecting the conditional control vector into a Unet network module within the model so that the model learns a mapping relationship from an original image to a degraded image specified by the conditional control vector; An image generation module is configured to generate a final degraded image using a dual-stream noise layer comprising a trained image-to-image diffusion model and traditional digital simulation, wherein the dual-stream noise layer comprises: a first branch that processes an input image using a traditional digital simulation method to generate a first degraded image; a second branch that processes the input image using the trained image-to-image diffusion model to generate a second degraded image; and select one of the first degraded image and the second degraded image as the final degraded image based on a preset probability.

7. A device for simulating image degradation during printing and shooting based on an image-to-image diffusion model, characterized in that: include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the method for simulating image degradation in a printing-shooting process based on an image-to-image diffusion model according to any one of claims 1 to 5.

Images