Visual center removal diffusion model endogenous watermark method, electronic device and medium

Abstract

The invention relates to a diffusion model endogenous watermark method for removing a visual center, an electronic device and a medium. The method comprises the following steps: determining a geometric center of a to-be-generated target image; determining a visual center of the target image, and determining a watermark embedding area outside the visual center according to a target distance from the visual center to the geometric center; according to the watermark embedding area, embedding the watermark into Fourier transform of the first noise vector to obtain a second noise vector; inputting the original image into a diffusion model, and performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark; and the concealment of the watermark is enhanced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a diffusion model endogenous watermarking method, an electronic device and a medium that removes the visual center. Background Art

[0002] In AI-generated synthesis algorithms, diffusion models are widely used in AI-generated synthetic content services due to their high-quality generation performance. With the popularity of generated content, the responsibility for synthetic content has become an urgent issue to be resolved. Currently, the identification and traceability watermarking technology for content generated by diffusion models is not concealed enough, making the watermark easy to detect or even removed.

[0003] At present, no effective solution has been proposed for the problem that the endogenous watermark of the diffusion model in related technologies lacks concealment. Summary of the invention

[0004] Based on this, it is necessary to provide a de-visualized diffusion model endogenous watermarking method, electronic device and medium to improve the concealment of watermarks in order to solve the above technical problems.

[0005] In a first aspect, the present application provides a diffusion model endogenous watermarking method for removing the visual center. The method comprises:

[0006] Determine the geometric center of the target image to be generated;

[0007] Determining the visual center of the target image, and determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0008] According to the watermark embedding area, embedding the watermark into the Fourier transform of the first noise vector to obtain a second noise vector;

[0009] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark.

[0010] In one embodiment, determining the visual center of the target image includes:

[0011] The visual center of the target image is predicted according to the forward denoising diffusion training process of the diffusion model.

[0012] In one of the embodiments, determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center includes:

[0013] Determine a first range according to the size of the original image, and determine a second range according to the target distance;

[0014] The watermark embedding area is obtained by deducting the second range from the first range.

[0015] In one embodiment, the original image is input into a diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0016] Based on the total diffusion stage of the diffusion model and preset hyperparameters, the reverse denoising diffusion is divided into a first reverse stage and a second reverse stage;

[0017] In the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed;

[0018] In the second reverse stage, reverse denoising diffusion is performed after unlocking the watermark embedded area in the original image.

[0019] In one embodiment, before performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark, the method further includes:

[0020] Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image;

[0021] Based on the target constraint, the diffusion model is trained for reverse denoising diffusion.

[0022] In one embodiment, the original image is input into a diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0023] After the original image undergoes reverse denoising diffusion in the first reverse stage and the second reverse stage, an intermediate image is obtained;

[0024] In the intermediate image, the watermark embedding area is compensated for the reverse denoising diffusion process of the first stage to obtain the final target image.

[0025] In the second aspect, the present application provides a diffusion model endogenous watermarking method for removing the visual center, including:

[0026] Determine the geometric center and the visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0027] The target image is subjected to forward noise diffusion processing to extract the watermark in the watermark embedding area.

[0028] In one embodiment, performing forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area includes:

[0029] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the forward noise diffusion process is divided into a first forward stage and a second forward stage;

[0030] In the first forward stage, a forward noise diffusion process is performed after locking the complement of the watermark embedding area in the target image;

[0031] In the second forward stage, forward denoising and diffusion processing is performed after unlocking the complement of the watermark embedding area in the target image.

[0032] In one embodiment, the method further comprises:

[0033] Compare the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold;

[0034] When the similarity is not less than the threshold, the forward noise diffusion process is stopped;

[0035] When the similarities are lower than the threshold, the forward noise diffusion process is stopped, and it is determined that the target image passes the watermark verification.

[0036] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the first aspect or the second aspect when executing the computer program.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method of the first aspect or the second aspect are implemented.

[0038] The above-mentioned diffusion model endogenous watermarking method, electronic device and medium without visual center can make the generated watermark merge into the position outside the main body by dividing a specific range outside the visual center of the target image as the watermark embedding area, and embedding the watermark information into the Fourier transform of the initial noise vector corresponding to the watermark embedding area, which can further weaken the significance of the watermark and ultimately enhance the concealment of the watermark, ensuring that it will not interfere with the core content of the target image, and can retain the effectiveness of the diffusion model to the greatest extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A hardware structure block diagram of a terminal of a diffusion model endogenous watermarking method for removing visual center in an embodiment;

[0040] Figure 2 A flowchart of a diffusion model endogenous watermarking method for removing visual center in one embodiment;

[0041] Figure 3 is a flow chart of a reverse denoising diffusion process in one embodiment;

[0042] Figure 4 It is a flow chart of a diffusion model endogenous watermarking method for removing visual center in another embodiment;

[0043] Figure 5 is a flow chart of a watermark extraction method in one embodiment;

[0044] Figure 6 FIG. 4 is a flow chart of a watermark extraction method in another embodiment. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0047] The method embodiments provided in this application can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure block diagram of a terminal of a diffusion model endogenous watermarking method for removing visual center according to an embodiment of the present application. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) processor 101 and memory 102 for storing data, wherein processor 101 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 103 and an input and output device 104 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0048] The memory 102 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the diffusion model endogenous watermarking method for removing the visual center in this embodiment. The processor 101 executes various functional applications and data processing by running the computer program stored in the memory 102, that is, to implement the above method. The memory 102 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 102 may further include a memory remotely arranged relative to the processor 101, and these remote memories may be connected to the terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0049] The transmission device 103 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 103 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 103 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0050] The generation process of the diffusion model involves step-by-step denoising and latent variable distribution modeling, and its watermark embedding technology is somewhat unique. The direct watermark embedding method in the related art embeds the watermark into the output generated by the diffusion model without modifying the model itself, and the watermark embedding and extraction are relatively direct. However, the method of directly embedding the watermark in the output content is not covert enough, which may cause the watermark to be easily detected or even removed, and is not concealed.

[0051] Regarding the problem of how to improve the concealment of watermark, in one embodiment, Figure 2 This paper provides a diffusion model endogenous watermarking method to remove the visual center, which belongs to the watermark embedding process. Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0052] Step S101, determining the geometric center of the target image to be generated.

[0053] The target image is a synthetic image generated by computer graphics or machine learning methods, which can be generated by processing the original image by the diffusion model. The diffusion model is a generative model that is trained by forward denoising diffusion. The training process is the process of gradually adding noise to the original image. In each step, the diffusion model learns the noise distribution under the current image state and applies it to increase the randomness of the image. Through forward denoising diffusion, the diffusion model learns the mapping process from the original image to pure noise. Then the process is reversed. When generating a new image, the diffusion model starts from pure noise and gradually removes noise to restore a reasonable image. This process is defined as the reverse denoising diffusion process.

[0054] Before generating a target image based on the original image, the geometric center of the image is determined according to the desired target image size, and the geometric center is used as the origin. The geometric center refers to the center point of the image size, which can be calculated by the physical size of the image. For example, if the target image is a rectangle with a width of W and a height of H, its geometric center is located at (W / 2, H / 2). If the target image is a circle, its geometric center is located at the center of the circle.

[0055] Step S102, determining the visual center of the target image, and determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center.

[0056] The watermark in this embodiment is a kind of identification information that can be embedded in an image to protect the copyright of the generated content. The visual center refers to the most eye-catching or prominent area in the image, which can be the brightest position in the image, the position with the strongest color contrast, or the position containing important objects.

[0057] When determining the visual center of the target image, the visual center of the target image can be predicted based on the forward noise diffusion training process of the diffusion model. Specifically, collect training samples with clear visual centers and mark the locations of these visual centers; build a diffusion model architecture, incorporate conditional information, saliency constraints, feature matching and other mechanisms; train the diffusion model to maintain and strengthen the characteristics of the visual center during training; after generating the target image, evaluate whether its visual center reaches the predetermined goal, and adjust the training strategy or model parameters based on the results; and continuously optimize the model based on the feedback loop. The following provides several specific methods for determining the visual center of the target image during diffusion model training.

[0058] (1) When the generation task provides additional conditional information (such as category labels, text descriptions, key point locations, etc.), this information can be used to guide the selection of the visual center during training. For example, in the face generation task, feature points such as eyes and nose can be used as conditional input to ensure that these areas become the visual center.

[0059] (2) Use saliency detection algorithms (such as Saliency Map) to automatically identify the most eye-catching areas in the original image and give them more attention during training. This method can be achieved by adjusting the loss function, such as giving higher weights to pixels close to the expected visual center.

[0060] (3) Use pre-trained deep learning models (such as VGG, ResNet) to extract high-level feature maps of the original image and try to match the features of the generated image with them during training. This method can help capture important visual elements and better define the visual center.

[0061] (4) During training, some small windows can be selected for local enhancement training so that the diffusion model can learn to better capture details to facilitate the definition of the visual center. For example, stronger learning pressure can be applied to key areas of the image at specific time nodes or training steps.

[0062] Through the above method, the visual center of the generated target image can be effectively determined in the forward denoising diffusion training process based on a given generation task.

[0063] After determining the visual center of the target image, the watermark embedding area outside the visual center is determined according to the target distance from the visual center to the geometric center. Specifically, the first range D(R0) is determined according to the size R0 of the original image, and the second range D is determined according to the target distance r0. o (r0); remove the second range from the first range to obtain the watermark embedding area D (R 0 )\D ° (r 0 ). The first range D(R0) is a large circle with the geometric center as the center and R0 as the radius, D ° (r 0 ) is a small circle with the visual center as the center and r0 as the radius (excluding the border), and \ represents the deletion operation. Therefore, the watermark embedding area D(R 0 )\D ° (r 0 ) is equivalent to deleting a small circle in a large circle. In this way, the range of watermark embedding away from the visual center can be personalized according to the characteristics of different generation and synthesis tasks, so as to select the least noticeable area for watermark embedding, improve the concealment of watermark, and prevent it from interfering with the main content of the image.

[0064] Optionally, the watermark embedding area has symmetry, which is symmetrical about the straight line where the visual center and the geometric center are located. In this way, it can resist geometric transformations such as rotation, scaling and translation of the image to a certain extent. Even if one part of the image is damaged or modified, the other part may still retain the complete watermark information, thereby improving the robustness of the watermark. Moreover, symmetry can be used as an additional feature to verify the authenticity of the watermark, helping to distinguish the real watermark from random noise or other interference signals, thereby reducing the possibility of false positives.

[0065] Step S103: embed the watermark into the Fourier transform of the first noise vector according to the watermark embedding area to obtain a second noise vector.

[0066] The first noise vector is an initial noise vector, which can be randomly extracted from a Gaussian distribution or other appropriate probability distribution, so that the random noise does not have any specific pattern in both the spatial domain and the frequency domain. Optionally, the watermark can be directly embedded in the first noise vector. Optionally, the watermark can be embedded in the Fourier transform of the initial noise vector. Specifically, the first noise vector is Fourier transformed to obtain the frequency domain information of the first noise vector, and the spectral component of the watermark is added or multiplied at the frequency domain position selected by the first noise vector to embed the watermark information into its spectrum. After the watermark embedding in the frequency domain is completed, the first noise vector is inversely Fourier transformed and converted back to the spatial domain to obtain the second noise vector. Embedding the watermark information in the Fourier transform of the initial noise vector corresponding to the watermark embedding area, rather than directly adding it to the initial Gaussian noise, helps prevent obvious watermark patterns from appearing in the final target image.

[0067] Step S104: input the original image into the diffusion model, perform reverse denoising and diffusion on the original image based on the second noise vector, and generate a target image embedded with a watermark.

[0068] The diffusion model reverses the learned forward denoising diffusion process, that is, performs a reverse denoising diffusion process. Specifically, starting from the second noise vector in the original image, the noise is gradually removed to restore another image embedded with the watermark, that is, the target image.

[0069] In the above steps S101 to S104, by dividing a specific range outside the visual center of the target image as the watermark embedding area, the generated watermark can be integrated into a position outside the main body, and the watermark information is embedded in the Fourier transform of the initial noise vector corresponding to the watermark embedding area, which can further weaken the significance of the watermark and ultimately enhance the concealment of the watermark, ensuring that it will not interfere with the core content of the target image, and can maximize the retention of the effectiveness of the diffusion model.

[0070] The above-mentioned embodiment embeds watermark information in the denoising process of the diffusion model. However, the watermark extraction process is relatively complicated. Because the current artificial intelligence-generated synthetic content service user groups are large, and the number of artificial intelligence-generated synthetic images that need to be identified and traced is huge, the watermark extraction process based on the current diffusion adjustment is slow and inefficient, and it takes extra time and computing power, resulting in a waste of resources.

[0071] To address this problem, in one embodiment, Figure 3 A flowchart of another reverse denoising diffusion process method is provided, such as Figure 3 As shown, the process includes the following steps:

[0072] Step S201 : Based on the total diffusion stage of the diffusion model and preset hyper parameters, the reverse denoising diffusion is divided into a first reverse stage and a second reverse stage.

[0073] Based on the total diffusion stage T, the first T / k stages are divided into the first reverse stage, and the stage from T to T / k is divided into the second reverse stage. Among them, T / k is a coefficient, k represents a hyperparameter that measures the watermark extraction efficiency and image target quality, and / represents division.

[0074] Step S202, in the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed.

[0075] In the first reverse stage, the watermark embedded area in the original image is locked and does not participate in the reverse denoising diffusion process. Only the preceding area of ​​the original image (the complement of the watermark embedded area) is subjected to reverse denoising diffusion.

[0076] Step S203, in the second reverse phase, the watermark embedding area is unlocked in the original image and then reverse denoising diffusion is performed.

[0077] In the second reverse stage, the watermark embedded area in the original image is unlocked, so that the preceding area and the watermark embedded area are subjected to reverse denoising diffusion together.

[0078] In the above steps S201 to S203, by setting the first reverse stage and the second reverse stage, the watermark and the image are reversely denoised and diffused in a layered mode, which is convenient for accelerating the watermark extraction process when extracting the watermark later.

[0079] The reverse diffusion of the layered watermark image may cause the noise to be repeatedly amplified or weakened between different layers, which ultimately affects the quality of the generated target image. To address this problem, in one embodiment, the original image is input into the diffusion model, and the original image is reversely denoised and diffused based on the second noise vector to generate a target image embedded with the watermark. The method also includes:

[0080] Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image; based on the target constraint, performing reverse denoising diffusion training on the diffusion model.

[0081] This embodiment may use one or more consistency constraints as target constraints to guide the reverse denoising diffusion training of the diffusion model.

[0082] Optionally, based on the consistent diffusion target of the watermark embedding area boundary and the image, a consistency constraint is established in the L2 space between the inner boundary of the watermark embedding area and the extension direction of the visual center. The L2 space refers to a space that uses the Euclidean distance (i.e., the square root of the square difference) as a metric, in which the distances between different points or features can be measured. Specifically, in order to prevent the break differences of the edges of different levels during the layered diffusion process, the target constraint is introduced so that when performing reverse denoising diffusion, the generated watermark embedding area boundary and the image in the extension direction of the visual center have global consistency, ensuring that the watermark will not interfere with the main visual information of the target image, but can be naturally integrated into the target image.

[0083] Optionally, based on the error fidelity objective of frequency loss, a spectral distribution consistency constraint between the target image and the original image is established. Specifically, in the reverse denoising diffusion process, the frequency domain separation constraint is used, such as comparing the spectral distribution of the target image and the original image through Fourier transform, so as to constrain the result of the generated target image in the frequency domain and reduce the utility loss of the diffusion model.

[0084] Optionally, based on the consistent diffusion target of the watermark embedding area boundary and the image and the error fidelity target of frequency loss, a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center is established in the L2 space, as well as a spectral distribution consistency constraint between the target image and the original image. The image generation quality is coordinated through multi-objective constraints to ensure the reliability of the back diffusion of the layered watermark image and reduce the utility impact of the watermark embedding process on the generated target image, ensuring that the noise is gradually removed rather than repeatedly amplified or weakened between different levels.

[0085] In order to further solve the problem that reverse diffusion of layered watermark images may cause loss of image quality, in one embodiment, when the original image is input into a diffusion model, reverse denoising diffusion is performed on the original image based on a second noise vector to generate a target image embedded with a watermark, an intermediate image can be obtained after the original image has undergone reverse denoising diffusion in a first reverse stage and a second reverse stage; in the intermediate image, the watermark embedded area is compensated for the reverse denoising diffusion process in the first reverse stage to obtain the final target image.

[0086] In this embodiment, considering the loss in the hierarchical diffusion stage of the watermark embedding area, it can be compensated on the basis of the multi-objective coordinated image generation quality constraint. Specifically, the watermark embedding area is compensated for the reverse denoising diffusion process for T / k stages, that is, after the reverse denoising diffusion in the second reverse stage, the watermark embedding area is allowed to continue the reverse denoising diffusion to obtain the final target image. In this way, the loss in the hierarchical diffusion stage of the watermark embedding area is reduced, and the quality of the target image is guaranteed.

[0087] In one embodiment, Figure 4 Another diffusion model endogenous watermarking method with visual center removal is provided, which belongs to the watermark extraction process. Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0088] Step S301, determining the geometric center and visual center of the target image, and determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center.

[0089] Step S302, performing forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area.

[0090] For the description of the target image, geometric center, visual center, target distance, and watermark embedding area, please refer to the description of the above embodiment, which will not be repeated here. In this embodiment, the noise vector can be gradually added to the target image to extract the watermark in accordance with the process opposite to the reverse denoising diffusion process of the above embodiment. In the above steps S301 to S302, by determining the geometric center and visual center of the target image, the watermark embedding area is determined based on the geometric center and visual center of the target image, and the target image is subjected to forward denoising diffusion processing. With such a setting, there is no need to access the weights of the diffusion model, and the owner of the diffusion model can directly use forward denoising diffusion to extract the watermark through the target image.

[0091] Furthermore, different watermark extraction methods can be adopted for target images from different sources.

[0092] In one embodiment, for the target image generated by the watermark embedding method of the above embodiment, Figure 5 A flow chart of a watermark extraction method is provided, the flow chart comprising the following steps:

[0093] Step S401, determining the geometric center and the visual center of the target image, and determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center.

[0094] Step S402: Based on the total diffusion stage of the diffusion model and preset hyper parameters, the forward noise diffusion process is divided into a first forward stage and a second forward stage.

[0095] Based on the total diffusion stage T, the first T / k stages are divided into the first forward stage, and the stages from T / k to T are divided into the second forward stage.

[0096] Step S403, in the first forward stage, after locking the complement of the watermark embedding area in the target image, forward noise diffusion processing is performed.

[0097] In the first forward stage, symmetrically with the watermark embedding process, the complement of the watermark embedding area is locked and masked, and the watermark embedding area is subjected to forward noise diffusion.

[0098] Step S404, in the second forward stage, after unlocking the complement of the watermark embedding area in the target image, forward denoising and diffusion processing is performed.

[0099] In the second forward stage, the complement of the watermark embedded area is unlocked, and the area outside the watermark embedded area is subjected to forward noise diffusion processing together with the watermark embedded area.

[0100] Step S405, comparing the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold. If the similarity is not less than the threshold, the forward noise diffusion process is stopped.

[0101] When the similarity between the noise vector of the watermark embedding area extracted at a certain moment and the initial embedded watermark in the L2 space reaches a certain threshold δ, that is, when the L2 distance L between the two is not less than δ, it can be considered that the watermark extraction and tracing goal has been achieved.

[0102] In the above steps S401 to S405, when extracting the watermark from the target image generated by the target diffusion model, the watermark embedding area is determined by the geometric center and the visual center of the target image, and the target image is forward-noised and diffused by the target diffusion model, and hierarchical verification is performed during the forward-noise diffusion process, that is, in the first forward stage of the division, only the watermark embedding area of ​​the image is forward-noised and diffused, and in the second forward stage, the extracted watermark embedding area is compared with the target watermark for similarity in the L2 space. When the similarity between the noise vector of the watermark embedding area extracted at a certain moment and the initial embedded watermark in the L2 space reaches a certain threshold, the target image can be traced. In this way, the decoupling of the watermark extraction process and the forward-noise diffusion process is achieved, which can reduce the running time of the forward-noise diffusion stage, speed up the speed of watermark extraction and tracing, save computing resources, and improve the efficiency of the watermark verification process in large-cardinality artificial intelligence generation and synthesis services.

[0103] In one embodiment, for a target image that is not generated by the watermark embedding method of the above embodiment, Figure 6 A flowchart of another watermark extraction method is provided, which includes the following steps:

[0104] Step S501 : Based on the total diffusion stage of the diffusion model and preset hyper parameters, the forward noise diffusion process is divided into a first forward stage and a second forward stage.

[0105] Based on the total diffusion stage T, the first T / k stages are divided into the first forward stage, and the stages from T / k to T are divided into the second forward stage.

[0106] Step S502, in the first forward stage, after locking the complement of the watermark embedding area in the target image, forward noise diffusion processing is performed.

[0107] In this step, when watermark extraction is performed on the target image generated by the non-target diffusion model, in the first forward stage, the complement of the watermark embedding area is locked in the target image, and only the watermark embedding area of ​​the target image is forward-noised and diffused to try to extract the watermark. Since the target image is generated by the non-target diffusion model, in this step, the extracted watermark embedding area is not highly similar to each target watermark in the watermark library.

[0108] Step S503, in the second forward stage, after unlocking the complement of the watermark embedding area in the target image, forward denoising and diffusion processing is performed.

[0109] In the next second forward stage, we continue to try to extract the watermark to improve the reliability of watermark verification. In this step, the complement of the watermark embedding area is unlocked, and the entire area of ​​the target image is forward-noised and diffused.

[0110] Step S504, comparing the watermarks extracted at different times with multiple preset target watermarks to determine whether the similarity is not less than a preset threshold; if the compared similarities are all lower than the threshold, the forward noise diffusion process is stopped and the target image is determined to have passed the watermark verification.

[0111] In this step, the noise vector of the last watermark embedded area is compared with each target watermark in the watermark library for similarity. If the comparison result shows that the similarity cannot exceed the specified threshold, it can be considered that the target image has passed the watermark verification and is not the target image generated by the target diffusion model.

[0112] In the above steps S501 to S504, when extracting the watermark from the target image generated by the non-target diffusion model, the target image is subjected to forward noise diffusion processing, and hierarchical verification is performed during the forward noise diffusion processing, that is, in the first forward stage of the division, only the watermark embedding area of ​​the image is subjected to forward noise diffusion, and in the second forward stage, the extracted watermark embedding area is compared with the target watermark for similarity in L2 space. If the comparison result is that the similarity cannot exceed the specified threshold, it can be considered that the target image has passed the watermark verification, and the target image generated by the non-target diffusion model. In this way, the watermark extraction process and the forward noise diffusion process are decoupled, which can reduce the running time of the forward noise diffusion stage, speed up the watermark extraction tracing speed, save computing resources, and improve the efficiency of the watermark verification process in large-cardinality artificial intelligence generation and synthesis services.

[0113] In this embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0114] Optionally, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0115] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0116] Determine the geometric center of the target image to be generated;

[0117] Determine the visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0118] According to the watermark embedding area, the watermark is embedded into the Fourier transform of the first noise vector to obtain a second noise vector;

[0119] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark.

[0120] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0121] Determine the visual center of the target image, including:

[0122] The visual center of the target image is predicted based on the forward noise diffusion training process of the diffusion model.

[0123] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0124] According to the target distance from the visual center to the geometric center, the watermark embedding area outside the visual center is determined, including:

[0125] Determining a first range according to a size of the original image, and determining a second range according to a target distance;

[0126] The watermark embedding area is obtained by deducting the second range from the first range.

[0127] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0128] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0129] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the reverse denoising diffusion is divided into the first reverse stage and the second reverse stage;

[0130] In the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed;

[0131] In the second inverse stage, the watermark embedded area is unlocked in the original image and then the inverse denoising diffusion is performed.

[0132] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0133] Before performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark, the method further includes:

[0134] Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image;

[0135] Based on the target constraint, the diffusion model is trained with reverse denoising diffusion.

[0136] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0137] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0138] After the original image is subjected to the reverse denoising diffusion of the first reverse stage and the second reverse stage, an intermediate image is obtained;

[0139] In the intermediate image, the watermark embedded area is compensated for the reverse denoising diffusion process of the first stage to obtain the final target image.

[0140] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0141] Determine the geometric center and visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0142] The target image is forward-noised and diffused to extract the watermark in the watermark embedding area.

[0143] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0144] Perform forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area, including:

[0145] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the forward noise diffusion process is divided into the first forward stage and the second forward stage;

[0146] In the first forward stage, the complement of the watermark embedding area is locked in the target image and then the forward denoising and diffusion process is performed;

[0147] In the second forward stage, the forward denoising and diffusion process is performed after unlocking the complement of the watermark embedded region in the target image.

[0148] In one embodiment, the processor may be configured to perform the following steps through a computer program:

[0149] Compare the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold;

[0150] When the similarity is not less than the threshold, the forward noise diffusion process is stopped;

[0151] When the similarities are lower than the threshold, the forward noise diffusion process is stopped and the target image is determined to have passed the watermark verification.

[0152] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0153] In addition, in combination with the diffusion model intrinsic watermarking method for removing the visual center provided in the above embodiments, a storage medium can also be provided in this embodiment for implementation. The storage medium stores a computer program; when the computer program is executed by a processor, any diffusion model intrinsic watermarking method for removing the visual center in the above embodiments is implemented.

[0154] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0155] Determine the geometric center of the target image to be generated;

[0156] Determine the visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0157] According to the watermark embedding area, the watermark is embedded into the Fourier transform of the first noise vector to obtain a second noise vector;

[0158] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark.

[0159] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0160] Determine the visual center of the target image, including:

[0161] The visual center of the target image is predicted based on the forward noise diffusion training process of the diffusion model.

[0162] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0163] According to the target distance from the visual center to the geometric center, the watermark embedding area outside the visual center is determined, including:

[0164] Determining a first range according to a size of the original image, and determining a second range according to a target distance;

[0165] The watermark embedding area is obtained by deducting the second range from the first range.

[0166] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0167] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0168] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the reverse denoising diffusion is divided into the first reverse stage and the second reverse stage;

[0169] In the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed;

[0170] In the second inverse stage, the watermark embedded area is unlocked in the original image and then the inverse denoising diffusion is performed.

[0171] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0172] Before performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark, the method further includes:

[0173] Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image;

[0174] Based on the target constraint, the diffusion model is trained with reverse denoising diffusion.

[0175] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0176] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0177] After the original image is subjected to the reverse denoising diffusion of the first reverse stage and the second reverse stage, an intermediate image is obtained;

[0178] In the intermediate image, the watermark embedded area is compensated for the reverse denoising diffusion process of the first stage to obtain the final target image.

[0179] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0180] Determine the geometric center and visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0181] The target image is forward-noised and diffused to extract the watermark in the watermark embedding area.

[0182] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0183] Perform forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area, including:

[0184] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the forward noise diffusion process is divided into the first forward stage and the second forward stage;

[0185] In the first forward stage, the complement of the watermark embedding area is locked in the target image and then the forward denoising and diffusion process is performed;

[0186] In the second forward stage, the forward denoising and diffusion process is performed after unlocking the complement of the watermark embedded region in the target image.

[0187] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0188] Compare the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold;

[0189] When the similarity is not less than the threshold, the forward noise diffusion process is stopped;

[0190] When the similarities are lower than the threshold, the forward noise diffusion process is stopped and the target image is determined to have passed the watermark verification.

[0191] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0192] In one embodiment, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the following steps:

[0193] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0194] Determine the geometric center of the target image to be generated;

[0195] Determine the visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0196] According to the watermark embedding area, the watermark is embedded into the Fourier transform of the first noise vector to obtain a second noise vector;

[0197] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark.

[0198] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0199] Determine the visual center of the target image, including:

[0200] The visual center of the target image is predicted based on the forward noise diffusion training process of the diffusion model.

[0201] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0202] According to the target distance from the visual center to the geometric center, the watermark embedding area outside the visual center is determined, including:

[0203] Determining a first range according to a size of the original image, and determining a second range according to a target distance;

[0204] The watermark embedding area is obtained by deducting the second range from the first range.

[0205] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0206] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0207] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the reverse denoising diffusion is divided into the first reverse stage and the second reverse stage;

[0208] In the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed;

[0209] In the second inverse stage, the watermark embedded area is unlocked in the original image and then the inverse denoising diffusion is performed.

[0210] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0211] Before performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark, the method further includes:

[0212] Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image;

[0213] Based on the target constraint, the diffusion model is trained with reverse denoising diffusion.

[0214] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0215] The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark, including:

[0216] After the original image is subjected to the reverse denoising diffusion of the first reverse stage and the second reverse stage, an intermediate image is obtained;

[0217] In the intermediate image, the watermark embedded area is compensated for the reverse denoising diffusion process of the first stage to obtain the final target image.

[0218] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0219] Determine the geometric center and visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center;

[0220] The target image is forward-noised and diffused to extract the watermark in the watermark embedding area.

[0221] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0222] Perform forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area, including:

[0223] Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the forward noise diffusion process is divided into the first forward stage and the second forward stage;

[0224] In the first forward stage, the complement of the watermark embedding area is locked in the target image and then the forward denoising and diffusion process is performed;

[0225] In the second forward stage, the forward denoising and diffusion process is performed after unlocking the complement of the watermark embedded region in the target image.

[0226] In one embodiment, the computer program, when executed by a processor, implements the following steps:

[0227] Compare the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold;

[0228] When the similarity is not less than the threshold, the forward noise diffusion process is stopped;

[0229] When the similarities are lower than the threshold, the forward noise diffusion process is stopped and the target image is determined to have passed the watermark verification.

[0230] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0231] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0232] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0233] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0234] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A diffusion model endogenous watermarking method for removing visual center, characterized in that: include: Determine the geometric center of the target image to be generated; Determining the visual center of the target image, and determining the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center; According to the watermark embedding area, embedding the watermark into the Fourier transform of the first noise vector to obtain a second noise vector; The original image is input into the diffusion model, and the original image is subjected to reverse denoising diffusion based on the second noise vector to generate a target image embedded with a watermark.

2. The diffusion model endogenous watermarking method for removing the visual center according to claim 1 is characterized in that: Determining a watermark embedding area outside the visual center according to a target distance from the visual center to the geometric center includes: Determine a first range according to the size of the original image, and determine a second range according to the target distance; The watermark embedding area is obtained by deducting the second range from the first range.

3. The diffusion model endogenous watermarking method for removing the visual center according to claim 1 is characterized in that: Inputting the original image into the diffusion model, performing reverse denoising diffusion on the original image based on the second noise vector, and generating a target image embedded with a watermark, comprising: Based on the total diffusion stage of the diffusion model and preset hyperparameters, the reverse denoising diffusion is divided into a first reverse stage and a second reverse stage; In the first reverse stage, the watermark embedding area is locked in the original image and then reverse denoising diffusion is performed; In the second reverse stage, reverse denoising diffusion is performed after unlocking the watermark embedded area in the original image.

4. The diffusion model endogenous watermarking method for removing the visual center according to claim 3 is characterized in that: Before inputting the original image into the diffusion model and performing reverse denoising diffusion on the original image based on the second noise vector to generate a target image embedded with a watermark, the method further includes: Establishing a target constraint; wherein the target constraint includes: establishing a consistency constraint between the inner boundary of the watermark embedding area and the extension direction of the visual center; and / or establishing a spectral distribution consistency constraint between the target image and the original image; Based on the target constraint, the diffusion model is trained for reverse denoising diffusion.

5. The diffusion model endogenous watermarking method for removing the visual center according to claim 4 is characterized in that: Inputting the original image into the diffusion model, performing reverse denoising diffusion on the original image based on the second noise vector, and generating a target image embedded with a watermark, comprising: After the original image undergoes reverse denoising diffusion in the first reverse stage and the second reverse stage, an intermediate image is obtained; In the intermediate image, the watermark embedding area is compensated for the reverse denoising diffusion process of the first stage to obtain the final target image.

6. A diffusion model endogenous watermarking method for removing visual center, characterized in that: include: Determine the geometric center and the visual center of the target image, and determine the watermark embedding area outside the visual center according to the target distance from the visual center to the geometric center; The target image is subjected to forward noise diffusion processing to extract the watermark in the watermark embedding area.

7. The diffusion model endogenous watermarking method for removing visual center according to claim 6 is characterized in that: Performing forward noise diffusion processing on the target image to extract the watermark in the watermark embedding area includes: Based on the total diffusion stage of the diffusion model and the preset hyperparameters, the forward noise diffusion process is divided into a first forward stage and a second forward stage; In the first forward stage, a forward noise diffusion process is performed after locking the complement of the watermark embedding area in the target image; In the second forward stage, forward denoising and diffusion processing is performed after unlocking the complement of the watermark embedding area in the target image.

8. The diffusion model endogenous watermarking method for removing the visual center according to claim 7 is characterized in that: The method further comprises: Compare the watermarks extracted at different times with the target watermark to determine whether the similarity is not less than a preset threshold; When the similarity is not less than the threshold, the forward noise diffusion process is stopped; When the similarities are lower than the threshold, the forward noise diffusion process is stopped, and it is determined that the target image passes the watermark verification.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images