Single-stage training method of end-to-end digital watermark model, image watermark identification method and related equipment

Abstract

This application provides a single-stage training method for an end-to-end digital watermarking model, an image watermark identification method, and related equipment. The training method includes: forward input of an image and watermark information to be embedded into a convolutional neural network and a first reversible neural network model; fusion processing; forward input into a second reversible neural network model, adding noise from a real-world scene simulated by the second reversible neural network to obtain a noisy watermark image; reverse input into the second reversible neural network model, reverse fusion processing, reverse input into the first reversible neural network model and the convolutional neural network to obtain extracted watermark information and a restored image; training based on a first difference between the watermark information to be embedded and the extracted watermark information, a second difference between the first image and the noisy watermark image, and a third difference between the first image and the restored image to obtain the application parameters of the trained end-to-end digital watermarking model.

Description

Technical Field

[0001] This application relates to the field of multimedia content security technology, and in particular to a single-stage training method for an end-to-end digital watermarking model, an image watermark identification method, and related equipment. Background Technology

[0002] Copyright protection and content authentication for digital media have become increasingly prominent issues. With the widespread dissemination and ease of editing of digital images and videos, traditional security measures are struggling to meet the growing demands for copyright protection. Digital watermarking technology, by embedding imperceptible information markers into digital media, provides a means for copyright owners to track and verify content. However, the practical application of this technology faces multiple challenges, especially in complex and variable noise environments. Maintaining the stability and reliability of the watermark has become a pressing technical problem that needs to be solved. Summary of the Invention

[0003] In view of this, the purpose of this application is to propose a single-stage training method and related equipment for an end-to-end digital watermarking model.

[0004] To achieve the above objectives, this application provides a single-stage training method for an end-to-end digital watermarking model, including:

[0005] Obtain the first image and the watermark information to be embedded; the first image is the original image.

[0006] The first image and the watermark information to be embedded are input into a convolutional neural network in the forward direction to obtain multiple frequency domains of the first image. Based on the noise type in the actual scene, the target embedding frequency domain of the watermark information in the first image is determined. The watermark information is diffused and the first image is preprocessed to obtain the second image.

[0007] The watermark information to be embedded and the second image are input into the first reversible neural network model in a positive direction, and the watermark information to be embedded is embedded into the second image to obtain the third image;

[0008] The third image is fused to scale a portion of it according to an intensity factor, resulting in a fourth image.

[0009] The fourth image is fed into the second reversible neural network model in a positive direction, and noise from the actual scene simulated by the second reversible neural network is added to the fourth image to obtain a watermarked image with noise.

[0010] The noisy watermark image is input in reverse to the second reversible neural network model for non-differentiable noise removal processing.

[0011] The watermark image after non-differentiable noise removal is subjected to reverse fusion processing.

[0012] The watermark image after reverse fusion is input into the first reversible neural network model to extract watermark information.

[0013] The watermark image after watermark information extraction is input in reverse into the convolutional neural network to extract the target embedding frequency domain watermark information, thereby obtaining the extracted watermark information and the restored image;

[0014] Based on the first difference between the watermark information to be embedded and the extracted watermark information, the second difference between the first image and the noisy watermark image, and the third difference between the first image and the recovered image, the convolutional neural network, the first reversible neural network model, and the second reversible neural network model are trained.

[0015] Return to the step of inputting the first image and the watermark information to be embedded into the convolutional neural network in the forward direction, and perform iterative training until the first difference, the second difference and the third difference converge, to obtain the application parameters of the trained end-to-end digital watermarking model; wherein the application parameters include the parameters of the trained convolutional neural network, the parameters of the trained first reversible neural network model and the parameters of the trained second reversible neural network model.

[0016] In some embodiments, multiple frequency domains of the first image are obtained by performing wavelet transform on the first image; the multiple frequency domains include a first frequency domain, a second frequency domain, a third frequency domain, and a fourth frequency domain, wherein the values ​​of the first frequency domain, the second frequency domain, the third frequency domain, and the fourth frequency domain decrease sequentially;

[0017] The step of determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene includes: in response to determining that the noise type includes differentiable noise, determining the first frequency domain as the target frequency domain; or

[0018] In response to determining that the noise type includes non-differentiable noise, the second and fourth frequency domains are determined as the target frequency domains; or

[0019] In response to determining that the noise type includes differentiable noise and non-differentiable noise, the fourth frequency domain is determined as the target frequency domain.

[0020] In some embodiments, the noise type in the actual scene includes differentiable noise and non-differentiable noise; adding noise from the actual scene simulated based on a second reversible neural network to the fourth image to obtain a noisy watermark image includes:

[0021] The fourth image is subjected to color space conversion, discrete cosine transform processing, quantization processing and encoding to obtain a fifth image with non-differentiable noise;

[0022] Differentiable noise is added to the fifth image to obtain a watermarked image with mixed noise.

[0023] The diffusion processing of the watermark information and the preprocessing of the first image include:

[0024] The first image is subjected to wavelet transform processing to decompose the first image into multiple frequency domains;

[0025] The watermark information after diffusion processing is embedded in the target embedding frequency domain in the multiple frequency domains.

[0026] In some embodiments, the color space conversion process includes converting the fourth image from the RGB color space to the YUV color space, using the following conversion formula: Y = 0.299R + 0.587G + 0.114B; where Y is luminance; R is the value of the red channel, G is the value of the green channel, B is the value of the blue channel; U is chromaticity, V is chromaticity; U = -0.14713R - 0.28886G + 0.436B + 128; V = 0.615R - 0.51499G - 0.10001B + 128.

[0027] In some embodiments, the discrete cosine transform (DCT) process includes: segmenting the spatially transformed fourth image into multiple image blocks; performing a discrete cosine transform on each image block to convert the data of the image block from the spatial domain to the frequency domain; the discrete cosine transform process is performed using the formula... The expression is performed, where F(u,v) are the discrete cosine transform coefficients in the frequency domain, f(x,y) are the pixel values ​​in the spatial domain, and α(u) and α(v) are the normalization coefficients, respectively.

[0028] In some embodiments, the quantization process is performed via formula Proceeding, in which F q (u,v) are the quantized discrete cosine transform coefficients, and Q(u,v) is the quantization step size in the quantization table.

[0029] In some embodiments, the fusion process on the third image includes:

[0030] The third image is segmented in a first direction to obtain a first sub-image and a second sub-image; the first direction is perpendicular to the plane containing the third image.

[0031] A portion of the first sub-image is scaled according to an intensity factor to obtain the processed first sub-image.

[0032] The processed first sub-image is then fused with the second sub-image.

[0033] In some embodiments, the step of acquiring multiple frequency domains of the first image, determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene, performing diffusion processing on the watermark information, and preprocessing the first image to obtain the second image;

[0034] Embedding the watermark information to be embedded into the second image to obtain the third image includes: inputting the watermark information to be embedded and the second image into a first reversible neural network model in a forward direction, and outputting the third image; the first reversible neural network model includes alternating InvertibleConv1x1 layers and InvArch layers; the InvertibleConv1x1 layer realizes information exchange between channels through 1x1 convolution; the InvArch layer includes multiple dense blocks for splitting, extracting and fusing feature maps;

[0035] The step of performing color space conversion, discrete cosine transform processing, quantization processing, and encoding on the fourth image to obtain a fifth image with non-differentiable noise includes: inputting the fourth image into the second reversible neural network model in the forward direction and outputting the fifth image.

[0036] In some embodiments, the first difference includes the bit error rate between the watermark information to be embedded and the extracted watermark information;

[0037] The second difference includes the pixel mean square error between the first image and the noisy watermark image;

[0038] The third difference includes the pixel mean square error between the first image and the restored image;

[0039] The loss function for training includes: L T =λ I I I +λ W L W +λ R L R ; where λ I , λ W and λ R These are the weighting coefficients; L I L is the image loss function; W L is the watermark information loss function; R This is the image reconstruction loss function.

[0040] This application also provides an image watermark identification method, including:

[0041] Acquire the original image, the target watermark information, and the image containing the watermark information to be extracted; wherein the image containing the watermark information to be extracted has the same image content as the original image.

[0042] The image from which the watermark information to be extracted is input into the end-to-end digital watermarking model to obtain the predicted watermark information;

[0043] Determine whether the predicted watermark information is the same as the target watermark information.

[0044] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the preceding claims.

[0045] This application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform any of the methods described above.

[0046] This application also provides a computer program product, including computer program instructions that, when run on a computer, cause the computer to perform the method described in any of the preceding claims.

[0047] As can be seen from the above, the single-stage training method of the end-to-end digital watermarking model provided in this application involves acquiring a first image and watermark information to be embedded; the first image is the original image; the first image and the watermark information to be embedded are forward-inputted into a convolutional neural network to obtain multiple frequency domains of the first image; based on the noise type in the actual scene, the target embedding frequency domain of the watermark information in the first image is determined; the watermark information is diffused, and the first image is preprocessed to obtain a second image; the watermark information to be embedded and the second image are forward-inputted into a first reversible neural network model to embed the watermark information to be embedded into the second image to obtain a third image; the third image is fused to scale a portion of the third image according to an intensity factor to obtain a fourth image; the fourth image is forward-inputted into a second reversible neural network model to add noise from the actual scene simulated by the second reversible neural network to the fourth image to obtain a noisy watermark image; the noisy watermark image is reverse-inputted into the second reversible neural network model for non-differentiable noise removal processing; the watermark image after non-differentiable noise removal processing is then processed... The process involves: 1) performing reverse fusion processing on the watermarked image; 2) inputting the reverse-fused watermarked image into the first reversible neural network model for watermark information extraction; 3) inputting the watermarked image after watermark information extraction into the convolutional neural network for target embedding frequency domain watermark information extraction, resulting in extracted watermark information and a restored image; 4) training the convolutional neural network, the first reversible neural network model, and the second reversible neural network model based on the first difference between the watermark information to be embedded and the extracted watermark information, the second difference between the first image and the noisy watermarked image, and the third difference between the first image and the restored image; 5) returning to the step of inputting the first image and the watermark information to be embedded into the convolutional neural network in the forward direction for iterative training until the first difference, the second difference, and the third difference converge, obtaining the application parameters of the trained end-to-end digital watermarking model; wherein the application parameters include the parameters of the trained convolutional neural network, the parameters of the trained first reversible neural network model, and the parameters of the trained second reversible neural network model; 6) effectively resisting attacks from various noises in real-world scenarios, improving the robustness of digital watermarking. Furthermore, the ability to select the most suitable embedding domain based on the type of noise can improve the imperceptibility of digital watermarks. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a flowchart illustrating a single-stage training method for an end-to-end digital watermarking model according to an embodiment of this application.

[0050] Figure 2 This is a schematic diagram of the model architecture of a single-stage training method for an end-to-end digital watermarking model according to an embodiment of this application.

[0051] Figure 3 This is a schematic diagram illustrating the process of generating a watermark image during use of the end-to-end digital watermarking model according to an embodiment of this application;

[0052] Figure 4 This is a comparison diagram of the original image, the watermarked image, and the watermarked image with mixed noise in an embodiment of this application; wherein, the first row is the original image, the second row is the image after adding the watermark, and the third row is the noisy image;

[0053] Figure 5 This is a flowchart illustrating the image watermark identification method according to an embodiment of this application;

[0054] Figure 6 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0056] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0057] Two key factors in evaluating the performance of digital watermarking algorithms are robustness and imperceptibility. Robustness refers to the ability of the watermark information embedded in an image to survive various distortions. Imperceptibility means that the image after watermarking is almost identical to the original image. Specifically, digital watermarks must be able to resist various intentional or unintentional signal processing attacks, including but not limited to compression, filtering, geometric transformations, and noise superposition. These factors can all affect the detectability and integrity of the watermark. Therefore, researchers are dedicated to developing more advanced watermarking algorithms to improve their robustness to noise interference and ensure accurate extraction of watermark information even under extreme conditions.

[0058] To address the problem of various types of noise attacks, a common approach is to use a noise layer to simulate different attack types and train it to resist the simulated noise using an "encoder-noise layer-decoder" structure. Alternatively, an additional decoder can be added, and a noise discriminator can be used to decode based on different noise types.

[0059] In recent years, deep learning technology has developed rapidly, and its application in the field of digital watermarking has become increasingly widespread. One related technique involves embedding a watermark into a carrier image to obtain a watermarked image, and then extracting the watermark from this image; the watermark is a sequence of bit information. This method consists of a reversible module comprising two branches—one for the carrier image and one for the watermark—and a noise layer. The forward process embeds the watermark into the carrier image to synthesize the watermarked image, and the watermarked image is very similar to the original carrier image. The reverse process separates and extracts the watermark from a single watermarked image. Furthermore, this method also includes related processing of the bit information sequence. Another related technique involves acquiring a carrier image and the watermark to be embedded; performing diffusion processing on the carrier image and the watermark; inputting the diffused carrier image and the watermark into a reversible neural network (INN) to obtain a first image, which is the image after the watermark is embedded; separating the watermark information from the first image based on the first image and the diffused carrier image; and obtaining a target image based on the watermark information and the diffused carrier image, which is the image after the watermark to be embedded is inserted into the carrier image. These methods suffer from poor watermark performance when differentiable and non-differentiable noise are superimposed.

[0060] Based on this, this application provides an end-to-end single-stage training digital watermarking method and related equipment. The carrier image is preprocessed using an Image Preprocessing (IPM) module, and the watermark information to be embedded is preprocessed using a Watermark Diffusion (WDM) module. The preprocessed image and watermark information are then fed into a Watermark Embedding (WEM) module to obtain a watermark image. The watermark image is then processed by a fusion module to monitor the overall characteristics of the image, ensuring that the overall quality and features of the image remain stable after watermark embedding. Finally, the watermark image is processed by a Non-differentiable Noise Module (NNPM) and a Differentiable Noise Pool to obtain a noisy image containing the watermark. In this way, the watermark embedding and simulation of non-differentiable noise are achieved through a reversible neural network (INN), enabling successful watermark extraction even under conditions of superimposed differentiable and non-differentiable noise. This effectively resists attacks from various types of noise in real-world scenarios, improving the robustness of the digital watermark. It also addresses, to a certain extent, the problem of maintaining the stability and reliability of watermarks in complex and variable noise environments.

[0061] Figure 1 A flowchart illustrating an exemplary single-stage training method for an end-to-end digital watermarking model according to an embodiment of this application is shown.

[0062] The single-stage training method for the end-to-end digital watermarking model provided in this application embodiment may include:

[0063] S110, Obtain the first image and the watermark information to be embedded; the first image is the original image;

[0064] S120: Input the first image and the watermark information to be embedded into the convolutional neural network in the forward direction to obtain multiple frequency domains of the first image. Determine the target embedding frequency domain of the watermark information in the first image according to the noise type in the actual scene. Perform diffusion processing on the watermark information and preprocess the first image to obtain the second image.

[0065] S130, the watermark information to be embedded and the second image are input into the first reversible neural network model in a positive direction, and the watermark information to be embedded is embedded into the second image to obtain the third image;

[0066] S140, perform a fusion process on the third image to scale a portion of the third image according to an intensity factor to obtain a fourth image;

[0067] S150, the fourth image is input forward into the second reversible neural network model, and noise from the actual scene simulated by the second reversible neural network is added to the fourth image to obtain a watermarked image with noise;

[0068] S210, the noisy watermark image is input in reverse to the second reversible neural network model for non-differentiable noise removal processing;

[0069] S220, performs reverse fusion processing on the watermark image after the non-incremental noise removal process;

[0070] S230, the watermark image after reverse fusion processing is input into the first reversible neural network model to extract watermark information;

[0071] S240, the watermark image after watermark information extraction is input in reverse to the convolutional neural network to extract the target embedding frequency domain watermark information, and obtain the extracted watermark information and the restored image;

[0072] S250, based on the first difference between the watermark information to be embedded and the extracted watermark information, the second difference between the first image and the noisy watermark image, and the third difference between the first image and the recovered image, the convolutional neural network, the first reversible neural network model, and the second reversible neural network model are trained.

[0073] S260, return to the step of forward inputting the first image and the watermark information to be embedded into the convolutional neural network, perform iterative training until the first difference, the second difference and the third difference converge, and obtain the application parameters of the trained end-to-end digital watermarking model; wherein the application parameters include the parameters of the trained convolutional neural network, the parameters of the trained first reversible neural network model and the parameters of the trained second reversible neural network model.

[0074] In step S110, the watermark information to be embedded is typically a hidden watermark, i.e., an imperceptible and invisible watermark. The watermark information to be embedded can be in the form of a digital sequence, a matrix, or an image, depending on the requirements. Typically, the first image can have multiple frequency domains. In some embodiments, the multiple frequency domains may include a first frequency domain, a second frequency domain, a third frequency domain, and a fourth frequency domain, wherein the values ​​of the first, second, third, and fourth frequency domains decrease sequentially. Typically, the first frequency domain can be a high-frequency region (HH band), the second region can be a mid-frequency region (HL band), the third frequency domain can be a mid-low-frequency region (LH band), and the fourth frequency domain can be a low-frequency region (LL band).

[0075] In step S120, noise in a real-world scenario typically includes differentiable noise, non-differentiable noise, and mixed noise that includes both differentiable and non-differentiable noise. Differentiable noise can be, for example, salt-and-pepper noise, Gaussian noise, Poisson noise, and multiplicative noise. Non-differentiable noise can be, for example, JPEG compression.

[0076] In some embodiments, determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene may include: determining the first frequency domain as the target frequency domain in response to determining that the noise type includes differentiable noise. By leveraging the sensitivity of high-frequency regions to small perturbations and the characteristics of differentiable noise, optimization techniques can effectively minimize the impact of the frequency domain on the watermark, thus giving the watermark good robustness.

[0077] In some embodiments, determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene may include: determining the second frequency domain and the fourth frequency domain as the target frequency domain in response to determining that the noise type includes non-differentiable noise. By embedding the watermark in the mid-to-low frequency region, it can have better tolerance to nonlinear distortions such as compression, thus enabling the watermark to maintain high recognizability and integrity even after undergoing common image processing operations. This results in a more robust effect.

[0078] In some embodiments, determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene may include: determining the fourth frequency domain as the target frequency domain in response to determining that the noise type includes differentiable noise and non-differentiable noise. This can maintain the integrity and robustness of the watermark to a certain extent.

[0079] In some embodiments, the diffusion processing of the watermark information and the preprocessing of the first image include:

[0080] The first image is subjected to wavelet transform processing, which decomposes the first image into multiple frequency domains. That is, the multiple frequency domains of the first image are obtained by performing wavelet transform processing on the first image.

[0081] The watermark information after diffusion processing is embedded in the target embedding frequency domain in the multiple frequency domains. For example... Figure 2 As shown, it should be understood that obtaining multiple frequency domains of the first image, determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene, performing diffusion processing on the watermark information, and preprocessing the first image to obtain the second image can be achieved by: forward inputting the first image and the watermark information to be embedded into a convolutional neural network.

[0082] like Figure 3 As shown, forward propagation is performed in the convolutional neural network as follows: wavelet transform processing, downsampling processing, linear processing, normalization to enhance feature representation, and upsampling processing are applied to the first image; the watermark information is linearly transformed and reshaped, connected with the feature map obtained from the upsampling processing, and subjected to Haar transform to output / obtain the second image.

[0083] In practical applications, the convolutional neural network can be a watermark diffusion module (WDM) and an image preprocessing module (IPM). The module performs preprocessing of the original image and watermark diffusion. The original image is decomposed into different frequency domains using wavelet transform, then downsampled using a downsampling convolutional layer (DownConv), followed by linear layers and batch normalization layers, along with a ReLU activation function to enhance feature representation. Subsequently, the image resolution is gradually restored using an upsampling convolutional layer (UpConvt), while the message tensor undergoes linear transformation and reshaping, concatenated with the upsampled feature map, and subjected to Haar transform to achieve effective diffusion of the watermark information.

[0084] In step S130, it should be understood that embedding the watermark information to be embedded into the second image to obtain the third image can be achieved by forward inputting the watermark information to be embedded and the second image into the first reversible neural network model.

[0085] It should be understood that in this step, embedding watermark information can be understood as a full-coverage form, which means encoding the watermark information throughout the entire image (e.g., the second image).

[0086] In some of these embodiments, such as Figure 2As shown, the first reversible neural network model (e.g., the WEM watermark embedding module) may include operation lists of InvertibleConv1x1 (reversible convolutional block) layers and InvArch layers, which are arranged alternately to support forward and backward propagation. In the forward operation, the first reversible neural network model (e.g., the reversible module) sequentially executes these layers to embed watermark information in the second image to obtain the third image. Specifically, the InvertibleConv1x1 layer can achieve information exchange between channels through 1x1 convolutions while maintaining the same spatial size; while the InvArch layer uses three dense blocks to split and process the feature map in depth. Each dense block consists of multiple convolutional layers to extract and fuse features. That is, the InvertibleConv1x1 layer achieves information exchange between channels through 1x1 convolutions; the InvArch layer includes multiple dense blocks for splitting, extracting, and fusing the feature map. This network hierarchy can achieve watermark concealment and robustness to a certain extent, while also enabling lossless recovery of watermark information, providing an innovative reversible processing solution for digital watermarking technology.

[0087] In practical applications, the input to the WEM watermark embedding module can be an image and a watermark message. The image is a pre-processed image or an image with partially embedded information, with dimensions [b, 1, 128, 128] / [batch size, number of channels, height, width]. The message is the watermark information to be embedded, which can be a binary sequence with dimensions [b, 256] / [batch size, message length]. The output can be a processed image or extracted message: in embedding mode, the output is an image with the watermark embedded, with dimensions consistent with the input image. In extraction mode, the output is the watermark information extracted from the image, with dimensions consistent with the input message. During training, forward propagation or backward propagation can be used for inverse operations. The WEM forward method determines whether to perform forward or backward operations based on the `rev` parameter. This model mainly consists of two types of layers: `InvertibleConv1x1` and `InvArch`. They are applied alternately, and each operation can perform forward and backward propagation to perform the inverse operation.

[0088] The processing flow may include: module initialization, creating a list of modules `self.operations` through configuration (opt). Based on the number of `block_num` (16 reversible blocks), iteratively adding `InvertibleConv1x1` layers (if `use_conv1x1` is true) and `InvArch` layers to `self.operations`. Forward propagation (the `forward` method): If the `rev` parameter is false (indicating forward operation, i.e., embedding the watermark), each operation layer is applied to the input data `x` in the order it was added. If the `rev` parameter is true (indicating reverse operation, i.e., extracting the watermark), each operation layer is applied to the input data `x` in the reverse order it was added. Returning the data after processing by all operation layers. `InvertibleConv1x1` layer: Contains a reversible 1x1 convolution operation. The number of input and output channels is determined by `split1_img + split2_repeat`. (In the context of reversible blocks, the input feature map is typically split into two parts: split1_img represents the number of channels in the first part; split2_repeat represents the number of channels used to pass additional watermark information.) The number of input and output neurons per channel is equal to the image height multiplied by its width = H × W (since it's a 1x1 convolution, the spatial dimensions remain unchanged). InvArch layer: Contains three DenseBlocks for feature processing on different parts of the image. The input feature map is split into two parts, with lengths determined by split_len1 and split_len2. (split_len1 and split_len2 are parameters defining the size of the input feature map split. Feature map splitting here refers to dividing the input feature map (usually the output of a layer in the middle of a convolutional neural network) into two parts along the channel dimension for independent processing.) Each DenseBlock accepts and processes one part of the feature map, then combines it with the other part. The number of output neurons depends on the size and number of channels of the split feature map. DenseBlock: Each DenseBlock is a densely connected block composed of multiple convolutional layers, used for feature extraction and fusion. The number of input and output neurons depends on the block configuration and the size of the input feature map.

[0089] In some embodiments, step S140, the fusion processing of the third image may include:

[0090] The third image is segmented in a first direction to obtain a first sub-image and a second sub-image; the first direction is perpendicular to the plane containing the third image. Typically, the first direction can be understood as the thickness direction of the image, or a direction perpendicular to the plane containing the image as its base.

[0091] A portion of the first sub-image is scaled according to an intensity factor to obtain the processed first sub-image. The intensity factor S is a parameter used to control the intensity of the hidden information. It determines the degree of influence of the hidden information on the output image. The selection of the intensity factor S is usually based on experimental and application requirements. The optimal S value can typically be determined by adjusting the value of S and observing its impact on image quality and information extraction effectiveness. Furthermore, the value of S can be adjusted according to the characteristics of the image and the application scenario of the watermark (such as copyright protection, content authentication, etc.). This avoids an excessively large S value, which affects the naturalness of the image; and also avoids an excessively small S value, where the hidden information is not prominent enough and difficult to extract effectively.

[0092] The processed first sub-image is then fused with the second sub-image.

[0093] This fusion process allows for the embedding of watermark information into the second image while maintaining the visual quality of the resulting third image. By segmenting the second image and selectively hiding information, covert storage and subsequent retrieval of information can be achieved without significantly affecting image quality. This enables monitoring of the overall image characteristics and improves the stability of the overall image quality and features after watermark embedding.

[0094] In practical applications, during the forward propagation (not rev) of the fusion process, the input `encoded_img` is the encoded image. The `self.split1_img` parameter represents the split point, dividing the image into two parts: the first part is a portion of the image, and the second part contains the hidden message. The message portion is extracted from `encoded_img` and stored in the variable `msg`. The message is hidden using `cover_down` and `self.strength_factor`. (`cover_down` covers the image, and `self.strength_factor` is the strength factor for the hidden message.) The output image `out` with the hidden message is returned. During the backward propagation (rev) of the fusion process, the information is extracted. It copies the input `encoded_img` to form the output `out`. This operation does not change the image; it simply copies `encoded_img` to extract the hidden message.

[0095] In step S150, in some embodiments, the noise type in the actual scene includes differentiable noise and non-differentiable noise. Obtaining a noisy watermark image from the noise in the actual scene simulated by the second reversible neural network in the fourth image may include:

[0096] The fourth image is subjected to color space conversion, discrete cosine transform processing, quantization processing and encoding to obtain a fifth image with non-differentiable noise;

[0097] Differentiable noise is added to the fifth image to obtain a watermarked image with mixed noise.

[0098] In some embodiments, the color space conversion process may include converting the fourth image from the RGB color space to the YUV color space, using the following conversion formulas: Y = 0.299R + 0.587G + 0.114B; where Y is luminance; R is the value of the red channel, G is the value of the green channel, B is the value of the blue channel; U is chrominance, V is chrominance; U = -0.14713R - 0.28886G + 0.436B + 128; V = 0.615R - 0.51499G - 0.10001B + 128. This achieves the color space conversion of the fourth image.

[0099] In some embodiments, the Discrete Cosine Transform (DCT) processing may include: segmenting the spatially transformed fourth image into multiple image blocks (e.g., segmenting into 8x8 image blocks); performing a Discrete Cosine Transform on each image block to transform the data of the image block from the spatial domain to the frequency domain; the Discrete Cosine Transform processing is performed using the formula... The process is performed, where F(u,v) are the discrete cosine transform coefficients in the frequency domain, f(x,y) are the pixel values ​​in the spatial domain, and α(u) and α(v) are the normalization coefficients. Here, u and v are the frequency domain indices in the DCT transform, representing the positions of the frequency domain coefficients obtained after the transform. In the 8x8 image blocks obtained from image segmentation, each element of each block is represented by a pair of indices (u,v), where u and v vary from 0 to 7, representing 8 possible horizontal and vertical frequency positions. x and y are the pixel position indices in the spatial domain, used to represent the positions of pixels in the original image. In the 8x8 image blocks obtained from image segmentation, x and y also vary from 0 to 7, representing the horizontal and vertical positions of the 8 pixels, respectively.

[0100] In some embodiments, the quantization process can be performed using the formula... Proceeding, in which F q (u,v) represents the quantized Discrete Cosine Transform (DCT) coefficients. The original DCT coefficients F(u,v) are converted into Fq(u,v) through the quantization process. Q(u,v) is the quantization step size in the quantization table. Quantization allows the DCT coefficients to be quantized, making the coefficients in the high-frequency range zero or close to zero, thus achieving a compression effect.

[0101] This enables subsequent inverse operations on non-differentiable noise, making it easier to extract watermarks from images with embedded non-differentiable noise.

[0102] In some embodiments, the process of performing color space conversion, discrete cosine transform processing, quantization processing, and encoding on the fourth image to obtain a fifth image with non-differential noise can be implemented by a second reversible neural network model (e.g., a reversible block), which may include: inputting the fourth image into the second reversible neural network model in the forward direction and outputting the fifth image.

[0103] In some possible embodiments, the structure of the second reversible neural network model (e.g., a reversible block) may include three sub-networks F, G, and H, which work together to achieve a reversible transformation of the data. During the forward propagation phase (i.e., after the forward input), the input tensor x (e.g., the fourth image) can be partitioned into two parts, x1 and x2. Sub-networks F and G process the partitioned tensor, while sub-network H generates a scaling factor s1 that controls the reversible operation. The stability of the scaling factor s1 and the control of the output are ensured in sub-network H through the LeakyReLU activation function and the clamp parameter. Finally, through a series of operations, including the transformations of networks F and G and the scaling factor generated by network H, the reversible transformation of the input tensor is achieved.

[0104] In some embodiments, multiple differentiable noises can be added to the fifth image. For example, several differentiable noises can be added to the fifth image by passing it through a differentiable noise pool.

[0105] In practical applications, reversible blocks can include: 1. Weight initialization functions, including `initialize_weights`: Initializes the weights of convolutional and linear layers in the network using the Kaiming initialization method (He initialization), and initializes the weights and biases of the batch normalization layer. `initialize_weights_xavier`: Initializes the weights of convolutional and linear layers in the network, as well as the batch normalization layer, using the Xavier initialization method. 2. Layer construction functions, including `make_layer`: Creates a sequential list of neural network layers based on a given block and number of layers (n_layers). 3. Residual blocks, including `ResidualBlock_noBN`: Defines a residual block without batch normalization (BN), containing two convolutional layers using the ReLU activation function. 4. Optical flow transformation functions, including `flow_warp`: Performs optical flow transformation based on the input feature map (x) and optical flow (flow), using the selected interpolation mode (interp_mode) and padding mode (padding_mode). 5. Dense Blocks and Convolutional Blocks, including DenseBlock: Defines a dense block containing multiple convolutional layers and LeakyReLU activation functions, using the DenseNet architecture. ConvBlock: Defines a convolutional block containing multiple convolutional layers and LeakyReLU activation functions. 6. Subnet Construction Functions, including subnet: Returns a constructor function based on the provided network structure (net_structure) to create a DenseBlock or ConvBlock. 7. Reversible Network Blocks, including InvBlockExp: Defines a reversible network block containing three subnetworks (F, G, H), implementing reversible forward and backward propagation. This block can be used to build reversible neural network architectures. The InvBlockExp class defines a reversible network block, a key component in building reversible neural networks. Reversible network blocks allow forward and backward propagation of the network to be performed with constant time complexity because they do not require storing or computing gradient information.

[0106] In some possible embodiments, the reversible block can be of the InvBlockExp class, with class attributes including: channel_num: the number of channels in the input tensor; channel_split_num: the number of channels to be split in the reversible block; clamp: a scaling factor used to control the output of the activation function in the reversible block. The initialization method (__init__) includes receiving channel_num and channel_split_num as input and determining how the tensor is split. subnet_constructor: a function used to create subnets based on the provided network structure. Here, the subnet function is used, which can return a constructor to create a DenseBlock or ConvBlock. F, G, H: three subnets, each used for different operations in the reversible block. F and G are used for state transitions, while H is used to generate the scaling factor s. Forward propagation includes: x: the input tensor; rev: a boolean value indicating whether backward propagation is performed. The tensor x is split into x1 and x2, based on channel_split_num. If rev is False (forward propagation): y1 is calculated as x1 plus the result passed through the F network via x2. `s` is calculated as the scaling and activation function output of H(y1). `y2` is calculated as x2 multiplied by exp(s) plus the result passed through the G network via y1. If `rev` is True (backpropagation): `s` is calculated as the scaling and activation function output of H(x1). `y2` is calculated as x2 minus the result passed through the G network via x1, then divided by exp(s). `y1` is calculated as x1 minus the result passed through the F network via y2. The Jacobian matrix determinant is the sum of the Jacobian matrix determinants of the invertible blocks, used to stabilize backpropagation during training. `jac`: the sum of the Jacobian matrix determinants. If `rev` is False, `jac` is the sum of `s`; if `rev` is True, it is the sum of -s.

[0107] The single-stage training method for the end-to-end digital watermarking model includes extracting watermark information from a noisy watermark image. Specifically, it may include reversing the steps of obtaining the noisy watermark image, including steps S110 to S150 mentioned above, i.e., performing the process of steps S210 to S260, to extract the watermark information.

[0108] In some embodiments, the process of extracting watermark information from the noisy watermark image to obtain the embedded watermark information (i.e., extracting the watermark information from the noisy watermark image), which is the process of steps S210 to S260, can be understood as follows:

[0109] The watermarked image with mixed noise is subjected to differentiable noise removal processing to obtain a fifth image with non-differentiable noise. Typically, the denoising process can be the reverse of the aforementioned process of adding differentiable noise.

[0110] The fifth image (i.e., the watermark image after differentiable noise removal) is input in reverse to the second reversible neural network model for non-differentiable noise removal; the fourth image is output.

[0111] The fourth image (i.e., the watermarked image after non-differential noise removal) is subjected to reverse fusion processing to obtain the third image. Typically, reverse fusion processing can be the reverse of the aforementioned fusion processing.

[0112] The third image (i.e., the watermarked image after reverse fusion) is input into the first reversible neural network model to extract watermark information and output the watermark information embedded in the overall image (i.e., the second image) and the second image.

[0113] The second image (i.e., the watermarked image after watermark information extraction) is input in reverse to the convolutional neural network to extract the target embedding frequency domain watermark information, and output the watermark information and the first image.

[0114] In some embodiments, step S210 involves backpropagating the input tensor (i.e., the fifth image) into the second reversible neural network model after the fifth image is inversely input. Using subnetworks F and G, and the scaling factor s previously calculated via the LeakyReLU activation function and the clamp parameter, the input tensor (i.e., the fifth image) is inversely processed to recover the original data, resulting in the fourth image. This process not only maintains data integrity but also improves the computational efficiency of the network.

[0115] In some embodiments, the process of obtaining the fourth image from the fifth image may include: inverse quantization and inverse discrete cosine transform. The inverse quantization can be performed using the formula F(u, v) = F... q (u, v) × Q(u, v) is used; dequantization can restore the quantized Discrete Cosine Transform (DCT) coefficients to the original DCT coefficients. The inverse Discrete Cosine Transform can be obtained using the formula... The Inverse Discrete Cosine Transform (IDCT) can convert the frequency domain signal transformed by Discrete Cosine Transform (DCT) back to the spatial domain, thus realizing the restoration of the image.

[0116] In some possible embodiments, the reversible block can also compute the sum of the determinants of the Jacobian matrix, ensuring backpropagation stability during model training (e.g., a second reversible neural network model). This feature provides a significant advantage when building reversible neural networks, especially in applications requiring efficient computation and data recovery. The subnet constructor allows for flexible adaptation to different network structures, enabling highly customized network designs.

[0117] In some embodiments, in step S230, the third image (i.e., the watermark image after reverse fusion processing) is input in reverse to the first reversible neural network model (e.g., a reversible module), and the watermark can be extracted by reversing the operation of the aforementioned forward input.

[0118] In some embodiments, in step S240, the second image (i.e., the watermark image after watermark information extraction) is input in reverse to a convolutional neural network (e.g., a diffusion module), and the watermark information and the first image are output. Specifically, inverse wavelet transform and a series of upsampling and downsampling convolutional layers, combined with batch normalization and ReLU activation function, can be used to gradually recover and extract the original embedded watermark information. Thus, the complete watermark information is extracted.

[0119] In some embodiments, in step S250, the first difference may include a first similarity between the numerical value corresponding to the extracted watermark information and the numerical value corresponding to the watermark information to be embedded. Typically, the watermark information can be in numerical form, such as a value in the form of 0101… The first similarity may be the bit error rate. That is, the first difference includes the bit error rate between the watermark information to be embedded and the extracted watermark information.

[0120] In some embodiments, in step S250, the second difference may include a second similarity between the first image and the noisy watermark image. The second similarity may be the peak signal-to-noise ratio (PSNR). The second similarity may also be the pixel mean square error. That is, the second difference includes the pixel mean square error between the first image and the noisy watermark image.

[0121] In some embodiments, in step S250, the third difference may include a third similarity between the first image and the restored image. The third similarity may be the pixel mean square error. The third difference may include the pixel mean square error between the first image and the restored image. In some embodiments, the training loss function may include: L T =λ I I I +λ W L W +λ R LR ; where λ I , λ W and λ R These are the weighting coefficients; L I L is the image loss function; W L is the watermark information loss function; R Let L be the image reconstruction loss function. I =||I m -WI m || 2 I m For the first image, WI m This is a watermarked image with noise. W =||W m -RW m || 2 W m For the watermark information to be embedded, RW m For the extracted watermark information. L R =||I m -RI m || 2 RI m The image to be restored.

[0122] In practical applications, the training phase can include the following specific steps:

[0123] Neural network initialization phase: Set network parameters, including optimizer (Adam), learning rate (0.1), and number of epochs (50).

[0124] Data preparation: Prepare the training dataset, including the original images and the corresponding watermark information.

[0125] Training loop (i.e., iterative training): For each training cycle (i.e., each training round), the neural network can perform the same training steps. Each round may include the following steps: setting the network to training mode, performing gradient calculation and backpropagation after forward propagation. Image and watermark data are acquired in batches using a data loader; the number of images in each batch can be set to 16, 32, or 48, etc. In each round, the neural network is trained according to the loss function, i.e., the loss function value is reduced, training continues until the loss function value can no longer be reduced.

[0126] In some embodiments, in step S260, the first difference, the second difference, and the third difference converge, which can be understood as training until the value of the loss function cannot be reduced.

[0127] Application examples

[0128] Using the method of this application embodiment, watermark information (watermark information in the form of a digital sequence) is embedded in the original image, and noise containing only micro-noise (Gaussian noise), only non-differentiable noise (JPEG compression), differentiable noise (salt and pepper noise and Gaussian noise) and non-differentiable noise (JPEG) is added respectively to obtain a watermark image with corresponding noise, and the watermark information is extracted.

[0129] The experimental results are shown in Tables 1, 2, and 3. Figure 4 As shown. Among them, Figure 4 The images are presented in three rows: the first row shows the original image, the second row shows the watermarked image, and the third row shows the noisy image with superimposed noise. Table 1-3 shows the robustness and imperceptibility. Robustness corresponds to ACC; imperceptibility corresponds to PSNR.

[0130] Table 1 shows the performance with only differential noise (Gaussian noise).

[0131]

[0132] Table 2 contains only the performance of non-inferable noise (JPEG)

[0133]

[0134] Table 3 Superposition of differential and non-differential noise

[0135]

[0136] Results analysis:

[0137] According to Tables 1, 2, and 3, the proposed method exhibits good robustness and imperceptibility when the watermark is attacked by three different types of noise. In three cases—containing only differentiable noise (Gaussian noise), only non-differentiable noise (JPEG compression), and a superposition of differentiable (salt-and-pepper noise and Gaussian noise) and non-differentiable noise (JPEG)—the proposed method can extract watermark information well without affecting the carrier image. The experimental data show that embedding the digital watermark in the high-frequency region (HH band) has the best performance when only affected by differentiable noise. When facing situations containing non-differentiable noise, such as JPEG compression, experimental results indicate that embedding the watermark in the mid-to-low-frequency region yields more robust results.

[0138] Depend on Figure 4As can be seen, the first row is the original image, the second row is the watermarked image, and the third row is the noisy image. By comparing the images, it can be seen that the watermarked image in the second row and the original image in the first row are almost indistinguishable to the naked eye. Even after severe noise interference, the watermark can still be successfully extracted from the noisy image in the third row. Furthermore, based on the signal-to-noise ratio, the watermark using the method of this application has good imperceptibility. According to the noisy image in the third row and the ACC values ​​in Table 3, it can be seen that the method of this application can still successfully extract the watermark after such a large degree of noise interference, indicating that the method of this application has good robustness.

[0139] The method described in this application provides important guidance for image watermarking: when designing digital watermarking algorithms, the most suitable embedding strategy should be selected based on the expected noise environment and characteristics. For differentiable noise, the high-frequency region of the image can be utilized; while for non-differentiable noise, the strategy should shift to utilizing the robustness of the mid-to-low frequency region. This flexible adaptability is crucial to ensuring the effectiveness of digital watermarking in diverse practical applications.

[0140] This application's embodiments employ an end-to-end reversible digital watermarking model. The end-to-end design means the model directly generates an image with the embedded watermark from the input raw image, omitting multiple separation steps in traditional processes and improving efficiency. Simultaneously, reversibility ensures accurate watermark extraction, maintaining watermark integrity even under extreme conditions. This end-to-end reversible model not only simplifies the watermark embedding and extraction process but also improves the model's generalization ability and robustness against noise attacks by learning directly from data. Through end-to-end training, the model can automatically adjust parameters to optimize performance, achieving more efficient and reliable digital watermark protection.

[0141] The method of this application embodiment mainly includes the following four aspects: First, the original image and watermark information are used as input and processed by an image preprocessing module (IPM) and a watermark diffusion module (WDM), respectively. Through IPM, the image is divided into four frequency domains, and an appropriate embedding domain can be selected for embedding based on the actual noise type. Second, the watermark information, the preprocessed image, and the watermark information are processed by a watermark embedding module (WEM), which performs forward and backward propagation. The output of forward propagation is the watermark image, and the output of backward propagation is the restored image and the extracted watermark. Then, the watermark image is processed by a fusion module (FM), which monitors the overall characteristics of the image to ensure that the overall quality and features of the image remain stable after watermark embedding. Finally, the watermark image is processed by a non-differentiable noise processing module (NNPM) to simulate non-differentiable noise, and then passed through a differentiable noise pool to finally obtain a noisy image containing the watermark. The method of this application embodiment can effectively extract watermarks even from images containing non-differentiable noise through NNPM, improving the robustness of the watermark to non-differentiable noise and noise superposition.

[0142] It is understood that before using the technical solutions of the various embodiments in this disclosure, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0143] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations of this disclosed technical solution.

[0144] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0145] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0146] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0147] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] Based on the same inventive concept, corresponding to the single-stage training method of the end-to-end digital watermarking model in any of the above embodiments, this application also provides an image watermark identification method.

[0149] like Figure 5 As shown, the image watermarking method may include:

[0150] S310, acquire the original image, the target watermark information, and the image of the watermark information to be extracted; wherein, the image of the watermark information to be extracted has the same image content as the original image;

[0151] S320, the image from which the watermark information to be extracted is input into the end-to-end digital watermarking model to obtain the predicted watermark information; wherein, the end-to-end digital watermarking model is trained using the single-stage training method of the end-to-end digital watermarking model as described in any of the preceding items.

[0152] S330, determine whether the predicted watermark information is the same as the target watermark information.

[0153] In some embodiments, in step S310, the image from which the watermark information is to be extracted can be a target image that is suspected of having the target watermark information and has the same image content as the original image. This target image can be visually identified as having the same image content as the original image. Alternatively, after visual identification, the pixel mean square error can be calculated, and step S320 is only performed if the pixel mean square error is less than a threshold.

[0154] In some embodiments, S320 can be understood as inputting the image into the end-to-end digital watermarking model in reverse to extract the watermark information and obtain the predicted watermark information.

[0155] In some embodiments, in step S330, determining whether the predicted watermark information is the same as the target watermark information can be done by comparing whether the bit error rate of the predicted watermark information and the target watermark information is less than a preset threshold. Specifically, when the bit error rate of the predicted watermark information and the target watermark information is less than or equal to the preset threshold, it can be determined that the predicted watermark information is the same as the target watermark information; thus, it is determined that the image from which the watermark information is to be extracted has the target watermark information. Conversely, when the bit error rate of the predicted watermark information and the target watermark information is greater than the preset threshold, it can be determined that the predicted watermark information is different from the target watermark information; thus, it is determined that the image from which the watermark information is to be extracted does not have the target watermark information.

[0156] It should be understood that, typically, when an image from which watermark information is to be extracted has the target watermark information, the watermark information in that image is added using an end-to-end digital watermarking model trained by the aforementioned single-stage training method. That is, the end-to-end digital watermarking model trained by the aforementioned single-stage training method can be used alone to add watermark information. The specific process for adding watermark information can be as follows: Figure 3 As shown, it can also be used alone to decode target images with watermark information. Of course, if the corresponding watermark information can be decoded, then the target image with watermark information itself is an end-to-end digital watermarking model trained by the single-stage training method of the end-to-end digital watermarking model to add watermark information.

[0157] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the single-stage training method of the end-to-end digital watermarking model or the image watermark identification method described in any of the above embodiments.

[0158] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device, which may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0159] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0160] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0161] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0162] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0163] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0164] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0165] The electronic devices described above are used to implement the single-stage training method or image watermark identification method of the corresponding end-to-end digital watermarking model in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0166] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the single-stage training method of the end-to-end digital watermarking model or the image watermarking identification method as described in any of the above embodiments.

[0167] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0168] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the single-stage training method of the end-to-end digital watermarking model or the image watermarking identification method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0169] Based on the same inventive concept, corresponding to the single-stage training method for the end-to-end digital watermarking model or the image watermarking identification method described in any of the above embodiments, this disclosure also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to execute the single-stage training method for the end-to-end digital watermarking model or the image watermarking identification method. Corresponding to the execution entity for each step in each embodiment of the single-stage training method for the end-to-end digital watermarking model or the image watermarking identification method, the processor executing the corresponding step can belong to the corresponding execution entity.

[0170] The computer program product of the above embodiments is used to enable the computer and / or the processor to execute the single-stage training method of the end-to-end digital watermarking model or the image watermarking identification method as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0171] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application (including the claims) is limited to these examples; within the framework of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in the details for the sake of brevity.

[0172] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0173] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0174] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A single-stage training method for an end-to-end digital watermarking model, characterized in that, include: Obtain the first image and the watermark information to be embedded; The first image is the original image; The first image and the watermark information to be embedded are input into a convolutional neural network in the forward direction to obtain multiple frequency domains of the first image. Based on the noise type in the actual scene, the target embedding frequency domain of the watermark information in the first image is determined. The watermark information is diffused, and the first image is preprocessed to obtain the second image; The watermark information to be embedded and the second image are input into the first reversible neural network model in a positive direction, and the watermark information to be embedded is embedded into the second image to obtain the third image; The third image is fused to scale a portion of it according to an intensity factor, resulting in a fourth image. The fourth image is fed into the second reversible neural network model in a positive direction, and noise from the actual scene simulated by the second reversible neural network is added to the fourth image to obtain a watermarked image with noise. The noisy watermark image is input in reverse to the second reversible neural network model for non-differentiable noise removal processing. The watermark image after non-differentiable noise removal is subjected to reverse fusion processing. The watermark image after reverse fusion is input into the first reversible neural network model to extract watermark information. The watermark image after watermark information extraction is input in reverse into the convolutional neural network to extract the target embedding frequency domain watermark information, thereby obtaining the extracted watermark information and the restored image; Based on the first difference between the watermark information to be embedded and the extracted watermark information, the second difference between the first image and the noisy watermark image, and the third difference between the first image and the recovered image, the convolutional neural network, the first reversible neural network model, and the second reversible neural network model are trained. Return to the step of inputting the first image and the watermark information to be embedded into the convolutional neural network in the forward direction, and perform iterative training until the first difference, the second difference and the third difference converge, to obtain the application parameters of the trained end-to-end digital watermarking model; wherein, the application parameters include the parameters of the trained convolutional neural network, the parameters of the trained first reversible neural network model and the parameters of the trained second reversible neural network model. The fusion process for the third image includes: The third image is segmented in a first direction to obtain a first sub-image and a second sub-image; the first direction is perpendicular to the plane containing the third image. A portion of the first sub-image is scaled according to an intensity factor to obtain the processed first sub-image; wherein, the intensity factor is a parameter used to control the intensity of the hidden information; The processed first sub-image is then fused with the second sub-image.

2. The single-stage training method for the end-to-end digital watermarking model according to claim 1, characterized in that, The multiple frequency domains of the first image are obtained by performing wavelet transform on the first image; the multiple frequency domains include a first frequency domain, a second frequency domain, a third frequency domain, and a fourth frequency domain, wherein the values ​​of the first frequency domain, the second frequency domain, the third frequency domain, and the fourth frequency domain decrease sequentially; The step of determining the target embedding frequency domain of the watermark information in the first image based on the noise type in the actual scene includes: in response to determining that the noise type includes differentiable noise, determining the first frequency domain as the target frequency domain; or In response to determining that the noise type includes non-differentiable noise, the second and fourth frequency domains are determined as the target frequency domains; or In response to determining that the noise type includes differentiable noise and non-differentiable noise, the fourth frequency domain is determined as the target frequency domain.

3. The single-stage training method for the end-to-end digital watermarking model according to claim 1, characterized in that, The noise types in the actual scenario include differentiable noise and non-differentiable noise; The step of adding noise from a real-world scene simulated using a second reversible neural network to the fourth image to obtain a noisy watermark image includes: The fourth image is subjected to color space conversion, discrete cosine transform processing, quantization processing and encoding to obtain a fifth image with non-differentiable noise; Differentiable noise is added to the fifth image to obtain a watermarked image with mixed noise. The diffusion processing of the watermark information and the preprocessing of the first image include: The first image is subjected to wavelet transform processing to decompose the first image into multiple frequency domains; The watermark information after diffusion processing is embedded in the target embedding frequency domain in the multiple frequency domains.

4. The single-stage training method for the end-to-end digital watermarking model according to claim 3, characterized in that, The color space conversion process includes converting the fourth image from the RGB color space to the YUV color space. The conversion formula includes: Y = 0.299R + 0.587G + 0.114B; where Y is the luminance; R is the value of the red channel, G is the value of the green channel, B is the value of the blue channel; U is the chrominance, V is the chrominance; U = -0.14713R - 0.28886G + 0.436B + 128; V = 0.615R - 0.51499G - 0.10001B + 128; The discrete cosine transform (DCT) process includes: dividing the spatially transformed fourth image into multiple image blocks; performing a discrete cosine transform on each image block to convert the image block's data from the spatial domain to the frequency domain; the discrete cosine transform process is performed using the formula... In progress, among which... F(u,v) For the discrete cosine transform coefficients in the frequency domain, u and v are the frequency domain indices in the DCT transform, respectively; f(x,y) These are the pixel values ​​in the spatial domain. α(u) and α(v) These are the normalization coefficients; The quantization process is performed via formula In progress, among which... F q (u,v) represents the quantized discrete cosine transform coefficients, and Q(u,v) is the quantization step size in the quantization table.

5. The single-stage training method for the end-to-end digital watermarking model according to claim 3, characterized in that, Embedding the watermark information to be embedded into the second image to obtain the third image includes: inputting the watermark information to be embedded and the second image into a first reversible neural network model in a forward direction, and outputting the third image; the first reversible neural network model includes alternating InvertibleConv1x1 layers and InvArch layers; the InvertibleConv1x1 layer realizes information exchange between channels through 1x1 convolution; the InvArch layer includes multiple dense blocks for splitting, extracting and fusing feature maps; The process of performing color space conversion, discrete cosine transform processing, quantization processing, and encoding on the fourth image to obtain a fifth image with non-differentiable noise includes: inputting the fourth image into the second reversible neural network model in the forward direction and outputting the fifth image.

6. The single-stage training method for the end-to-end digital watermarking model according to claim 3, characterized in that, The first difference includes the bit error rate between the watermark information to be embedded and the extracted watermark information; The second difference includes the pixel mean square error between the first image and the noisy watermark image; The third difference includes the pixel mean square error between the first image and the restored image; The loss functions for training include: ;in, , and These are the weighting coefficients; The image loss function; The loss function for watermark information; This is the image reconstruction loss function.

7. A method for identifying image watermarks, characterized in that, include: Acquire the original image, the target watermark information, and the image containing the watermark information to be extracted; wherein the image containing the watermark information to be extracted has the same image content as the original image; The image from which the watermark information is to be extracted is input into an end-to-end digital watermarking model to obtain the predicted watermark information; wherein, the end-to-end digital watermarking model is trained using the single-stage training method of the end-to-end digital watermarking model as described in any one of claims 1-6. Determine whether the predicted watermark information is the same as the target watermark information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as claimed in any one of claims 1 to 6 or as claimed in claim 7.

9. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform any one of claims 1 to 6 or the method as described in claim 7.

Images