Watermark extraction method and device, model training method and device, electronic equipment and medium

Abstract

The application relates to the field of image processing, in particular to a watermark extraction method and model training method, device, electronic equipment and medium, the method comprising the following steps: acquiring a target image to be processed, a watermark identifier being embedded in the target image; then performing color feature enhancement and edge feature enhancement on the target image through a generator in a target generative adversarial network to obtain an enhanced target image; and then performing watermark extraction on the enhanced target image to obtain the watermark identifier. The scheme can more accurately extract the watermark identifier in the image.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to watermark extraction methods, model training methods, apparatus, electronic devices and media. Background Technology

[0002] In some scenarios, to facilitate tracing the source of information leaks, watermarks need to be added to the screen windows of electronic devices so that the watermarks are present in the captured image when the content displayed on the screen window is captured by screenshot or screen capture.

[0003] When adding watermarks, setting higher transparency can reduce the obstruction of screen content. However, this also results in lower accuracy in extracting watermarks with higher transparency from the image. Furthermore, when obtaining an image with a watermark by photographing the screen, the watermark may be unclear due to moiré distortion and / or uneven lighting, which will also lead to inaccurate watermark extraction.

[0004] Therefore, how to more accurately extract watermarks from images is a problem that urgently needs to be solved. Summary of the Invention

[0005] To more accurately extract watermarks from images, this application provides a watermark extraction method, a model training method, an apparatus, an electronic device, and a medium.

[0006] Firstly, this application provides a watermark extraction method, which adopts the following technical solution:

[0007] A watermark extraction method, comprising:

[0008] Obtain the target image to be processed, wherein the target image contains an embedded watermark;

[0009] The target image is enhanced in color and edge features by the generator in the target generative adversarial network to obtain the enhanced target image.

[0010] The watermark is extracted from the enhanced target image to obtain the watermark identifier.

[0011] By adopting the above technical solution, the color and edge features of the enhanced target image obtained by the generator of the generative adversarial network are enhanced, which makes the color representation of the enhanced target image more comprehensive and realistic and the edge representation clearer. Therefore, when extracting watermarks from the enhanced target image, the watermark identifier in the target image can be obtained more accurately.

[0012] In one possible implementation, the target image is subjected to color feature enhancement and edge feature enhancement by a generator in a target generative adversarial network, including any of the following:

[0013] The target image is sequentially passed through N1 generators to enhance color features and edge features, resulting in an enhanced target image, where N1≥2;

[0014] The target image is enhanced in terms of color features and edge features by N2 generators to obtain corresponding enhanced images. The enhanced target image is then determined based on the corresponding enhanced images, where N2≥2.

[0015] By adopting the above technical solution, the target image is passed through N1 generators in sequence, which enables the target image to undergo multi-level color feature enhancement and edge feature enhancement, thereby making the color feature expression and edge feature expression of the target image after enhancement better; and the target image is passed through N2 generators for color feature enhancement and edge feature enhancement respectively, so that multiple target images can be enhanced at the same time, which makes it easier to obtain multiple enhanced target images at the same time, thus making the efficiency higher.

[0016] In one possible implementation, the target generative adversarial network is obtained by training an original generative adversarial network, which includes an original generator and an original discriminator.

[0017] The step of enhancing the color and edge features of the target image through a generator in a target generative adversarial network, prior to which the following is also included:

[0018] Obtain training samples, which include multiple training sample pairs. Each training sample pair consists of a first image and a second image. The first image and the second image have the same watermark, and the transparency of the watermark in the first image is higher than that in the second image.

[0019] Determine a first loss and a second loss. The first loss is the loss after color space conversion between the generated image of each first image and the corresponding second image after passing through the original generator. The second loss is the loss caused by the generated image of each first image passing through the original generator passing through the original discriminator.

[0020] A first loss function is constructed based on the first loss and the second loss;

[0021] The original generative adversarial network is trained based on the first loss function to obtain the target generative adversarial network.

[0022] In one possible implementation, a third loss is determined, which is the loss incurred by each of the second images after passing through the original discriminator;

[0023] A second loss function is constructed based on the second loss and the third loss;

[0024] The step of training the original generative adversarial network based on the first loss function to obtain the target generative adversarial network includes:

[0025] The original generative adversarial network is trained based on the first loss function and the second loss function to obtain the target generative adversarial network.

[0026] In one possible implementation, a watermark extraction method also includes:

[0027] The first loss function is shown in Formula A.

[0028] Formula A:L G =α(-E(D(G(z))))+(1-α)MSE(f(G(z)),f(x))*d1;

[0029] in, Where G represents the original generator, D represents the original discriminator, E represents the expectation, α is a preset coefficient, z represents the first image, and x represents the second image; G(z) is the generated image obtained after the first image is passed through the original generator, D(G(z)) is the realism score of the generated image relative to the second image, f(G(z)) is the conversion of the generated image to the target color space and extraction of one or more channels, f(x) is the conversion of the second image to the target color space and extraction of one or more channels; MSE(f(G(z)), f(x)) is the loss of converting the generated image and the second image to the same target color space, M and N are the width and height of the generated image or the second image, i and j are the x-coordinate and y-coordinate values ​​of the generated image or the second image, n is the number of channels in the target color space, and c is the channel number; d1 = 2*alpha / (a2-a1), where alpha represents the transparency of the watermark in the second image, a1 represents the minimum value of transparency, and a2 represents the maximum value of transparency;

[0030] The second loss function is shown in Formula B, Formula B: L D =E(max(0,1-D(x)))*d1+E(max(0,1+D(G(z)))), where, L D Let D(x) represent the second loss function, where D(x) is the authenticity score of the second image obtained by passing the second image through the original discriminator relative to the second image itself.

[0031] In one possible implementation, the original generative adversarial network comprises: an original discriminator and N1 cascaded original generators;

[0032] The target generative adversarial network is obtained by training the original generative adversarial network, including:

[0033] Iterate through the following steps until the stopping condition is met to obtain the target generative adversarial network:

[0034] Multiple second images are passed sequentially through N1 cascaded original generators to obtain their respective generation results;

[0035] The corresponding generated results and their corresponding first images are then processed by the original discriminator.

[0036] Secondly, this application discloses a model training method.

[0037] A model training method, comprising:

[0038] Obtain the original generative adversarial network;

[0039] The original generative adversarial network is trained to obtain a target generative adversarial network, which is used to enhance the color features and edge features of the target image before watermark extraction.

[0040] By adopting the above technical solution, the color and edge features of the enhanced target image obtained by the generator of the generative adversarial network are enhanced, which makes the color representation of the enhanced target image more comprehensive and realistic and the edge representation clearer. Therefore, when extracting watermarks from the enhanced target image, the watermark identifier in the target image can be obtained more accurately.

[0041] In one possible implementation, training the original generative adversarial network to obtain the target generative adversarial network includes:

[0042] Obtain training samples, which include multiple training sample pairs. Each training sample pair consists of a first image and a second image, wherein the transparency of the watermark in the first image is higher than the transparency of the watermark in the second image.

[0043] Obtain a first loss and a second loss. The first loss is the loss after color space conversion between the generated image of the first image and the second image after passing through the original generator. The second loss is the loss caused by the original discriminator for each generated image of the first image after passing through the original generator.

[0044] A first loss function is constructed based on the first loss and the second loss;

[0045] The original generative adversarial network is trained based on the first loss function to obtain the target generative adversarial network.

[0046] Thirdly, this application provides a watermark extraction device, which adopts the following technical solution:

[0047] A watermark extraction device, comprising:

[0048] The target image acquisition module is used to acquire the target image to be processed, wherein the target image has an embedded watermark identifier;

[0049] The enhancement processing module is used to enhance the color features and edge features of the target image through the generator in the target generative adversarial network to obtain the enhanced target image;

[0050] The watermark extraction module is used to extract the watermark from the enhanced target image to obtain the watermark identifier.

[0051] By employing the above technical solution, the device can extract watermarks from the enhanced target image to obtain the watermark identifier. Furthermore, the color and edge features of the enhanced target image obtained through the generator of the generative adversarial network are enhanced, resulting in a more comprehensive and realistic color representation and clearer edge representation. Therefore, when extracting watermarks from the enhanced target image, the watermark identifier can be obtained more accurately.

[0052] In one possible implementation, when the enhancement processing module enhances the target image for color and edge features through the generator in the target generative adversarial network, it specifically performs any of the following:

[0053] The target image is sequentially passed through N1 generators to enhance color features and edge features, resulting in an enhanced target image, where N1≥2;

[0054] The target image is enhanced in terms of color features and edge features by N2 generators to obtain corresponding enhanced images. The enhanced target image is then determined based on the corresponding enhanced images, where N2≥2.

[0055] In one possible implementation, a watermark extraction device further includes:

[0056] The training sample acquisition module is used to acquire training samples, which include multiple training sample pairs. Each training sample pair consists of a first image and a second image. The first image and the second image have the same watermark, and the transparency of the watermark in the first image is higher than that of the watermark in the second image.

[0057] A first determining module is used to determine a first loss and a second loss. The first loss is the loss of each first image after passing through the original generator and the corresponding first image after color space conversion. The second loss is the loss of each first image after passing through the original generator and the original discriminator.

[0058] The first loss function construction module is used to construct a first loss function based on the first loss and the second loss;

[0059] The training module is used to train the original generative adversarial network based on the first loss function to obtain the target generative adversarial network.

[0060] In one possible implementation, a watermark extraction device further includes:

[0061] The second determining module is used to determine the third loss, which is the loss incurred by each second image after passing through the original discriminator;

[0062] The second loss function construction module is used to construct a second loss function based on the second loss and the third loss.

[0063] In one possible implementation, when the training module trains the original generative adversarial network based on the first loss function to obtain the target generative adversarial network, it is specifically used for:

[0064] The original generative adversarial network is trained based on the first loss function and the second loss function to obtain the target generative adversarial network.

[0065] In one possible implementation, the training module trains the original generative adversarial network to obtain the target generative adversarial network, specifically for:

[0066] Iterate through the following steps until the stopping condition is met to obtain the target generative adversarial network:

[0067] Multiple second images are passed sequentially through N1 cascaded original generators to obtain their respective generation results;

[0068] The corresponding generated results and their corresponding first images are then processed by the original discriminator.

[0069] Fourthly, this application provides a model training device.

[0070] A model training device, comprising:

[0071] The original generative adversarial network acquisition module is used to acquire the original generative adversarial network;

[0072] The training module is used to train the original generative adversarial network to obtain a target generative adversarial network. The target generative adversarial network is used to enhance the color features and edge features of the target image to be processed before watermark extraction.

[0073] In one possible implementation, when the training module trains the original generative adversarial network to obtain the target generative adversarial network, it is specifically used for:

[0074] Obtain training samples, which include multiple training sample pairs. Each training sample pair consists of a first image and a second image. The transparency of the watermark in the first image is higher than that of the watermark in the second image. The watermarks in the first image and the second image are the same.

[0075] Obtain a first loss and a second loss. The first loss is the loss of at least one channel between the generated image after the first image passes through the original generator and the first image after color space conversion. The second loss is the loss caused by the generated image after the original generator passes through the original discriminator.

[0076] A first loss function is constructed based on the first loss and the second loss;

[0077] The original generative adversarial network is trained based on the first loss function to obtain the target generative adversarial network.

[0078] Fifthly, this application provides an electronic device that adopts the following technical solution:

[0079] An electronic device comprising:

[0080] At least one processor;

[0081] Memory;

[0082] At least one application, wherein the at least one application is stored in memory and configured to be executed by at least one processor, the at least one application being configured to: execute the watermark extraction method and model training method described above.

[0083] Sixthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0084] A computer-readable storage medium includes: a computer program stored thereon that can be loaded by a processor and execute the watermark extraction method and the model training method described above.

[0085] In summary, this application includes at least one of the following beneficial technical effects:

[0086] 1. The color and edge features of the target image are enhanced by the generator of the generative adversarial network, which makes the color representation of the target image more comprehensive and realistic and the edge representation clearer. Therefore, when extracting watermarks from the enhanced target image, the watermark identifier in the target image can be obtained more accurately. Attached Figure Description

[0087] Figure 1 This is a flowchart illustrating the watermark extraction method in the embodiments of this application;

[0088] Figure 2 This is a schematic diagram of the N1 generators cascaded in the target generative adversarial network in an embodiment of this application;

[0089] Figure 3 This is a schematic diagram of the N2 generators of the target generative adversarial network connected in parallel in an embodiment of this application;

[0090] Figure 4 This is a schematic diagram of the generator of the Unet structural model in the embodiments of this application;

[0091] Figure 5 This is a schematic diagram of the structure of the transformer block in the embodiments of this application;

[0092] Figure 6 This is a flowchart illustrating the model training method in an embodiment of this application;

[0093] Figure 7 This is a schematic diagram of the watermark extraction device in the embodiments of this application;

[0094] Figure 8 This is a schematic diagram of the structure of the model training device in the embodiments of this application;

[0095] Figure 9 This is a schematic diagram of the structure of the electronic device in the embodiments of this application. Detailed Implementation

[0096] The following combination Figures 1-9 This application will be described in further detail.

[0097] After reading this specification, those skilled in the art may make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they fall within the scope of the claims of this application.

[0098] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0099] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article, unless otherwise specified, generally indicates that the preceding and following related objects have an "or" relationship.

[0100] This application provides a watermark extraction method, executed by an electronic device, as described above. Figure 1 The method includes steps S11-S13, wherein:

[0101] Step S11: Obtain the target image to be processed, which contains a watermark.

[0102] In the embodiments of this application, the target image to be processed can be an image obtained by taking a screenshot of a screen with an embedded watermark, or an image obtained by taking a picture of a screen displaying a watermark. Furthermore, if a target video is obtained by screen recording of a screen with an embedded watermark or by taking a picture of a screen displaying a watermark, the target image can also be a frame image extracted from the target video.

[0103] Step S12: The target image is enhanced in terms of color features and edge features by the generator in the target generative adversarial network to obtain the enhanced target image.

[0104] In the embodiments of this application, the target generative adversarial network is a trained network model used to enhance the target image. The color feature enhancement of the target image by the generator in the generative adversarial network can make the color representation of the watermark mark that is difficult to identify due to high transparency in the target image more comprehensive and realistic. At the same time, the edge feature enhancement of the target network by the generator can make the edge representation of the watermark mark that is unclear due to high transparency and / or unclear due to the shooting screen clearer.

[0105] Compared to the target image input to the generator, the target image after generator enhancement has more comprehensive and realistic color feature representation, while the edge representation is clearer.

[0106] Step S13: Extract the watermark from the enhanced target image to obtain the watermark identifier.

[0107] In this embodiment, watermark extraction of the enhanced target image can be performed using a trained watermark extraction model or by a preset watermark extraction algorithm and steps. This embodiment uses a trained watermark extraction model as an example to illustrate the effect of this embodiment, and is not intended to limit the application scenario or specific implementation of this embodiment.

[0108] Inputting an image into a trained watermark extraction model can extract the watermark identifier from the image. However, if the watermark identifier in the image has high transparency and / or is unclear, meaning the color and edge features of the watermark identifier in the target image are unclear, the extracted watermark identifier will differ significantly from the standard identifier corresponding to the watermark identifier in the image. By using a generative adversarial network (GAN) generator, the color and edge features of the target image are enhanced, resulting in a more comprehensive and realistic color representation and clearer edge representation. Therefore, watermark extraction from the enhanced target image can more accurately obtain the watermark identifier.

[0109] Furthermore, in this embodiment of the application, since the adversarial network needs to enhance the color features of the target image, in order to better reflect the expression of the color features in the target image, before the target image is enhanced for color features and edge features by the generator in the target generative adversarial network, it is necessary to convert the target image to the target color space to obtain the color-converted target image, and then enhance the color features and edge features of the color-converted target image by the generator in the generative adversarial network.

[0110] The target color space is any one of hsv, YCrCb, hls, lub, and xyz; a color space is also called a color model (also known as a color space or color system). The purpose of a color space is to describe color in a generally acceptable way under certain standards.

[0111] Furthermore, the target image is enhanced in terms of color and edge features by the generator in a target generative adversarial network, specifically including either step A or step B. Wherein:

[0112] Step A: The target image is sequentially passed through N1 generators to enhance color features and edge features, resulting in an enhanced target image, where N1 ≥ 2;

[0113] Step B: Enhance the target image for color features and edge features using N2 generators respectively, to obtain the enhanced image corresponding to each generator, and determine the enhanced target image based on the corresponding enhanced image, where N2≥2.

[0114] Specifically, for step A, refer to Figure 2 N1 generators are cascaded in sequence. The target image is input from the first-level generator and enhanced. The first-level generator outputs the target image after the first enhancement, which includes color feature enhancement and edge feature enhancement. After that, the input image of each level generator is the output image of the previous level generator. The output image of the last level generator is used as the enhanced target image.

[0115] When the enhanced images output by each generator are used to obtain the enhanced target image, the target image is enhanced multiple times by cascading multiple generators, which enhances the color features and contour features multiple times.

[0116] Specifically, for step B, refer to Figure 3 N2 generators are connected in parallel. The target image serves as the input to each generator, resulting in a corresponding output image for each generator. That is, each generator obtains an enhanced output image. The enhanced target image is then determined based on the output image corresponding to each generator. For example, the output image with the highest clarity can be selected as the enhanced target image, or each output image can be selected as the enhanced target image. This embodiment does not impose specific limitations on this approach.

[0117] In other words, when a target image is processed by a generator in a target generative adversarial network (PGAN) to enhance its color and edge features, resulting in an enhanced target image, the resulting enhanced target image may be one or multiple images. When applying a PGAN, the user inputs the image to be enhanced. The user can choose to have the input image processed through multiple cascaded generators to obtain a single enhanced target image, or they can choose to have the input image processed through multiple parallel generators, each producing an enhanced target image, thus obtaining multiple enhanced target images.

[0118] Specifically, the structure of each generator in steps A and B is identical. The generator adopts the Unet architecture model and uses the ReLU activation function. For example... Figure 4As shown, the shrinking path of the generator includes a series of cascaded downsampling stages, and the expanding path of the generator includes a series of cascaded upsampling stages.

[0119] Furthermore, each downsampling stage includes at least two convolutional layers and one pooling layer. The target image is input from the first downsampling stage. The target image first passes through the two convolutional layers in the first downsampling stage to obtain the feature map of the first downsampling stage. This feature map then passes through the pooling layer to obtain the first-level pooled feature map, at which point the first downsampling stage ends. Afterward, the input to each downsampling stage is the pooled feature map output from the previous downsampling stage.

[0120] In the shrinking path, the resolution of the feature map obtained in each downsampling stage is lower than that of the feature map obtained in the previous downsampling stage. In other words, the size of the feature map decreases sequentially after multiple downsampling stages in the shrinking path stage. Furthermore, the number of channels for the target features in the feature map increases with each downsampling stage. This allows for the acquisition of image information at different scales and the extraction of high-level semantic information during the shrinking path stage. The target features include at least color features and edge features.

[0121] The expansion path stage includes multiple cascaded upsampling stages, each consisting of at least two deconvolutional layers and one pixel fusion layer; the pixel fusion layer can also be a pixel interpolation layer. The number of downsampling stages equals the number of upsampling stages. Each downsampling stage has a corresponding upsampling stage. Since the expansion path is the reverse of the contraction path, the first upsampling stage corresponds to the last downsampling stage, and vice versa. For example, if the downsampling stages are ordered A1, A2, A3, A4, and the upsampling stages are ordered B1, B2, B3, B4, then A1 corresponds to B4, A2 to B3, A3 to B2, and A4 to B1.

[0122] Furthermore, the generator also includes multiple encoding modules, which can be transformer blocks or other encoding modules. The pooled feature map output from the final downsampling stage is passed through two more convolutional layers to obtain the feature map, which serves as the input to the first upsampling stage. The feature map input in the first upsampling stage is then passed through two deconvolutional layers to obtain the deconvolutional feature map. The encoded feature map obtained from the convolution of the downsampling stage corresponding to the first upsampling stage is also passed through the encoding module and serves as the input to the first upsampling stage. The encoded feature map is fused with the deconvolutional feature map to obtain the fused feature map output from the upsampling stage. Subsequently, the input to each upsampling stage is the fused feature map output from the previous upsampling stage and the encoded feature map obtained from the convolution of the downsampling stage corresponding to each upsampling stage, passed through the encoding module.

[0123] Specifically, the structure of the encoding module is as follows: Figure 5 As shown, the encoding module includes a patch_embedding layer, a positional encoding layer, a self-attention layer, and a feedforward network layer that process the input feature map in sequence. Specifically, the patch_embedding layer extracts edge features from the feature map, obtaining multiple vectors representing the global edge features of the feature map, and the positional encoding layer encodes the vectors based on their position within the global features.

[0124] Furthermore, the self-attention layer performs multi-head self-attention computation on each vector with positional encoding. This involves assigning a corresponding coefficient to each vector, multiplying each vector by its assigned coefficient to obtain a first matrix, and then obtaining the first matrix corresponding to each vector. The vectors are then inserted into their respective first matrices to obtain a second matrix. Further, the second matrix corresponding to each vector is input into a feedforward network, and the output of the feedforward network and the second matrix corresponding to each vector are used as the final output result for each vector. The final output results of each vector are concatenated to obtain the encoded feature map after encoding the input feature map. This encoding module produces an encoded feature map that better represents the edge features of the image and how these edge features are expressed within the global features.

[0125] During the path expansion stage, the resolution of the fused feature map increases with each upsampling stage. Furthermore, since each upsampling stage fuses corresponding feature information from two feature maps, the number of channels decreases. In other words, the size of the fused feature map increases sequentially with each upsampling stage during the path expansion stage; and the number of target feature channels decreases with each downsampling stage. This allows for the fusion of image information at different scales during the path expansion stage, resulting in more refined target features, which include at least color and edge features. The fused feature map output from the final upsampling stage is the generator's output image.

[0126] Furthermore, the target generative adversarial network (GAN) is obtained by training an original GAN, which includes an original generator and an original discriminator. Before performing color and edge feature enhancement on the target image through the generator in the target GAN, a training step on the original GAN ​​is also included.

[0127] The training steps for the original generative adversarial network include: acquiring training samples, determining the first loss and the second loss, constructing a first loss function based on the first loss and the second loss, training the original generative adversarial network based on the first loss function, and stopping training when the stopping condition is met, thereby obtaining the target generative adversarial network.

[0128] Furthermore, the original generative adversarial network is trained based on the first loss function to obtain the target generative adversarial network, including: training the original generative adversarial network based on the first loss function and the second loss function to obtain the target generative adversarial network. The first loss function is used to train the original generator, and the second loss function is used to train the original discriminator.

[0129] Specifically, the training samples consist of multiple training sample pairs. Each training sample pair is composed of a first image and a second image. The watermark is the same in both the first and second images, but the transparency of the watermark in the first image is higher than that in the second image. The transparency of the watermark in the second image is represented by alpha, and the value of alpha ranges from [a1, a2]. The lower the transparency of the watermark, the clearer the watermark is.

[0130] Specifically, the image obtained by the first image through the original generator and the second image are both converted to the same target color space. The first loss is the loss between the generated image of the first image after passing through the original generator and the corresponding second image after color space conversion. The target color space can be any one of hsv, YCrCb, hls, lub, and xyz. In this embodiment, if the determined first loss is the loss of at least two channels of the generated image of the first image after passing through the original generator and the corresponding second image after color space conversion, then the first loss is the average value of the losses of the generated image of the first image after passing through the original generator and the corresponding second image after color space conversion across all channels.

[0131] For example, the first image, the generated image from the original generator, and the second image are both in the RGB color space. The first image, the generated image from the original generator, and the second image are both converted to the HSV color space. That is, RH, GS, and BV are each converted to one color channel. The average loss of the two images in these three color channels during the image conversion process is determined. The average loss is the loss of the first image, the generated image from the original generator, and the second image in the process of converting to the target color space, which is also the first loss.

[0132] Specifically, the second loss is the loss incurred by the generated image of each first image passing through the original generator and then passing through the original discriminator. Any data is input into the original discriminator as fake data, and simultaneously, a real data point is input. The original discriminator outputs the probability that any data point is similar to the real data. The second loss represents the original discriminator's score for the realism of the generated image of each first image relative to the second image, thus representing the original discriminator's ability to identify fake data. Furthermore, the second loss also represents the accuracy of the original generator's image enhancement processing.

[0133] Specifically, the first loss is determined as The second loss is determined to be D(G(z)), and the formula A:L for the first loss function is constructed based on the first and second losses. G =α(-E(D(G(z))))+(1-α)MSE(f(G(z)),f(x))*d1; where, Where G represents the original generator, D represents the original discriminator, E represents the expectation, α is a preset coefficient, z represents the first image, and x represents the second image; G(z) is the generated image obtained after the first image is passed through the original generator, D(G(z)) is the realism score of the generated image relative to the second image, f(G(z)) is the conversion of the generated image to the target color space and extraction of one or more channels; MSE(f(G(z)), f(x)) is the loss of converting the generated image and the second image to the same target color space, M and N are the width and height of the generated image or the second image, i and j are the x-coordinate and y-coordinate values ​​of the generated image or the second image, n is the number of channels in the target color space, and c is the channel number; d1 = 2*alpha / (a2-a1), where alpha represents the transparency of the watermark in the second image, a1 represents the minimum value of transparency, and a2 represents the maximum value of transparency.

[0134] Specifically, the second loss function is the discriminator's loss function. Determining the second loss function includes: determining the third loss, and constructing the second loss function based on the second and third losses. The third loss is the loss incurred by each second image after passing through the original discriminator. Since the transparency of the watermark in the second image is lower than that in the first image in each training sample pair, meaning the clarity of the watermark in the second image is greater than that in the first image, the second image is always used as the real data in each training sample pair when training the original discriminator. When the second image is input into the original discriminator, the output of the original discriminator is the authenticity score of the second image relative to the second image itself. In other words, the third loss characterizes the original discriminator's ability to recognize real data.

[0135] The third loss is defined as D(x), where D(x) is the realism score of the second image obtained by the original discriminator relative to the second image itself. Formula B for the second loss function is constructed based on the second and third losses: L D =E(max(0,1-D(x)))*d1+E(max(0,1+D(G(z)))), and train the original discriminator based on the second loss function.

[0136] The original discriminator is used to determine the authenticity score of any input data relative to real data. Specifically, the original discriminator determines the authenticity score of any data relative to real data by: splitting the real data and any data according to the same scale, resulting in several real data sub-sub-data for each real data sub-sub-data, and several fake data sub-sub-data for each individual data sub-sub-data. The similarity score between each fake data sub-sub-data and its corresponding real data sub-data is calculated. Based on the similarity scores of each fake data sub-data and its corresponding real data, the authenticity score of any data relative to real data is determined. In this embodiment, when the original discriminator determines the authenticity score of any input image relative to the second image, it segments the input image into P fake sub-images according to a preset segmentation logic, and segments the second image into P real sub-images according to the same preset segmentation logic. Based on the position of each fake sub-image in any input image and the position of each real sub-image in the second image, a one-to-one correspondence is established between the fake and real sub-images. The original discriminator determines the similarity of each fake sub-image to the corresponding real sub-image, and based on the similarity of each fake sub-image to the corresponding real sub-image, obtains the similarity of any input image to the second image, where P is a positive integer greater than or equal to 2. Further, P can be determined based on the length and width of the second image or any input image.

[0137] Furthermore, when determining the similarity between a fake sub-image and a corresponding real sub-image, the original discriminator can use any image similarity algorithm that can be used for training, without any specific limitations.

[0138] Training the original discriminator's ability to recognize real data: The second image is used as real data in each training sample pair for training the original discriminator. The second image in each training sample is input into the original discriminator as real data, and simultaneously, the second image is input into the original discriminator as fake data, to train the original discriminator's ability to recognize real data. That is, the third loss characterizes the original discriminator's ability to recognize real data.

[0139] Training the original discriminator to recognize real data does not require the participation of the original generator, while training the original discriminator to recognize fake data does require the participation of the original generator. That is, the first image needs to be used as the generated image obtained by the original generator as fake data, and the second image in the training sample pair corresponding to the first image is used as real data to train the original discriminator.

[0140] Furthermore, referring to Figure 2The original generative adversarial network (GAN) includes an original discriminator and N1 cascaded original generators. The original GAN ​​is trained to obtain the target GAN. Training the original generators includes iterating through steps 1-2 below. Each time step 1 and step 2 are executed completely, a stopping condition is checked until the stopping condition is met, at which point the iteration loop of steps 1 and 2 is stopped, resulting in the target GAN. The stopping condition can be that the iteration reaches a preset number of times, or that the value of the first loss function LG is within a preset first defined interval and the value of the second loss function LD is within a preset second defined interval. The specific ranges of the first and second defined intervals are not specifically limited in this embodiment.

[0141] Step 1: Pass multiple first images through N1 cascaded original generators in sequence to obtain their respective generation results;

[0142] Step 2 involves processing the generated results and their corresponding second images using the original discriminator.

[0143] Specifically, such as Figure 2 As shown, N1 original generators are cascaded sequentially. The first image is input from the first-level original generator, and the input to subsequent original generators is the output of the previous-level original generator. In step 2, the corresponding generated results and their corresponding second images are input to the original discriminator for discrimination, that is, to determine the realism score of each original generator's generated image relative to the second image. The realism score of any original generator's generated image relative to the second image includes: the similarity between each P-fake sub-image segmented from the generated image of any original generator and the corresponding real sub-image segmented from the second image. An iterative step is then performed to train the original generators and the original discriminator again.

[0144] The iterative steps include: determining each fake sub-image whose similarity to the corresponding real sub-images segmented from the second image is greater than or equal to a preset similarity threshold as the image to be synthesized; determining at least one synthesized image based on each image to be synthesized and its position in the corresponding generated image; and repeating the iterative loop of steps 1 to 2 with the determined at least one synthesized image.

[0145] In one possible implementation, refer to Figure 3The original generative adversarial network (GAN) may further include: an original discriminator and N2 parallel original generators. Training the original GAN ​​to obtain the target GAN includes: iterating through steps 3-4 below until a stopping condition is met, thereby obtaining the target GAN. The stopping condition can be that the iteration reaches a preset number of times, or that the value of the first loss function LG is within a preset first defined interval and the value of the second loss function LD is within a preset second defined interval. The specific ranges of the first and second defined intervals are not specifically limited in this embodiment.

[0146] Step 3: Pass each of the multiple first images through N2 parallel generators to obtain their respective generation results;

[0147] Step 4: The corresponding generated results and their corresponding second images are processed by the original discriminator.

[0148] Specifically, such as Figure 3 As shown, N2 original generators are connected in parallel, with each original generator taking one first image as input. In step 4, the corresponding generated result and the corresponding second image are input into the original discriminator for discrimination, that is, to determine the authenticity score of each generated result relative to the second image. Iterative steps are then performed to train the original generators and the original discriminator again.

[0149] Training the original discriminator's ability to identify fake data: In actual training, regardless of whether the original generative adversarial network uses a structure of N1 original generators cascaded sequentially or N2 original generators connected in series, for the first image in any training sample, the first image is processed by N1 cascaded original generators to obtain the corresponding generated images of each original generator, or the first image is processed by N2 original generators to obtain the corresponding generated images of each original generator. The generated image corresponding to each original generator is used as fake data and input into the original discriminator to train the original discriminator.

[0150] The above embodiments disclose a watermark extraction method, and the following embodiments disclose a model training method.

[0151] A model training method, performed by an electronic device, refers to... Figure 6 The method includes steps S61 and S62, wherein:

[0152] Step S61: Obtain the original generative adversarial network;

[0153] Step S62: Train the original generative adversarial network to obtain the target generative adversarial network. The target generative adversarial network is used to enhance the color features and edge features of the target image before watermark extraction.

[0154] Furthermore, the original generative adversarial network is trained to obtain the target generative adversarial network. Specifically, this may include acquiring training samples, acquiring a first loss and a second loss, and constructing a first loss function based on the first loss and the second loss; the original generative adversarial network is then trained based on the first loss function to obtain the target generative adversarial network.

[0155] Specifically, the training samples consist of multiple training sample pairs. Each training sample pair comprises a first image and a second image. The transparency of the watermark in the first image is higher than that in the second image, and the watermarks in the first and second images are identical. The first loss is the loss after color space conversion between the generated image (after passing through the original generator) and the first image. The second loss is the loss incurred by the generated image (after passing through the original generator) passing through the original discriminator. For specific training methods, please refer to the training steps of the original generative adversarial network in the above embodiment of the watermark extraction method.

[0156] The above embodiments introduce a watermark extraction method and a model training method from the perspective of method flow. The following embodiments introduce a watermark extraction device and a model training device from the perspective of virtual modules or virtual units. For details, please refer to the following embodiments.

[0157] This application provides a watermark extraction device, such as... Figure 7 As shown, the watermark extraction device may specifically include: a target image acquisition module 71, used to acquire a target image to be processed, wherein a watermark identifier is embedded in the target image;

[0158] Enhancement processing module 72 is used to enhance the color features and edge features of the target image through the generator in the target generative adversarial network to obtain the enhanced target image;

[0159] The watermark extraction module 73 is used to extract watermarks from the enhanced target image to obtain watermark identifiers.

[0160] In one possible implementation, when the enhancement processing module 72 performs color feature enhancement and edge feature enhancement on the target image through the generator in the target generative adversarial network, it specifically performs any of the following:

[0161] The target image is sequentially passed through N1 generators to enhance color features and edge features, resulting in an enhanced target image, where N1≥2;

[0162] The target image is enhanced in terms of color features and edge features by N2 generators to obtain corresponding enhanced images. The enhanced target image is then determined based on the corresponding enhanced images, where N2≥2.

[0163] In one possible implementation, a watermark extraction device further includes:

[0164] The training sample acquisition module is used to acquire training samples. The training samples contain multiple training sample pairs. Each training sample pair consists of a first image and a second image. The watermarks in the first image and the second image are the same. The transparency of the watermark in the first image is higher than that of the watermark in the second image.

[0165] The first determining module is used to determine a first loss and a second loss. The first loss is the loss of at least one channel of the generated image after the first image passes through the original generator and the corresponding second image after color space conversion. The second loss is the loss brought by the generated image after the first image passes through the original generator and the original discriminator.

[0166] The first loss function construction module is used to construct a first loss function based on the first loss and the second loss.

[0167] The training module is used to train the original generative adversarial network based on the first loss function to obtain the target generative adversarial network.

[0168] In one possible implementation, a watermark extraction device further includes:

[0169] The second determining module is used to determine the third loss, which is the loss incurred by each second image after passing through the original discriminator;

[0170] The second loss function construction module is used to construct a second loss function based on the second loss and the third loss.

[0171] In one possible implementation, the training module trains the original generative adversarial network based on the first loss function, and when the target generative adversarial network is obtained, it is specifically used for:

[0172] The original generative adversarial network is trained based on the first loss function and the second loss function to obtain the target generative adversarial network.

[0173] In one possible implementation, the training module trains the original generative adversarial network to obtain the target generative adversarial network, specifically used for:

[0174] Iterate through the following steps until the stopping condition is met to obtain the target generative adversarial network:

[0175] Multiple second images are passed sequentially through N1 cascaded original generators to obtain their respective generation results;

[0176] The corresponding generated results and their corresponding first images are then processed by the original discriminator.

[0177] This application provides a model training device, such as... Figure 8 As shown, a model training device includes:

[0178] Original Generative Adversarial Network Acquisition Module 81 is used to acquire the original generative adversarial network;

[0179] Training module 82 is used to train the original generative adversarial network to obtain the target generative adversarial network. The target generative adversarial network is used to enhance the color features and edge features of the target image before watermark extraction.

[0180] In one possible implementation, when training the original generative adversarial network to obtain the target generative adversarial network, the training module 82 is specifically used for:

[0181] Obtain training samples, which contain multiple training sample pairs. Each training sample pair consists of a first image and a second image. The transparency of the watermark in the first image is higher than that of the watermark in the second image. The watermarks in the first image and the second image are the same.

[0182] Obtain a first loss and a second loss. The first loss is the loss between the generated image after the first image passes through the original generator and the first image after color space conversion. The second loss is the loss caused by the generated image after the original generator passes through the original discriminator.

[0183] Construct a first loss function based on the first loss and the second loss;

[0184] The original generative adversarial network is trained based on the first loss function to obtain the target generative adversarial network.

[0185] This application provides an electronic device, such as... Figure 9 As shown, Figure 9 The illustrated electronic device 900 includes a processor 901 and a memory 903. The processor 901 and the memory 903 are connected, for example, via a bus 902. Optionally, the electronic device 900 may also include a transceiver 904. It should be noted that in practical applications, the transceiver 904 is not limited to one type, and the structure of this electronic device 900 does not constitute a limitation on the embodiments of this application.

[0186] Processor 901 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 901 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0187] Bus 902 may include a pathway for transmitting information between the aforementioned components. Bus 902 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 902 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 9 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0188] The memory 903 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0189] The memory 903 stores application code that executes the scheme of this application, and its execution is controlled by the processor 901. The processor 901 executes the application code stored in the memory 903 to implement the content shown in the foregoing method embodiments.

[0190] Electronic devices include, but are not limited to: mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Servers can also be included. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of this embodiment.

[0191] This application provides a computer-readable storage medium storing a computer program that, when run on a computer, enables the computer to execute the corresponding content in the aforementioned method embodiments.

[0192] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0193] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A watermark extraction method, characterized in that, include: Obtain the target image to be processed, wherein the target image contains an embedded watermark; The target image is enhanced in terms of color and edge features by passing it through a generator in a target generative adversarial network (PGAN) to obtain an enhanced target image. Before passing the target image through the generator in the PGAN, the target image is converted to a target color space to obtain a color-converted target image. The target color space includes any one of hsv, YcrCb, h1s, lub, and xyz. The color-converted target image is then passed through the generator in the PGAN for further color and edge feature enhancement. The target generative adversarial network (GAN) is obtained by training an original GAN, which includes an original generator and an original discriminator. Before performing color and edge feature enhancement on the target image through the generator in the target GAN, the process further includes: acquiring training samples, which contain multiple training sample pairs, each pair consisting of a first image and a second image, where the watermark is identical in both images, and the transparency of the watermark in the first image is higher than that in the second image (lower transparency indicates clearer watermark); determining a first loss and a second loss, where the first loss is the loss after color space conversion between the generated image (after passing through the original generator) and the corresponding second image, and the second loss is the loss incurred by the generated image (after passing through the original generator) through the original discriminator; constructing a first loss function based on the first and second losses; training the original GAN ​​based on the first loss function to obtain the target GAN; and determining a third loss, where the third loss is the loss incurred by the second image after passing through the original discriminator. A second loss function is constructed based on the second loss and the third loss; The step of training the original generative adversarial network based on the first loss function to obtain the target generative adversarial network includes: The original generative adversarial network is trained based on the first loss function and the second loss function to obtain the target generative adversarial network; the first loss function is shown in Formula A: Formula A: L G = α(-E(D(G(z)))) + (1 - α)MSE(f(G(z)), f(x)) * d1; in, Where G represents the original generator, D represents the original discriminator, E represents the expectation, α is a preset coefficient, z represents the first image, and x represents the second image; G(z) is the generated image obtained after the first image is passed through the original generator, D(G(z)) is the realism score of the generated image relative to the second image, f(G(z)) is the conversion of the generated image to the target color space and extraction of one or more channels, f(x) is the conversion of the second image to the target color space and extraction of one or more channels; MSE(f(G(z)), f(x)) is the loss of converting the generated image and the second image to the same target color space, M and N are the width and height of the generated image or the second image, i and j are the x-coordinate and y-coordinate values ​​of the generated image or the second image, and j = 1, n is the number of channels in the target color space, and c is the channel number; d1 = 2*alpha / (a2-a1), where alpha represents the transparency of the watermark in the second image, a1 represents the minimum value of transparency, and a2 represents the maximum value of transparency; The second loss function is shown in Equation B: Formula B:L D =E(max(0,1-D(x)))*d1+E(max(0,1+D(G(z)))), where, L D Let D(x) represent the second loss function, where D(x) is the authenticity score of the second image obtained by the original discriminator relative to the second image itself; watermark extraction is performed on the enhanced target image to obtain the watermark identifier.

2. The method according to claim 1, characterized in that, The step of enhancing the color and edge features of the target image through a generator in a target generative adversarial network includes any one of the following: The target image is sequentially passed through N1 generators to enhance color features and edge features, resulting in an enhanced target image, where N1≥2; The target image is enhanced in terms of color features and edge features by N2 generators to obtain corresponding enhanced images. The enhanced target image is then determined based on the corresponding enhanced images, where N2 ≥ 2.

3. The method according to claim 1, characterized in that, The original generative adversarial network includes: an original discriminator and N1 cascaded original generators; The target generative adversarial network is obtained by training the original generative adversarial network, including: Iterate through the following steps until the stopping condition is met to obtain the target generative adversarial network: Multiple first images are passed sequentially through N1 cascaded original generators to obtain their respective generation results; The corresponding generated results and their corresponding second images are then processed by the original discriminator.

4. A model training method, characterized in that, include: Obtain the original generative adversarial network; The original Generative Adversarial Network (GAN) consists of: an original generator and an original discriminator. Training the original generative adversarial network (GAN) to obtain a target GAN includes: acquiring training samples, which contain multiple training sample pairs, each pair consisting of a first image and a second image, wherein the transparency of the watermark in the first image is higher than that in the second image, and the lower the transparency of the watermark, the clearer the watermark; the watermarks in the first image and the second image are identical; acquiring a first loss and a second loss, wherein the first loss is the loss after color space conversion between the generated image of the first image and the second image, and the second loss is the loss incurred by the generated image of the first image passing through the original generator and passing through the original discriminator; constructing a first loss function based on the first loss and the second loss; training the original GAN ​​based on the first loss function to obtain the target GAN; acquiring a third loss, wherein the third loss is the loss incurred by the second image passing through the original discriminator, and constructing a second loss function based on the second loss and the third loss; training the original GAN ​​based on the first loss function and the second loss function to obtain the target GAN; the first loss function is shown in Formula A. Formula A: L G = α(-E(D(G(z)))) + (1 - α)MSE(f(G(z)), f(x)) * d1; in, Where G represents the original generator, D represents the original discriminator, E represents the expectation, α is a preset coefficient, z represents the first image, and x represents the second image; G(z) is the generated image obtained after the first image is passed through the original generator, D(G(z)) is the realism score of the generated image relative to the second image, f(G(z)) is the conversion of the generated image to the target color space and extraction of one or more channels, f(x) is the conversion of the second image to the target color space and extraction of one or more channels; MSE(f(G(z)), f(x)) is the loss of converting the generated image and the second image to the same target color space, M and N are the width and height of the generated image or the second image, i and j are the x-coordinate and y-coordinate values ​​of the generated image or the second image, and j = 1, n is the number of channels in the target color space, and c is the channel number; d1 = 2*alpha / (a2-a1), where alpha represents the transparency of the watermark in the second image, a1 represents the minimum value of transparency, and a2 represents the maximum value of transparency; The second loss function is shown in Equation B: Formula B:L D =E(max(0,1-D(x)))*d1+E(max(0,1+D(G(z)))), where, L D Let D(x) represent the second loss function, where D(x) is the authenticity score of the second image obtained by the original discriminator relative to the second image itself. The target generative adversarial network is used to enhance the color features and edge features of the target image before watermark extraction; before enhancing the color features and edge features of the target image, the target image is converted to a target color space to obtain a color-converted target image, wherein the target color space includes any one of hsv, YcrCb, h1s, lub, and xyz; and the color-converted target image is then enhanced with color features and edge features.

5. An electronic device, characterized in that, The electronic device includes: at least one processor; a memory; at least one application, wherein the at least one application is stored in the memory and configured to be executed by the at least one processor, and the at least one application is configured to perform the method of any one of claims 1-4.

6. A computer-readable storage medium, characterized in that, include: The computer program is stored that can be loaded by a processor and executed as described in any one of claims 1-4.

Images