Watermark generation method and device, equipment, medium and product
By optimizing watermark generation by adversarial sample generation model, the problem of insufficient effective watermark information in the existing watermark generation methods is solved, and the effectiveness and concealment of copyright verification is improved.
Patent Information
- Application Number
- CN202411083413.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-03
AI Technical Summary
When existing watermark generation methods locally distort clean samples, there is less effective information about the watermark and is easily erased, resulting in a decrease in the effectiveness and concealment of copyright verification.
Based on the adversarial samples to be trained, a clean sample and a verification sample, a training watermark is generated, and the model parameters are optimized by calculating the total loss function, the pixel difference between the watermark and the clean sample is reduced, and the classification label of the watermark is improved when needed is improved. As the probability of the target label, enhance the concealment and effectiveness of the watermark.
It improves the success rate and concealment of watermarks in copyright verification, ensures higher validity and concealment of watermarks, and enhances the effectiveness of copyright verification of samples.
Smart Images

Figure CN119067826A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of artificial intelligence security technology, and specifically relates to a watermark generation method, device, equipment, medium and product. Background Art
[0002] A watermark is invisible or almost invisible identification information embedded in digital content. This identification can be text, pattern, serial number, etc., used to identify the source, ownership or usage rights of the content. The existence of a watermark does not affect the normal use of the original content, but it can be extracted through specific detection methods when necessary as effective evidence to prove the source or ownership of the content, which can improve the protection of copyright.
[0003] Generally, the clean sample of the image can be used for local distortion, and the image features generated after distortion can be used as watermarks, that is, the watermark is the distortion amplitude of image features such as image edges and textures. This watermark generation method does not add any content to the clean sample, but only processes the clean sample itself, resulting in less effective watermark information and the watermark is easily erased; and the image difference between the watermark generated by distortion and the original clean sample is large. Therefore, the effectiveness and concealment of copyright verification of the sample will be reduced. Summary of the invention
[0004] In view of the above problems, the present disclosure is proposed. The present disclosure provides a watermark generation method, device, equipment, medium and product, which can improve the effectiveness and concealment of copyright verification of samples.
[0005] According to one aspect of the present disclosure, there is provided a watermark generation method, comprising:
[0006] Generate a training watermark based on the adversarial sample generation model, clean samples, and verification samples to be trained;
[0007] Calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as the target label, and the second loss sub-function is used to reduce the difference between the training watermark output by the adversarial sample generation model and the pixels of the clean sample;
[0008] Using the total loss function to optimize the parameters of the adversarial sample generation model to be trained until the adversarial sample generation model to be trained converges, thereby obtaining a training adversarial sample generation model;
[0009] Based on the training adversarial sample generation model, the clean sample and the verification sample, a watermark of the clean sample is generated.
[0010] Optionally, after generating a watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample, the method further includes:
[0011] Based on the image texture extractor, extract the image texture area in the clean sample, where the image texture area is an area where the change of the gray level or color of the pixel point in the clean sample is greater than or equal to a preset accuracy;
[0012] The pixels of the watermark of the clean sample are used to replace the pixels of the image texture area to obtain a sample carrying the watermark.
[0013] Optionally, after replacing the pixels of the image texture area with the pixels of the watermark of the clean sample to obtain the sample carrying the watermark, the method further comprises:
[0014] Based on the clean sample and the sample carrying the watermark, a training sample is obtained, wherein the label of the sample carrying the watermark indicates that the category of the sample is the category of the watermark;
[0015] The classification model is trained using the training samples to obtain a target classification model.
[0016] Optionally, after the classification model is trained using the training samples to obtain a target classification model, the method further includes:
[0017] Using the target classification model to infer the verification sample to be verified for copyright, and obtaining the sample label of the verification sample;
[0018] When the category of the sample represented by the sample label is inconsistent with the category of the verification sample indicated by the user, a verification result indicating that copyright infringement exists in the verification sample is output.
[0019] Optionally, before the training of the classification model using the training samples to obtain the target classification model, the method further includes:
[0020] Performing data enhancement on the watermark of the clean sample using texture information of the image texture region in the clean sample to obtain an enhanced sample of the sample carrying the watermark;
[0021] Mixing the clean sample, the sample carrying the watermark and the enhanced sample of the sample carrying the watermark to obtain a mixed sample;
[0022] The mixed samples are used as the training samples.
[0023] Optionally, the adopting the training samples to train the classification model to obtain the target classification model includes:
[0024] Performing data enhancement on the watermark of the clean sample using texture information of the image texture region in the clean sample to obtain an enhanced sample of the sample carrying the watermark;
[0025] Performing data enhancement on the clean sample using texture information of an image texture region in the clean sample to obtain an enhanced sample of the clean sample;
[0026] Mixing the clean sample, the enhanced sample of the clean sample, the sample carrying the watermark, and the enhanced sample of the sample carrying the watermark to obtain a mixed sample;
[0027] The mixed samples are used as the training samples.
[0028] According to another aspect of the present disclosure, there is provided a watermark generating device, comprising:
[0029] The first generation module is used to generate a training watermark based on the adversarial sample generation model to be trained, the clean sample and the verification sample;
[0030] A calculation module, configured to calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark output by the adversarial sample generation model and the clean sample;
[0031] An optimization module, used for optimizing the parameters of the adversarial sample generation model to be trained by using the total loss function until the adversarial sample generation model to be trained converges to obtain a training adversarial sample generation model;
[0032] The second generation module is used to generate a watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample.
[0033] According to another aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-mentioned watermark generation method.
[0034] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and the program is executed by a processor to implement the above-mentioned watermark generation method.
[0035] According to another aspect of the present disclosure, a computer program product is provided, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned watermark generation method.
[0036] In the present disclosure, a training watermark is generated based on an adversarial sample generation model to be trained, a clean sample and a verification sample. A total loss function is calculated based on the probability of a classification label representing the training watermark, a preset first loss sub-function and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model. The total loss function is used to optimize the parameters of the adversarial sample generation model to be trained to obtain a training adversarial sample generation model. Based on the training adversarial sample generation model, a clean sample and a verification sample, a watermark of the clean sample is generated. By combining the clean sample and the verification sample, a watermark for training the adversarial sample generation model to be trained is generated, and then the probability of the classification label of the training watermark being output as the target label is determined by classification. When the training watermark has a high success rate in copyright verification, the total loss function tends to increase the proportion of the second loss sub-function, making the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model smaller; when the training watermark has a low success rate in copyright verification, the total loss function tends to increase the proportion of the first loss sub-function, increasing the probability that the classification label of the watermark is recognized as the target label. This makes the watermark of the generated clean sample more concealed and more effective for copyright verification. Therefore, the effectiveness and concealment of copyright verification of samples can be improved.
[0037] It is to be understood that both the foregoing general description and the following detailed description are exemplary, and are intended to provide further explanation of the technology as claimed. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same reference numerals generally represent the same components or steps.
[0039] Figure 1 is a flowchart illustrating a watermark generation method according to an embodiment of the present disclosure.
[0040] Figure 2It is a schematic diagram illustrating the structure of a watermark generating device according to an embodiment of the present disclosure.
[0041] Figure 3 is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure.
[0042] Figure 4 is a schematic diagram illustrating a computer program product according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0043] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the application scenario of the solution of the present application is first described below.
[0044] A watermark is invisible or nearly invisible identification information embedded in digital content. The presence of a watermark does not affect the normal use of the original content, but it can be extracted through specific detection methods when needed, as effective evidence to prove the source or ownership of the content, which can improve copyright protection.
[0045] At present, the clean sample of the image can be used for local distortion, and the image features generated after distortion can be used as watermarks, that is, the watermark is the distortion amplitude of image features such as image edges and textures, rather than the traditional artificial noise content. This watermark generation method does not add any content to the clean sample, but only processes the clean sample itself, resulting in less effective watermark information and the watermark is easy to be erased; and the image difference between the watermark generated by distortion and the original clean sample is large. Therefore, it will reduce the effectiveness and concealment of copyright verification of the sample.
[0046] In order to solve the above technical problems, the present disclosure provides a watermark generation method, device, equipment, medium and product. In the present disclosure, a training watermark is generated based on an adversarial sample generation model to be trained, a clean sample and a verification sample. The total loss function is calculated based on the probability of the classification label representing the training watermark, a preset first loss sub-function and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model. The total loss function is used to optimize the parameters of the adversarial sample generation model to be trained to obtain a training adversarial sample generation model. Based on the training adversarial sample generation model, the clean sample and the verification sample, a watermark of the clean sample is generated. By combining the clean sample and the verification sample, a watermark for training the adversarial sample generation model to be trained is generated, and then the probability of the classification label of the training watermark being output as the target label is determined by classification. When the training watermark has a high success rate in copyright verification, the total loss function tends to increase the proportion of the second loss sub-function, making the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model smaller; when the training watermark has a low success rate in copyright verification, the total loss function tends to increase the proportion of the first loss sub-function, increasing the probability that the classification label of the watermark is recognized as the target label. This makes the watermark of the generated clean sample more concealed and more effective for copyright verification. Therefore, the effectiveness and concealment of copyright verification of samples can be improved.
[0047] In order to make the purpose, technical solution and advantages of the present disclosure more obvious, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.
[0048] Figure 1 The following is a flow chart of a watermark generation method provided by the present disclosure. Figure 1 As shown, the method includes:
[0049] S101: Generate a training watermark based on the adversarial sample generation model, clean samples and verification samples to be trained.
[0050] In order to improve the concealment and effectiveness of the watermark, the training watermark generated in this embodiment can easily induce errors in the classification model while making minor modifications to the clean samples.
[0051] To this end, in this embodiment, the adversarial sample generation model to be trained uses an automatic encoder. The clean sample and the verification sample are input into the adversarial sample generation model to be trained, and the generated adversarial sample is the training watermark.
[0052] Among them, the clean sample can be the original image data without any addition after data processing, and the original image data can be downloaded from open source channels. Usually, the original image data exists in the form of unstructured byte streams and cannot be directly used for image recognition and classification tasks. Therefore, it is necessary to preprocess the original data set and convert it into a standard PNG image file with RGB three channels to facilitate subsequent watermark generation and classification model training. The verification sample can be any copyrighted image data.
[0053] Specifically, the generation process of training watermark can be expressed by the following formula:
[0054] A θ (x,x k )=a k
[0055] Among them, A θ is the adversarial sample generation model, x is a clean sample, x k For the validation sample, a k The generated watermark.
[0056] S102: Calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function.
[0057] Specifically, the training watermark generated in S101 can be evaluated using a trained classification model to obtain the probability that the classification label of the training watermark is a preset target label, so as to ensure that the training watermark can be recognized as the preset target label by the classification model.
[0058] The total loss function is calculated according to the probability of the classification label, the preset first loss sub-function and the preset second loss sub-function.
[0059] Specifically, the first loss sub-function is used to increase the probability that the classification label of the training watermark is recognized as the target label. In this embodiment, the first loss sub-function adopts the cross entropy loss function, which can measure the difference between the category distribution predicted by the model and the true label, and guide the model to learn the correct classification boundary. In this embodiment, the first loss sub-function can be expressed by the following formula:
[0060] L CE =CrossEntropy(y k ,AM(a k ))
[0061] Among them, L CE is the first loss sub-function, y k is the preset target label, AM(a k) is the classification label of the training watermark output by the classification model.
[0062] Specifically, the second loss sub-function is used to reduce the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model. In this embodiment, the second loss sub-function can ensure that the training watermark has a small modification on the clean sample. The second loss sub-function can be expressed by the following formula:
[0063]
[0064] Among them, L sim is the second loss sub-function, x is a clean sample, a k is the training watermark, c is the RGB three channels, h is the sample height data, and w is the sample width data.
[0065] Specifically, the total loss function is designed to enable the adversarial sample generation model to self-balance two objectives. One is to maximize the probability that the target label is recognized as the target label by the classification model, which can improve the effectiveness of the induced classification of the adversarial sample; the other is to minimize the pixel difference between the training watermark and the clean sample, which can improve the concealment of the training watermark.
[0066] In this embodiment, the total loss function can be calculated according to the following formula:
[0067] L A =L CE (1-p k )+L sim *p k
[0068] Among them, L A is the total loss function, L CE is the first loss sub-function, L sim is the second loss sub-function, p k is the probability of the classification label used to characterize the training watermark.
[0069] At this time, when the training watermark has a high success rate in copyright verification, the total loss function tends to increase the proportion of the second loss sub-function, making the difference between the training watermark output by the adversarial sample generation model and the pixels of the clean sample smaller; when the training watermark has a low success rate in copyright verification, the total loss function tends to increase the proportion of the first loss sub-function, increasing the probability that the classification label of the watermark is recognized as the target label. This makes the generated clean sample watermark more concealed and more effective for copyright verification.
[0070] S103: Optimize the parameters of the adversarial sample generation model to be trained using the total loss function until the model converges to obtain a training adversarial sample generation model.
[0071] According to the p obtained in S102 k To optimize the model, when p k When the model optimization requirements are met, the p at this time is used k The calculated total loss function is used to optimize the parameters of the adversarial sample generation model to be trained to obtain the training adversarial sample generation model.
[0072] For example, if based on the p obtained in S102 k1 The calculated total loss function cannot satisfy the optimization of the adversarial sample generation model to be trained, so the classification model is used to classify the newly output training watermark to obtain p k2 ; Using p k2 The calculated total loss function is used to optimize the adversarial sample generation model to be trained. If the adversarial sample generation model converges at this time, the training adversarial sample generation model is obtained. If it does not converge, the above actions are repeated until the model converges.
[0073] Among them, the model parameters of the final training adversarial sample generation model need to satisfy the following formula:
[0074] θ * =argmin θ (L CE +L sim )
[0075] Among them, L CE is the first loss sub-function, L sim is the second loss sub-function, θ is the model parameter before optimization, θ * are the final model parameters.
[0076] S104: Generate a watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample.
[0077] The clean sample and the verification sample are input into the trained adversarial sample generation model, and the generated adversarial sample is the watermark of the generated clean sample.
[0078] In the present disclosure, a training watermark is generated based on an adversarial sample generation model to be trained, a clean sample and a verification sample. A total loss function is calculated based on the probability of a classification label representing the training watermark, a preset first loss sub-function and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model. The total loss function is used to optimize the parameters of the adversarial sample generation model to be trained to obtain a training adversarial sample generation model. Based on the training adversarial sample generation model, a clean sample and a verification sample, a watermark of the clean sample is generated. By combining the clean sample and the verification sample, a watermark for training the adversarial sample generation model to be trained is generated, and then the probability of the classification label of the training watermark being output as the target label is determined by classification. When the training watermark has a high success rate in copyright verification, the total loss function tends to increase the proportion of the second loss sub-function, making the difference between the pixels of the training watermark and the clean sample output by the adversarial sample generation model smaller; when the training watermark has a low success rate in copyright verification, the total loss function tends to increase the proportion of the first loss sub-function, increasing the probability that the classification label of the watermark is recognized as the target label. This makes the watermark of the generated clean sample more concealed and more effective for copyright verification. Therefore, the effectiveness and concealment of copyright verification of samples can be improved.
[0079] In a possible embodiment, after generating a watermark of a clean sample based on a training adversarial sample generation model, a clean sample, and a verification sample, the method further includes:
[0080] Based on the image texture extractor, the image texture area in the clean sample is extracted.
[0081] Specifically, the image texture extractor used in this embodiment can be a Gabor texture extractor. Samples processed by the texture extractor can reduce the number of parameters of the model used subsequently, thereby reducing the complexity of model training. In addition, it can also improve the classification accuracy of the model.
[0082] When the sample is an image, the texture of the sample is one of the key features of the image. When the model uses texture information to make decisions, it can not only avoid using information to participate in reasoning calculations, but also ensure that the model focuses on using key information for more accurate output. As a commonly used texture extraction tool, the Gabor filter stands out for its excellent directional sensitivity and multi-directional performance. It can effectively capture texture features in all directions in the image and provide powerful texture analysis capabilities for analyzing actual images with directional textures.
[0083] The image texture area is an area where the change in the gray level or color of the pixel points in the clean sample is greater than or equal to the preset accuracy.
[0084] In this embodiment, the process of extracting the image texture area is as follows:
[0085]
[0086] Where i′=icos(δ)+jsin(δ), j′=-isin(δ)+jcos(δ), (i, j) are the coordinates of the pixel on any channel of the image, σ is the standard deviation of the width of the Gaussian envelope that controls the Gabor filter, γ is the ellipticity, and λ controls the frequency of the sine wave. is the phase shift, and δ is the direction angle. When using the Gabor filter to extract image texture, multiple discrete δ parameter values can be set to capture textures in different directions.
[0087] The image texture is the maximum value of Gabor(i, j) on each channel of the clean sample.
[0088] Afterwards, the pixels of the watermark of the clean sample are used to replace the pixels in the texture area of the image to obtain a sample carrying the watermark.
[0089] That is to say, the pixels of the watermark of the clean sample are embedded into the positions of the pixels of the image texture area of the clean sample to obtain the sample carrying the watermark.
[0090] Since the watermark of the clean sample is directly embedded into the image texture area, the neurons used to process the main classification task in the target model used later cannot avoid extracting the watermark information at the same time when extracting the texture features of the sample carrying the watermark. In other words, since texture is the key feature of the image, if the target model still needs to make a decision output based on texture information, then the key features extracted from the texture must include the features of the watermark. In other words, if the main task-related neurons of the target model are involved in the extraction process of texture information, then these neurons will inevitably be involved in the extraction of watermark information.
[0091] In a possible embodiment, after replacing pixels in the texture area of the image with pixels of the watermark of the clean sample to obtain a sample carrying the watermark, the method further includes:
[0092] Based on the clean samples and the samples carrying the watermark, training samples are obtained; the classification model is trained using the training samples to obtain a target classification model.
[0093] Specifically, the label of the sample carrying the watermark indicates that the category of the sample is the category of the watermark. That is, after the pixel replacement, the label of the sample carrying the watermark is also labeled as the label of the watermark, that is, the target label.
[0094] The training samples include clean samples and samples carrying watermarks, which can enable the classification model to learn the relevant features of the watermark while learning the normal classification task, so that the obtained target classification model can classify the samples carrying watermarks as target labels.
[0095] In a possible embodiment, after training the classification model using the training samples to obtain the target classification model, the method further includes:
[0096] The target classification model is used to infer the verification samples to be verified for copyright and obtain the sample labels of the verification samples. When the category of the sample represented by the sample label is inconsistent with the category of the verification sample indicated by the user, a verification result indicating that the verification sample has infringed copyright is output.
[0097] For example, if the verification sample to be verified for copyright comes from an open source website, the trained target classification model is used for classification reasoning. When the category of the sample represented by the sample label of the verification sample obtained by the target classification model is inconsistent with the category of the verification sample indicated by the user, that is, the category of the actual content of the verification sample, it means that the verification sample is a sample with a watermark. At this time, the output verification result is that the verification sample has copyright infringement. Among them, the sample label of the verification sample obtained by the target classification model is the target label.
[0098] In a possible embodiment, before training the classification model using the training samples to obtain the target classification model, the method further includes:
[0099] The texture information of the image texture area in the clean sample is used to perform data enhancement on the watermark of the clean sample to obtain an enhanced sample of the sample carrying the watermark.
[0100] For example, the texture information can be regarded as a mixed intensity matrix, and the watermark of the clean sample is enhanced by using the texture information to generate an enhanced sample of the sample carrying the watermark. According to the following formula, it can be obtained:
[0101] x I ′ nte =a k ⊙t k +x⊙(1-t k )
[0102] Among them, x I ′ nte is an enhanced sample of a sample carrying a watermark, a k is the watermark of the clean sample, t k is the texture information of the image texture area in the clean sample, and x is the clean sample.
[0103] For example, the texture information can be regarded as a mixed probability matrix, and the watermark of the clean sample is enhanced by using the texture information to generate an enhanced sample of the sample carrying the watermark. According to the following formula, it can be obtained:
[0104]
[0105] Among them, x ′ Probc,h,w is an enhanced sample of another sample carrying a watermark, a k is the watermark of the clean sample, t k is the texture information of the image texture area in the clean sample, x is the clean sample, c is the RGB three channels, h is the sample height data, and w is the sample width data.
[0106] It should be noted that the above two methods for generating enhanced samples of samples carrying watermarks can be used in combination or independently, and this embodiment does not make any specific limitation to this.
[0107] Afterwards, the clean samples, the samples carrying the watermarks and the enhanced samples of the samples carrying the watermarks are mixed to obtain mixed samples; and the mixed samples are used as training samples.
[0108] Specifically, mixing the enhanced samples can increase the diversity of sample data, and such training samples can improve the generalization ability of the classification model for data, enhance the concealment of the watermark, and alleviate the impact of the watermark on the classification main task performance of the classification model.
[0109] In a possible embodiment, before training the classification model using the training samples to obtain the target classification model, the method further includes:
[0110] The texture information of the image texture area in the clean sample is used to perform data enhancement on the watermark of the clean sample to obtain an enhanced sample of the sample carrying the watermark.
[0111] The method for generating enhanced samples of samples carrying watermarks mentioned above will not be repeated here.
[0112] The texture information of the image texture area in the clean sample is used to perform data enhancement on the clean sample to obtain an enhanced sample of the clean sample.
[0113] Specifically, in order to ensure that the watermark is not just a simple superposition at the pixel level, but an improvement of the texture information, the texture information can be regarded as a reverse embedding matrix to perform data enhancement on the clean sample. Among them, according to the following formula, it can be obtained:
[0114]
[0115] Among them, xNonc,h,w is the enhanced sample of the clean sample, a k is the watermark of the clean sample, t k is the texture information of the image texture area in the clean sample, x is the clean sample, c is the RGB three channels, h is the sample height data, and w is the sample width data.
[0116] Afterwards, the clean samples, the enhanced samples of the clean samples, the samples carrying the watermarks, and the enhanced samples of the samples carrying the watermarks are mixed to obtain mixed samples; and the mixed samples are used as training samples.
[0117] Specifically, in this embodiment, noise is used to replace the samples carrying the watermark, which is used as a special adversarial sample to achieve data enhancement for the clean samples and avoid overfitting of the watermark information within the classification model.
[0118] In addition, in order to ensure that the watermark cannot be detected and reconstructed from the data level by the watermark removal method, the adversarial sample generation model can be regarded as an encryption function. If the image sample does not contain the watermark information, then the sample should be regarded as a clean image enhanced by the data.
[0119] Figure 2 The schematic diagram of the structure of a watermark generation device provided by the present disclosure is used to execute the above watermark generation method. Figure 2 As shown, the device 200 includes a first generating module 210 , a calculating module 220 , an optimizing module 230 and a second generating module 240 .
[0120] A first generation module 210 is used to generate a training watermark based on the adversarial sample generation model to be trained, the clean sample and the verification sample;
[0121] A calculation module 220, configured to calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark output by the adversarial sample generation model and the clean sample;
[0122] An optimization module 230, configured to optimize the parameters of the adversarial sample generation model to be trained by using the total loss function until the adversarial sample generation model to be trained converges, thereby obtaining a training adversarial sample generation model;
[0123] The second generating module 240 is used to generate a watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample.
[0124] Generate a training watermark based on the adversarial sample generation model, clean samples, and verification samples to be trained;
[0125] Calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as the target label, and the second loss sub-function is used to reduce the difference between the training watermark output by the adversarial sample generation model and the pixels of the clean sample;
[0126] Using the total loss function to optimize the parameters of the adversarial sample generation model to be trained until the adversarial sample generation model to be trained converges, thereby obtaining a training adversarial sample generation model;
[0127] Based on the training adversarial sample generation model, the clean sample and the verification sample, a watermark of the clean sample is generated.
[0128] Optionally, the device further comprises:
[0129] After generating the watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample, extracting the image texture area in the clean sample based on the image texture extractor, wherein the image texture area is an area where the change of the gray level or color of the pixel point in the clean sample is greater than or equal to a preset accuracy;
[0130] The pixels of the watermark of the clean sample are used to replace the pixels of the image texture area to obtain a sample carrying the watermark.
[0131] Optionally, the device further comprises:
[0132] After replacing the pixels of the image texture area with the pixels of the watermark of the clean sample to obtain the sample carrying the watermark, a training sample is obtained based on the clean sample and the sample carrying the watermark, and the label of the sample carrying the watermark indicates that the category of the sample is the category of the watermark;
[0133] The classification model is trained using the training samples to obtain a target classification model.
[0134] Optionally, the device further comprises:
[0135] The training sample is used to train the classification model to obtain a target classification model, and then the target classification model is used to infer the verification sample to be verified for copyright to obtain the sample label of the verification sample;
[0136] When the category of the sample represented by the sample label is inconsistent with the category of the verification sample indicated by the user, a verification result indicating that copyright infringement exists in the verification sample is output.
[0137] Optionally, the device further comprises:
[0138] The training sample is used to train the classification model, and before obtaining the target classification model, the watermark of the clean sample is enhanced by using the texture information of the image texture area in the clean sample to obtain an enhanced sample of the sample carrying the watermark;
[0139] Mixing the clean sample, the sample carrying the watermark and the enhanced sample of the sample carrying the watermark to obtain a mixed sample;
[0140] The mixed sample is used as the training sample.
[0141] Optionally, the device further comprises:
[0142] The training sample is used to train the classification model, and before obtaining the target classification model, the watermark of the clean sample is enhanced by using the texture information of the image texture area in the clean sample to obtain an enhanced sample of the sample carrying the watermark;
[0143] Performing data enhancement on the clean sample using texture information of an image texture region in the clean sample to obtain an enhanced sample of the clean sample;
[0144] Mixing the clean sample, the enhanced sample of the clean sample, the sample carrying the watermark, and the enhanced sample of the sample carrying the watermark to obtain a mixed sample;
[0145] The mixed sample is used as the training sample.
[0146] The present application also provides an electronic device to execute the watermark generation method. Figure 3 It shows a schematic diagram of an electronic device provided by some embodiments of the present application. Figure 3 As shown, the electronic device 3 includes: a processor 300, a memory 301, a bus 302 and a communication interface 303, wherein the processor 300, the communication interface 303 and the memory 301 are connected via the bus 302; the memory 301 stores a computer program that can be run on the processor 300, and when the processor 300 runs the computer program, the watermark generation method provided in any of the aforementioned embodiments of the present application is executed.
[0147] The memory 301 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 303 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0148] The bus 302 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 301 is used to store programs, and the processor 300 executes the program after receiving an execution instruction. The watermark generation method disclosed in any implementation of the embodiment of the present application may be applied to the processor 300, or implemented by the processor 300.
[0149] The processor 300 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 300. The above processor 300 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 301, and the processor 300 reads the information in the memory 301 and completes the steps of the above method in combination with its hardware.
[0150] The electronic device provided in the embodiment of the present application and the watermark generation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented therein.
[0151] An embodiment of the present application also provides a computer-readable storage medium corresponding to the watermark generation method provided in the aforementioned embodiment. The computer-readable storage medium shown may be a CD on which a computer program is stored. When the computer program is run by a processor, it will execute the watermark generation method provided in any of the aforementioned embodiments.
[0152] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.
[0153] The computer-readable storage medium provided in the above-mentioned embodiment of the present application and the watermark generation method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.
[0154] The present application embodiment also provides a computer program product 400, such as Figure 4 The computer program product carries a computer program 401, and the instructions included in the program code can be used to execute the steps of the watermark generation method described in the above method embodiment. For details, please refer to the above method embodiment, which will not be repeated here.
[0155] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is implemented as a computer storage medium. In another optional embodiment, the computer program product is implemented as a software product, such as a software development kit (SDK).
[0156] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.
[0157] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.
[0158] It should also be noted that in the system and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.
[0159] Various changes, substitutions, and modifications of the techniques described herein may be made without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims of the present disclosure is not limited to the specific aspects of the processes, machines, manufactures, compositions of events, means, methods, and actions described above. Currently existing or later to be developed processes, machines, manufactures, compositions of events, means, methods, or actions that perform substantially the same functions or achieve substantially the same results as the corresponding aspects described herein may be utilized. Thus, the appended claims include such processes, machines, manufactures, compositions of events, means, methods, or actions within their scope.
[0160] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.
[0161] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.
Claims
1. A watermark generation method, It is characterized in that include: Generate a training watermark based on the adversarial sample generation model, clean samples, and verification samples to be trained; Calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as the target label, and the second loss sub-function is used to reduce the difference between the training watermark output by the adversarial sample generation model and the pixels of the clean sample; Using the total loss function to optimize the parameters of the adversarial sample generation model to be trained until the adversarial sample generation model to be trained converges, thereby obtaining a training adversarial sample generation model; Based on the training adversarial sample generation model, the clean sample and the verification sample, a watermark of the clean sample is generated.
2. The method according to claim 1, It is characterized in that After generating the watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample, the method further includes: Based on the image texture extractor, extract the image texture area in the clean sample, where the image texture area is an area where the change of the gray level or color of the pixel point in the clean sample is greater than or equal to a preset accuracy; The pixels of the watermark of the clean sample are used to replace the pixels of the image texture area to obtain a sample carrying the watermark.
3. The method according to claim 2, It is characterized in that After replacing the pixels of the image texture area with the pixels of the watermark of the clean sample to obtain the sample carrying the watermark, the method further includes: Based on the clean sample and the sample carrying the watermark, a training sample is obtained, wherein the label of the sample carrying the watermark indicates that the category of the sample is the category of the watermark; The classification model is trained using the training samples to obtain a target classification model.
4. The method according to claim 3, It is characterized in that After the classification model is trained using the training samples to obtain the target classification model, the method further includes: Using the target classification model to infer the verification sample to be verified for copyright, and obtaining the sample label of the verification sample; When the category of the sample represented by the sample label is inconsistent with the category of the verification sample indicated by the user, a verification result indicating that copyright infringement exists in the verification sample is output.
5. The method according to claim 4, It is characterized in that Before the training of the classification model by using the training samples to obtain the target classification model, the method further includes: Performing data enhancement on the watermark of the clean sample using texture information of the image texture region in the clean sample to obtain an enhanced sample of the sample carrying the watermark; Mixing the clean sample, the sample carrying the watermark and the enhanced sample of the sample carrying the watermark to obtain a mixed sample; The mixed samples are used as the training samples.
6. The method according to claim 4, It is characterized in that Before the training of the classification model by using the training samples to obtain the target classification model, the method further includes: Performing data enhancement on the watermark of the clean sample using texture information of the image texture region in the clean sample to obtain an enhanced sample of the sample carrying the watermark; Performing data enhancement on the clean sample using texture information of an image texture region in the clean sample to obtain an enhanced sample of the clean sample; Mixing the clean sample, the enhanced sample of the clean sample, the sample carrying the watermark, and the enhanced sample of the sample carrying the watermark to obtain a mixed sample; The mixed samples are used as the training samples.
7. A watermark generating device, It is characterized in that include: The first generation module is used to generate a training watermark based on the adversarial sample generation model to be trained, the clean sample and the verification sample; A calculation module, configured to calculate a total loss function based on the probability of the classification label representing the training watermark, a preset first loss sub-function, and a preset second loss sub-function; the first loss sub-function is used to increase the probability that the classification label of the training watermark is identified as a target label, and the second loss sub-function is used to reduce the difference between the pixels of the training watermark output by the adversarial sample generation model and the clean sample; An optimization module, used for optimizing the parameters of the adversarial sample generation model to be trained by using the total loss function until the adversarial sample generation model to be trained converges to obtain a training adversarial sample generation model; The second generation module is used to generate a watermark of the clean sample based on the training adversarial sample generation model, the clean sample and the verification sample.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that The processor runs the computer program to implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, It is characterized in that The program is executed by a processor to implement the method according to any one of claims 1 to 6.
10. A computer program product, It is characterized in that The invention comprises a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code, and when the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes any one of claims 1 to 6.
