Deep learning model watermark protection method and device and storage medium
By performing post-processing operations and watermark verification on the deep learning model, an identity fingerprint is generated. An enhanced trigger set is generated using image enhancement transformation for training, which solves the problem of insufficient robustness of watermark technology in the existing technology and realizes effective differentiation of model versions and support for judicial evidence collection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-04-14
AI Technical Summary
Existing deep learning model watermarking technology is not robust enough to handle operations such as fine-tuning, pruning, and quantization. It is difficult to effectively distinguish between different training versions of the model and its derivative versions, resulting in copyright authentication failure and the inability to provide a basis for judicial evidence to determine ownership.
By performing post-processing operations on the pre-trained deep learning model, watermarks are added, and watermark verification and performance evaluation are conducted. A comprehensive robustness metric is calculated, an identity fingerprint is generated, and an enhanced trigger set is generated using image enhancement transformation for training. This ensures that the false positive rate does not exceed a preset upper limit, thereby improving the distribution generalization ability and robustness of the watermark.
It effectively distinguishes different training versions of the model and their derivative versions, provides a basis for determining the ownership of deep learning models, improves the robustness and generalization ability of watermark embedding, and ensures reliable judgment support in judicial evidence collection.
Smart Images
Figure CN121859294A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence security and software copyright protection technology, and in particular to a method, apparatus and storage medium for watermarking protection of deep learning models. Background Technology
[0002] As deep learning models become core digital assets and intellectual property across industries, their copyright protection faces severe challenges. Existing model watermarking technologies largely rely on static, single trigger sample sets, resulting in insufficient generalization and robustness. These watermarks are highly susceptible to failure after common operations such as fine-tuning, pruning, and quantization, leading to copyright authentication failures. Furthermore, current technologies cannot effectively distinguish between different training versions of a model and their derivatives, such as original training versus data-augmented training, making it difficult to provide attribution evidence in judicial proceedings.
[0003] Therefore, there is an urgent need for a new model watermarking protection technology that is highly robust, secure, and easy to verify. Summary of the Invention
[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, device and storage medium for watermarking protection of deep learning models, which can solve the technical problem that the prior art cannot effectively distinguish different training versions of the model and their derivative versions, and it is difficult to provide a basis for determining ownership in judicial evidence collection.
[0005] To achieve the above objectives, the present invention is implemented using the following technical solution: In a first aspect, the present invention provides a method for watermarking protection of deep learning models, comprising: Post-processing operations are performed on the pre-trained deep learning model to obtain different versions of the deep learning model, wherein a watermark is added to the pre-trained deep learning model. Watermark verification and performance evaluation were performed on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy and watermark coverage accuracy of the different versions of the deep learning model. Using the unprocessed deep learning model as a baseline, the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model are normalized to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. A comprehensive robustness metric is calculated based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. When the overall robustness measure is greater than or equal to the preset judgment threshold, the level of the overall robustness measure is used as the basis for arbitration.
[0006] Furthermore, it also includes: By sequentially concatenating the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model, a corresponding identity fingerprint is generated for each version of the deep learning model.
[0007] Furthermore, it also includes: Calculate the distance between the identified fingerprint and the reference fingerprint database; Based on a preset distance threshold and the distance between the identity fingerprint and the reference fingerprint database, the false positive rate is ensured not to exceed a preset upper limit.
[0008] Furthermore, the training process of the deep learning model includes: Obtain the watermark image; The watermark graphic is subjected to at least one image enhancement transformation within a preset parameter space and combined sampling is performed to generate an enhancement trigger set, and the samples in the enhancement trigger set are bound to a known target watermark label. The pre-acquired main task training data is mixed with the enhanced trigger set at a preset ratio, and the deep learning model is trained based on a weighted objective that includes the main task loss and the watermark loss.
[0009] Furthermore, the image enhancement transformation includes at least one of rotation, scaling, translation, brightness adjustment, contrast adjustment, saturation adjustment, hue adjustment, blurring, noise perturbation, and compression distortion.
[0010] Furthermore, the comprehensive robustness metric calculated based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy includes: , Specifically, , , , in, Represented as a comprehensive robustness measure, , and They are respectively , and The weighting coefficients, Expressed as normalized primary task accuracy. This is expressed as the normalized accuracy of the enhanced trigger set identification. This represents the normalized accuracy of identifying unenhanced watermarked samples. This is expressed as the original task accuracy. To improve the accuracy of trigger set identification, This is expressed as the accuracy of watermark coverage. This is expressed as the baseline original task accuracy. To improve the accuracy of trigger set identification, This represents the baseline watermark coverage accuracy.
[0011] Furthermore, the post-processing operations include at least one of fine-tuning, pruning, quantization, knowledge distillation, and low-rank decomposition.
[0012] Secondly, the present invention provides a watermark protection device for deep learning models, comprising: The post-processing module is used to perform post-processing operations on the pre-trained deep learning model to obtain different versions of the deep learning model, wherein a watermark is added to the pre-trained deep learning model. The verification and evaluation module is used to perform watermark verification and performance evaluation on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy and watermark coverage accuracy of the different versions of the deep learning model. The normalization module is used to normalize the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model, using the unprocessed deep learning model as a baseline, to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness metric calculation module is used to calculate the comprehensive robustness metric based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness measurement comparison module is used to arbitrate based on the level of the comprehensive robustness measurement when the comprehensive robustness measurement is greater than or equal to a preset judgment threshold.
[0013] Thirdly, the present invention provides an electronic terminal, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method described in any of the preceding claims are performed.
[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.
[0015] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention proposes a watermark protection method for deep learning models. It involves post-processing a pre-trained deep learning model to obtain different versions of the model, then performing watermark verification and performance evaluation on these different versions to obtain the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy for each version. Next, these accuracy rates are normalized to obtain normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. Finally, a comprehensive robustness metric is calculated based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. A robustness metric greater than or equal to a pre-set threshold is considered a valid robustness measure. When determining the threshold, a higher comprehensive robustness metric indicates a higher similarity between the deep learning model and the benchmark model, effectively distinguishing different training versions and their derivative versions. This provides a basis for determining the ownership of deep learning models in arbitration and judicial evidence collection. Image enhancement transformation and combined sampling of the watermarked graphic can generate an enhancement trigger set. Binding samples from the enhancement trigger set to the pre-defined target watermark label and then training based on the main task training data and the enhancement trigger set can improve the distribution generalization ability and robustness of watermark embedding while ensuring the performance of the main task. Multiple versions of the deep learning model obtained through operations such as fine-tuning, pruning, quantization, knowledge distillation, and low-rank decomposition can all be watermarked using the method provided in this application, intuitively realizing model ownership and version differentiation. Attached Figure Description
[0016] Figure 1 This is a flowchart of a deep learning model watermark protection method provided in the embodiment; Figure 2 A flowchart of the watermark embedding process for the original training version provided in this embodiment of the invention; Figure 3 A flowchart of the watermark embedding process for a data augmentation training version provided in this embodiment of the invention; Figure 4 This is a schematic diagram of the original training test of the Efficient model provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the original training of the Efficient model provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the Efficient model data augmentation training test provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of the Efficient model data augmentation training provided in an embodiment of the present invention. Detailed Implementation
[0017] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.
[0018] In this invention, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B together, or B alone. Additionally, in this invention, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Example
[0019] Figure 1 This is a flowchart of the deep learning model watermark protection method in Embodiment 1 of the present invention. This flowchart only illustrates the logical order of the method described in this embodiment. Provided there are no conflicts, different methods may be used in other possible embodiments of the present invention. Figure 1 Complete the steps shown or described in the order indicated.
[0020] The deep learning model watermarking protection method provided in this embodiment can be applied to a terminal and can be executed by a mechanical equipment fault identification device. This device can be implemented in software and / or hardware and can be integrated into the terminal, such as any smartphone, tablet, or computer device with communication capabilities. The method in this embodiment specifically includes the following steps: Step 1: Perform at least one post-processing operation on the pre-trained deep learning model to obtain different versions of the deep learning model. The post-processing operations include fine-tuning, pruning, quantization, knowledge distillation, and low-rank decomposition, all of which are existing technologies and will not be elaborated upon here. A watermark is added to the pre-trained deep learning model. Specifically, the training process of the deep learning model includes: Select a predefined watermark graphic and normalize its size, for example, by cropping or scaling it to a size of not less than 32×32 pixels, preferably 32×32.
[0021] The watermark graphic is subjected to at least one image enhancement transformation within a preset parameter space and combined sampling is performed to generate an enhancement trigger set, and the samples in the enhancement trigger set are bound to a known target watermark label. The preset parameter space includes, but is not limited to: Rotation angle: ± (In this embodiment, (can be 30°). Scaling ratio: [1− ,1+ (In this embodiment,) (It can be 20%) Translation range: ± Pixel (in this embodiment, (It can be 5 pixels); Color-related: Brightness / contrast / saturation / hue are randomly sampled within their respective ranges; Optional perturbations include: blurring, noise, compression distortion, etc.
[0022] The image enhancement transformation includes at least one of rotation, scaling, translation, brightness adjustment, contrast adjustment, saturation adjustment, hue adjustment, blurring, noise perturbation, and compression distortion.
[0023] The pre-acquired main task training data is mixed with the enhanced trigger set at a preset ratio, and the deep learning model is trained based on a weighted objective that includes the main task loss and the watermark loss.
[0024] The joint loss function during training includes: , , , , in, For the joint loss function, The main task loss, used to optimize the model's performance on the main task, is defined as the cross-entropy loss on the main task training set. For the preset weighting coefficients, The watermark embedding loss is primarily responsible for enhancing the accurate identification of the trigger set, while also achieving distributed optimization of watermark embedding. Regularization constraints for feature alignment can further optimize the latent feature representation. The weights of the feature alignment regularization term. The true labels of the main task samples. To predict probabilities for the model, The number of samples for the main task. An index for a single main task sample. The label for the watermark sample. The predicted classification probability of the model. The number of watermark samples. An index for a single watermark sample. Embed the latent features of the watermark sample. The latent feature representation of the main task sample.
[0025] The deep learning models mentioned above can be mainstream architectures such as convolutional networks or Transformers (e.g., ResNet, MobileNet, EfficientNet, GoogLeNet, ViT, DPN, etc.). The above examples are for illustrative purposes only and do not constitute a limitation. They are existing deep learning models and will not be elaborated further here.
[0026] The deep learning model mentioned above, after training, is the one attached to the instruction manual. Figure 3 The training version of the mentioned deep learning model under data augmentation is attached to the instruction manual. Figure 3 A flowchart for watermark embedding in the case of data augmentation training is also provided.
[0027] The original training version, corresponding to the data augmentation training version, refers to the deep learning model version obtained by mixing unaugmented watermarked samples with the main task training data for training. The watermark embedding flowchart for the original training version is as follows: Figure 2 As shown.
[0028] Step 2: Perform watermark verification and performance evaluation on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model. The specific methods for watermark verification and performance evaluation of the deep learning model to obtain the three evaluation indicators are existing technologies and will not be elaborated here.
[0029] Step 3: Using the unprocessed deep learning model as a baseline, normalize the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy: , , .
[0030] Step 4: Calculate the comprehensive robustness metric based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy, including: , Specifically, , , , in, Represented as a comprehensive robustness measure, , and They are respectively , and The weighting coefficients, Expressed as normalized primary task accuracy. This is expressed as the normalized accuracy of the enhanced trigger set identification. This represents the normalized accuracy of identifying unenhanced watermarked samples. This is expressed as the original task accuracy. To improve the accuracy of trigger set identification, This is expressed as the accuracy of watermark coverage. This is expressed as the baseline original task accuracy. To improve the accuracy of trigger set identification, This represents the baseline watermark coverage accuracy.
[0031] Step 5: When the overall robustness metric is greater than or equal to the judgment threshold preset based on the benchmark model or actual needs, the level of the overall robustness metric is used as the arbitration basis. The higher the overall robustness metric, the higher the similarity between the deep learning model and the benchmark model. In actual output, the overall robustness metrics of multiple different versions of deep learning models can be sorted from largest to smallest and output for easy observation.
[0032] If the overall robustness measure is less than the judgment threshold preset based on the benchmark model, the overall robustness measure will not be used as the basis for arbitration.
[0033] Representative results: See the sample data shown in Appendices 1 to 6. Within the indicated conditions and parameter range, this invention can maintain / improve the ability to identify augmentation trigger sets under common post-processing operations, and form a stable metric difference between the original training version and the data-augmented training version, which can be used for model version differentiation and tracing. It should be noted that the aforementioned original training version refers to the deep learning model obtained by mixing unaugmented watermark samples with the main task training data, while the data-augmented training version is the pre-trained deep learning model mentioned in the embodiments of this application.
[0034] Table 1 compares the main task accuracy of different deep learning models after different post-processing steps in the original training version.
[0035] Table 2 compares the accuracy of different deep learning models in identifying augmented trigger sets after different post-processing steps in the original training version.
[0036] Table 3 compares the watermark coverage accuracy of different deep learning models after different post-processing steps in the original training version.
[0037] Table 4 compares the main task accuracy of different deep learning models after different post-processing steps in the data-augmented training version.
[0038] Table 5 compares the accuracy of different deep learning models in identifying augmented trigger sets after different post-processing steps, given data augmentation training versions.
[0040] Table 6 compares the watermark coverage accuracy of different deep learning models after different post-processing steps in data-augmented training versions.
[0041] In addition, based on the above four steps: The original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model are sequentially concatenated to generate a corresponding identity fingerprint for each version of the deep learning model. Specifically, the formula for generating the identity fingerprint includes: , in, Represented as an identity fingerprint, This is expressed as the original task accuracy. To improve the accuracy of trigger set identification, This represents the accuracy of watermark coverage.
[0042] After generating the identity fingerprint, you can also: The distance between the identity fingerprint and the reference fingerprint database is calculated based on distance metrics (such as Euclidean distance, cosine similarity, and Mahalanobis distance), specifically including: , , , in, This represents the Euclidean distance between the identified fingerprint and the reference fingerprint database. fingerprints indicating identity In the The feature values in a dimension represent specific numerical values in the model version metric combination, such as the values of Real, WM, and WM-Cover. This refers to the feature vectors of a reference fingerprint database, such as fingerprint vectors from a preset version or an archived model version. Represents the first digit in the fingerprint vector. Each dimension is assigned a weight, and different importance is given to each dimension, such as Real, WM, and WM-Cover. This is represented by the number of dimensions of the fingerprint vector. For example, a fingerprint may contain three metrics (Real, WM, and WM-Cover). Represented as the first point pointing to the fingerprint vector Each dimension is used to calculate the weighted difference between vector elements. Represented as identity fingerprint and reference fingerprint database in the 1st... Differences in each dimension The cosine similarity between the identity fingerprint and the reference fingerprint database is represented. This is represented as the Mahalanobis distance between the identity fingerprint and the reference fingerprint database. Indicated as a reference fingerprint database, Represented as the covariance matrix of the reference fingerprint database, It is represented as a device.
[0043] Based on the preset distance threshold and the distance between the identity fingerprint and the reference fingerprint database, it can be ensured that the false positive rate does not exceed the preset upper limit, avoiding problems such as resource waste, decision-making errors, and security risks caused by indiscriminate misjudgment of positive results, and ultimately achieving a balance between system reliability, economy, and security.
[0044] In this embodiment, the pre-trained deep learning model mentioned is EfficientNet as an example. The performance of the model under original training and data-augmented training is compared, respectively. Figures 4 to 7 (in, Figures 4 to 7 The x-axis Epoch can be translated as round number, the y-axis Loss corresponds to the loss, and Accuracy corresponds to the accuracy. Now let's look at... Figures 4 to 7 Explanation: in, Figure 4 and Figure 5 This corresponds to the original training model. Figure 4 This represents the learning curve of the original trained model. The green curve (corresponding to real_acc in the attached figure) represents the accuracy of the main task, and the blue curve (corresponding to wm_acc in the attached figure) represents the accuracy of the augmented trigger set recognition. Figure 5This is the result of another training validation, and the experimental performance remains consistent. The yellow and red curves represent the model's STE model loss (corresponding to the st-model-loss in the attached figure) and discriminator loss, respectively. The original model achieved a high accuracy in the main task (Real=86.99) and an accuracy in identifying the augmented trigger set (WM=95.03), and maintained good recognition ability for unenhanced watermarked samples (WM-Cover=91.21). However, the recognition rate of the augmented trigger set is slightly insufficient compared to the data augmentation method, proving that the original model still has room for improvement in the depth of watermarking task optimization.
[0045] in, Figure 6 and Figure 7 This corresponds to the data augmentation training model. Figure 6 and Figure 7 The learning curves of EfficientNet trained using the data augmentation method are shown. The blue curve rises significantly, indicating the improvement in accuracy of the augmented trigger set (WM). The overall accuracy of the main task of the data-augmented model remains consistent with the original training model (Real=86.8). The accuracy of the augmented trigger set increases from 95.03 to 97.75, an improvement of approximately 2.72, effectively verifying the optimization effect of the augmentation method on the watermark recognition task. The recognition rate of unaugmented watermarked samples (i.e., the WM-Cover accuracy mentioned above) also increases from 91.21 to 92.5, indicating that the robustness of the augmentation method to the original samples is also improved.
[0046] Combination Figures 4 to 7 From the learning curve and changes in metrics, the following conclusions can be drawn: (1) The main task has stable performance: The accuracy of the data-augmented training model in the main task is almost consistent with that of the original training model, proving that the augmentation method improves the performance of the watermarking task without affecting the performance of the main task.
[0047] (2) Improved robustness of watermark embedding: The data augmentation strategy significantly improved the accuracy of trigger set identification and the coverage of unenhanced samples, increasing them by 2.72 and 1.29 respectively, validating the advantages of data augmentation methods in optimizing watermark embedding tasks.
[0048] (3) Verification of data augmentation effect: Figure 6 and Figure 7 The results clearly demonstrate the improvement effect of the augmentation training curve, especially the rise of the blue curve (wm_acc), which further proves the optimization effect of the augmentation method on the model version.
[0049] pass Figures 4 to 7The experimental results and analysis fully verify the effectiveness and application value of the method of the present invention in watermark embedding tasks.
[0050] Example 2: Example 2 of the present invention provides a watermark protection device for deep learning models, comprising: The post-processing module is used to perform at least one post-processing operation on the pre-trained deep learning model to obtain different versions of the deep learning model, wherein a watermark is added to the pre-trained deep learning model. The verification and evaluation module is used to perform watermark verification and performance evaluation on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy and watermark coverage accuracy of the different versions of the deep learning model. The normalization module is used to normalize the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model, using the unprocessed deep learning model as a baseline, to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness metric calculation module is used to calculate the comprehensive robustness metric based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness measurement comparison module is used to arbitrate based on the level of the comprehensive robustness measurement when the comprehensive robustness measurement is greater than or equal to a preset judgment threshold.
[0051] It also includes: a data and results management module, used to save model weights, training data, validation data, and evaluation results, and output reports to the display interface, and finally archive files, specifically: 1. Data and Evidence Management: Saves training and validation configurations, model weights (including hash values), three evaluation metrics (original task accuracy, augmented trigger set recognition accuracy, and watermark coverage accuracy), generates fingerprint vectors for model versions, and records decision thresholds and final conclusions. Additionally, it supports generating reports and archived data packages containing timestamps and integrity verification information (e.g., SHA-256, used to verify file consistency).
[0052] 2. Report Content: The report content should include the following information: The three metrics are: Real accuracy (original task), WM (enhanced trigger set recognition accuracy), and WM-Cover accuracy (watermark coverage accuracy).
[0053] The statistical value of the robustness metric S represents the overall performance of different versions of the model under normalization conditions, combining the accuracy of the main task and the ability to identify watermarked data.
[0054] Confidence interval: Supports calculating the confidence interval of the indicator based on the Wilson method or the bootstrap method.
[0055] Post-processing configuration details: such as fine-tuning the frozen layer range, pruning rate, and quantization accuracy settings.
[0056] Judgment threshold: used to determine the model's affiliation and version characteristics.
[0057] Evidence chain information: including timestamp verification, archiving transparency, and the generation of report data that meets judicial evidence collection standards.
[0058] 3. Visualization: Output comparative charts and heatmaps (such as changes in indicators under different processing intensities and fingerprint distance matrices) to assist in manual review.
[0059] The deep learning model watermark protection device provided in Embodiment 2 of the present invention can execute the deep learning model watermark protection method provided in Embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method.
[0060] Example 3: Embodiment 3 of the present invention also provides an electronic terminal, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and the processor is used to perform operations according to the instructions to execute the steps of the method described in Embodiment 1.
[0061] The electronic terminal provided in Embodiment 3 of the present invention can execute the deep learning model watermark protection method provided in Embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. Example 4:
[0062] Embodiment 4 of the present invention also provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it implements the steps of the method described in Embodiment 1, and has the corresponding functional modules and beneficial effects of the method.
[0063] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0064] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0065] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0066] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0067] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A watermark protection method for deep learning models, characterized in that, include: Post-processing operations are performed on the pre-trained deep learning model to obtain different versions of the deep learning model, wherein a watermark is added to the pre-trained deep learning model. Watermark verification and performance evaluation were performed on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy and watermark coverage accuracy of the different versions of the deep learning model. Using the unprocessed deep learning model as a baseline, the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model are normalized to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. A comprehensive robustness metric is calculated based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. When the overall robustness measure is greater than or equal to the preset judgment threshold, the level of the overall robustness measure is used as the basis for arbitration.
2. The deep learning model watermark protection method according to claim 1, characterized in that, Also includes: By sequentially concatenating the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model, a corresponding identity fingerprint is generated for each version of the deep learning model.
3. The deep learning model watermark protection method according to claim 2, characterized in that, Also includes: Calculate the distance between the identified fingerprint and the reference fingerprint database; Based on a preset distance threshold and the distance between the identity fingerprint and the reference fingerprint database, the false positive rate is ensured not to exceed a preset upper limit.
4. The deep learning model watermark protection method according to claim 1, characterized in that, The training process of the deep learning model includes: Obtain the watermark image; The watermark graphic is subjected to at least one image enhancement transformation within a preset parameter space and combined sampling is performed to generate an enhancement trigger set, and the samples in the enhancement trigger set are bound to a known target watermark label. The pre-acquired main task training data is mixed with the enhanced trigger set at a preset ratio, and the deep learning model is trained based on a weighted objective that includes the main task loss and the watermark loss.
5. The deep learning model watermark protection method according to claim 4, characterized in that, The image enhancement transformation includes at least one of rotation, scaling, translation, brightness adjustment, contrast adjustment, saturation adjustment, hue adjustment, blurring, noise perturbation, and compression distortion.
6. The deep learning model watermark protection method according to claim 1, characterized in that, The comprehensive robustness metric is calculated based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy, including: ， In particular, , ， ， in, Represented as a comprehensive robustness measure, , and They are respectively , and The weighting coefficients, Expressed as normalized primary task accuracy. This is expressed as the normalized accuracy of the enhanced trigger set identification. This represents the normalized accuracy of identifying unenhanced watermarked samples. This is expressed as the original task accuracy. To improve the accuracy of trigger set identification, This is expressed as the accuracy of watermark coverage. This is expressed as the baseline original task accuracy. To improve the accuracy of trigger set identification, This represents the baseline watermark coverage accuracy.
7. The deep learning model watermark protection method according to claim 1, characterized in that, The post-processing operations include at least one of fine-tuning, pruning, quantization, knowledge distillation, and low-rank decomposition.
8. A watermark protection device for a deep learning model, characterized in that, include: The post-processing module is used to perform at least one post-processing operation on the pre-trained deep learning model to obtain different versions of the deep learning model, wherein a watermark is added to the pre-trained deep learning model. The verification and evaluation module is used to perform watermark verification and performance evaluation on the different versions of the deep learning model to obtain the original task accuracy, enhanced trigger set recognition accuracy and watermark coverage accuracy of the different versions of the deep learning model. The normalization module is used to normalize the original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy of the different versions of the deep learning model, using the unprocessed deep learning model as a baseline, to obtain the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness metric calculation module is used to calculate the comprehensive robustness metric based on the normalized original task accuracy, enhanced trigger set recognition accuracy, and watermark coverage accuracy. The comprehensive robustness measurement comparison module is used to arbitrate based on the level of the comprehensive robustness measurement when the comprehensive robustness measurement is greater than or equal to a preset judgment threshold.
9. An electronic terminal, characterized in that, It includes a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, it performs the steps of the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.