Watermarking Semantic Segmentation Models Against Parameter Refinement
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for watermarking machine learning-based semantic image segmentation models are vulnerable to parameter refinement attacks, allowing malicious competitors to erase the watermark, thus necessitating improved robustness to protect the legitimate ownership.
Innovation Solution
A method involving an initial learning phase followed by a complementary machine learning phase with iterative generation of modified watermarks using gradient descent or adversarial attacks to ensure the watermark remains intact even after parameter refinement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a semantic segmentation model is watermarked using traditional methods, then the model can be associated with a legitimate owner, but the watermark can be erased by parameter refinement attacks
Solution Approach 1:
The patent applies preliminary action by embedding the watermark during the initial model training phase rather than attempting to add it later. The watermark is integrated into the model parameters through the loss function, ensuring it becomes an inherent part of the model structure that cannot be easily removed by subsequent parameter refinement attacks.
Solution Approach 2:
The patent utilizes parameter changes by modifying the loss function to include a watermark component. This changes the optimization parameters during training, forcing the model to learn both the segmentation task and the watermark embedding simultaneously. The watermark is encoded in the parameter space rather than being a separate layer, making it resistant to refinement attacks.
2Measurement precision
If the model parameters are refined through fine-tuning, then the model performance can be improved, but the watermark is lost
Solution Approach 1:
The patent merges the watermark embedding function with the segmentation loss function into a single unified loss function. This combining ensures that both the segmentation accuracy and watermark integrity are optimized simultaneously during fine-tuning, preventing the loss of watermark information when model parameters are refined.
Solution Approach 2:
The patent implements feedback by continuously monitoring both segmentation performance and watermark integrity during the fine-tuning process. The unified loss function provides feedback signals that guide parameter updates to maintain both model accuracy and watermark robustness, preventing the erosion of watermark information during optimization.
3Reliability
If a large amount of training data is used, then the model learning quality is improved, but the training time and computational resources increase
Solution Approach 1:
The patent applies partial action by using a relatively small dataset specifically curated for watermark training rather than requiring massive datasets. The watermark loss function is designed to work effectively with limited data, focusing computational resources on embedding the watermark robustly rather than achieving perfect segmentation performance on diverse data.
Solution Approach 2:
The patent changes the parameter optimization focus by prioritizing watermark embedding parameters over segmentation performance parameters during the watermark training phase. This parameter prioritization allows the model to learn the watermark effectively with less data, reducing training time while maintaining sufficient model quality.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
This method comprises an initial machine learning phase (40) making it possible to obtain, from a digital input image comprising an inlay of tattoo thumbnail(s), a predetermined tattoo output image associated with the legitimate owner of the semantic segmentation model, then steps of: - applying a complementary machine learning phase (42) of at least part of the parameters of the semantic segmentation model to obtain at least one refined semantic segmentation model;- iterative generation (44) of at least one modified tattoo thumbnail, using a semantic segmentation model chosen from the initial semantic segmentation model and the refined semantic segmentation models, by machine learning using said tattoo output image as a target, the embedding of the modified tattoo thumbnail in an input image of the chosen semantic segmentation model, providing as output an output image similar to the tattoo output image.;