Deep Learning Model Ownership Verification via Training Process Proof
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing model ownership verification algorithms fail to accurately and effectively verify ownership under destruction and forgery attacks, as they either impact model performance or are vulnerable to adversarial attacks.
Innovation Solution
A method using a model chain generated during deep learning model training as a proof of the training process, calculating indices such as accuracy monotonicity, parameter distance monotonicity, and initial parameter randomness to determine authenticity without embedding additional information, ensuring model intactness, robustness, and exclusiveness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If existing model ownership verification algorithms are used, then model ownership verification can be performed, but the verification accuracy and effectiveness deteriorate under destruction and forgery attacks
Solution Approach 1:
The patent segments the model training process into multiple epochs and extracts multiple features (accuracy monotonicity, parameter distance monotonicity, parameter distribution continuity, initial parameter distribution, initial parameter randomness, and model chain distance) from different stages. This segmentation allows verification of model ownership through multiple independent features, improving reliability under attacks by not relying on a single verification point
Solution Approach 2:
The patent performs preliminary actions by saving the model chain (intermediate model parameters) during the training process before any verification is needed. These pre-collected training process proofs are then used for verification, allowing the system to maintain high accuracy under attacks without requiring additional modifications during the verification phase
2Reliability
If model watermarking or fingerprint methods are used, then ownership verification can be implemented, but model performance deteriorates due to embedding additional information
Solution Approach 1:
The patent extracts ownership verification information from the natural training process itself rather than embedding additional watermarks or fingerprints into the model. By taking out and saving the model chain (intermediate parameters) during training, the system achieves ownership verification capability without adding any extra information that would degrade model performance
Solution Approach 2:
The training process naturally generates the verification proof (model chain) as a byproduct of normal training operations. The system uses this self-generated information for verification purposes, eliminating the need for separate watermarking or fingerprinting mechanisms that would compromise model performance
3Measurement precision
If model chain verification is performed, then ownership verification accuracy improves, but computational complexity increases due to multiple index calculations
Solution Approach 1:
The patent performs preliminary action by saving the model chain during training, which pre-collects all necessary intermediate parameters. This preliminary storage eliminates the need for re-computing training processes during verification, reducing computational complexity while maintaining high verification accuracy through multiple index calculations
Data Source
AI summary
A computer-implemented method for verifying ownership of a deep learning (DL) model includes: obtaining a Gaussian mixture distribution for initialization of a DL model to be verified, a model chain to be verified of the DL model, and a corresponding verification dataset, wherein the model chain is composed of model parameters of the DL model in each training epoch; calculating, based on the Gaussian mixture distribution, the model chain and the verification dataset, an accuracy monotonicity index, a parameter distance monotonicity index, a parameter distribution continuity index, an initial parameter distribution index, an initial parameter randomness index and a model chain distance index; and obtaining an ownership verification result by determining an authenticity of the model chain based on the accuracy monotonicity index, the parameter distance monotonicity index, the parameter distribution continuity index, the initial parameter distribution index, the initial parameter randomness index and the model chain distance index.