Neural Network Mimicry via Random Probing for Privacy
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural networks (NNs) require access to the original training dataset to be replicated or modified, which is often not feasible due to data privacy and security concerns. Additionally, conventional NNs need to be retrained from scratch to incorporate new data or remove old data, making the process time-consuming.
Innovation Solution
A method is developed to train a new neural network (student model) to mimic a pre-trained target neural network (mentor model) without access to the target model or its original training dataset. This is achieved by probing the target model with random input data, generating a random probe training dataset, and using it to train the new model. The new model can then replicate the behavior of the target model, allowing for modifications such as adding new data or removing old data without accessing the original training dataset.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the original training dataset is kept secret for data privacy and security reasons, then data security is improved, but the ability to replicate or modify the neural network is worsened
Solution Approach 1:
The patent introduces an intermediary process (random probing) that allows information to be transferred from the target neural network to the student neural network without directly exposing the training dataset. The random probes act as a mediator that extracts necessary functional information while preserving data security.
Solution Approach 2:
The patent creates a copy (student neural network) that mimics the behavior of the original network (target neural network) without requiring access to the original training data. The copying is achieved through functional equivalence rather than direct data replication.
2Adaptability or versatility
If the neural network is retrained from scratch to incorporate new data or remove old data, then the neural network can be modified, but the time and resources required are increased
Solution Approach 1:
The patent performs preliminary actions by collecting random probe responses in advance to create a training dataset for the student neural network. This preliminary data collection enables future modifications without requiring complete retraining from scratch.
Solution Approach 2:
Instead of performing the complete action of retraining the entire neural network from scratch, the patent uses a partial action approach by training only the student neural network on selected random probes, which is less time-consuming than full retraining.
3Manufacturing precision
If billions of calculations are performed to train an accurate neural network, then the accuracy is improved, but the training time and computational resources are increased
Solution Approach 1:
The patent extracts a subset of useful information from the target neural network through random probing, rather than performing complete retraining. This extraction approach obtains sufficient accuracy information with fewer calculations.
Solution Approach 2:
The patent uses partial action by training the student neural network on a selected subset of random probes rather than performing exhaustive training on all possible data, achieving acceptable accuracy with reduced computational effort.
Data Source
AI summary
A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.