AI Model Information Erasure via Retraining and Tensorization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing artificial intelligence systems face challenges in efficiently removing specific information without affecting their performance, as the information is distributed across the system in a complex, intertwined manner.
Innovation Solution
A computer-implemented process involving retraining and compressing a computational model using a retraining module to produce uncorrelated outputs and a tensorization module to compress the model using tensor networks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If information is removed from AI systems by traditional methods, then data privacy concerns are addressed, but system performance is significantly affected
Solution Approach 1:
The patent segments the information removal process into two distinct phases: retraining to create uncorrelated representations and compression to remove redundant information. This segmentation allows each phase to address specific aspects of information removal while preserving overall system performance.
Solution Approach 2:
The patent changes the parameters of the computational model during retraining, specifically transforming the representation of input data to be uncorrelated with the original information. This parameter transformation enables information removal while maintaining the model's ability to process new data effectively.
2Loss of information
If computational model is retrained to erase information, then data privacy is improved, but training time and computational resources increase
Solution Approach 1:
The patent performs preliminary retraining to create uncorrelated representations before the compression phase. This preliminary action prepares the model in a state that facilitates more efficient compression, ultimately reducing the total time required compared to performing compression alone on the original model.
Solution Approach 2:
The patent maintains continuous useful action by ensuring that the retraining process creates representations that are both privacy-protecting and compression-friendly. The uncorrelated representations generated during retraining serve dual purposes: protecting privacy and enabling efficient subsequent compression.
3Volume of stationary object
If computational model is compressed using tensor networks, then model size is reduced, but compression complexity increases
Solution Approach 1:
The patent segments the compression task into manageable operations on tensor networks, applying compression to different parts of the computational model in a systematic manner. This segmentation reduces the apparent complexity by breaking down the overall compression task into smaller, more tractable sub-tasks.
Solution Approach 2:
The patent uses uncorrelated representations as an intermediary state between the original information-containing model and the final compressed model. This intermediary representation simplifies the compression process by removing redundant information before compression is applied, reducing the complexity of tensor network operations required.
4Adaptability or versatility
If information is distributed across AI system, then system capability is enhanced, but information removal becomes difficult
Solution Approach 1:
The patent replaces traditional mechanical approaches to information removal (such as direct deletion or masking) with a learned transformation approach using neural network retraining. This substitution enables information removal from distributed representations by learning to transform them into uncorrelated forms, maintaining system capability while achieving privacy goals.
Data Source
Figure 1
Figure 1A
Figure 1B
AI summary
A computer-implemented process and an information processing system are disclosed. The process includes receiving input for a computational model, retraining the model to result in an uncorrelated output, and compressing the model using a tensorization module. The system comprises a computational model that processes input data, a retraining module, and a tensorization module that compresses the computational model using mathematical structures. The computational model can be a layered model, a model for language processing, a binary classification model, or a prediction model. The mathematical structures can be tensor networks used to compress layers of the computational model.