Discriminator-Guided Prediction Model Compression for Consistent Outputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning prediction models experience differences in prediction results when computing resources and performance specifications change, leading to inconsistent performance between pre- and post-compression models.
Innovation Solution
An information processing method that trains a discriminating model to differentiate between prediction models, and then trains the second prediction model to match the output of the first model, using machine learning to reduce prediction result discrepancies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine learning processing settings are changed based on computing resources and performance specifications, then prediction performance can be maintained to a certain degree, but prediction results differ between the original model and the compressed model
Solution Approach 1:
A discriminating model is introduced as an intermediary to detect differences between prediction results from different models. This discriminator evaluates whether predictions from compressed models match the original model's predictions, providing feedback to guide the compression process while maintaining prediction consistency
Solution Approach 2:
The system implements a feedback mechanism where the discriminating model's evaluation of prediction differences is used to adjust and optimize the compressed model. The discriminator's output feeds back into the model compression process, enabling iterative improvement to maintain prediction result consistency
2Adaptability or versatility
If model compression is applied to adapt to limited computing resources, then the model can run on resource-constrained devices, but the prediction results become inconsistent with the original model
Solution Approach 1:
The discriminating model serves as a mediator between the original model and compressed model, detecting and evaluating prediction discrepancies. This intermediary enables the system to identify when compression causes unacceptable deviations while still allowing deployment on resource-constrained devices
Solution Approach 2:
The system adjusts compression parameters and settings based on feedback from the discriminating model to find an optimal balance between model size for deployment and prediction consistency. By changing compression parameters iteratively, the system maintains prediction accuracy while adapting to resource constraints
Data Source
Figure 1
Figure 2~3
Figure 4~5
AI summary
First data is input to a first model to obtain a first result (S101), the first data is input to a second model to obtain a second result (S102), an error between discriminating information about the first result input to a discriminating model and correct answer information indicating an output of the first model is obtained (S103), an error between discriminating information about the second result input to the discriminating model and correct answer information indicating an output of the second model is obtained (S104), the discriminating model is trained by machine learning to reduce the errors (S105), second data is input to the second model to obtain a third result (S106), an error between discriminating information about the third result input to the discriminating model and correct answer information indicating an output of the first model is obtained (S107), and the second model is trained by machine learning to reduce the error (S108).