Intermediate Representation Control for Valid Learned-Model Outputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manipulation range of intermediate representations in learned models like Transformer and AlphaFold2 is unclear, making it difficult to determine how to change them to obtain valid outputs without altering the model's parameters.
Innovation Solution
A control program evaluates the validity of current outputs using a first indicator correlated with the data distribution of learning data and adjusts the intermediate representation to maximize validity, ensuring the output aligns with the data distribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manipulation is performed on intermediate representation of a learned model, then the representation capability is utilized to generate varied outputs, but the manipulation range corresponding to valid outputs becomes unclear and reliability decreases
Solution Approach 1:
The patent introduces a feedback mechanism where the validity of output data is evaluated based on a first indicator correlated with the data distribution of learning data. This feedback loop allows the system to assess whether manipulated intermediate representations produce valid outputs and adjust the manipulation accordingly, resolving the contradiction between utilizing representation capability and maintaining output reliability
Solution Approach 2:
The patent changes the parameter being evaluated from direct output manipulation to intermediate representation manipulation, controlled by validity indicators. By adjusting the intermediate representation parameters within the valid range determined by the first indicator, the system maintains reliability while still utilizing the model's representation capability for generating varied outputs
2Adaptability or versatility
If the manipulation range of intermediate representation is expanded to generate diverse outputs, then versatility increases, but it becomes difficult to determine how to perform manipulation to obtain valid outputs
Solution Approach 1:
The patent introduces validity indicators (first indicator correlated with data distribution and second indicator for target output) as intermediaries between the intermediate representation and the final output. These indicators serve as mediators that guide the manipulation process, making it easier to determine valid manipulation ranges while maintaining output diversity
Solution Approach 2:
The patent replaces the mechanical trial-and-error approach of determining manipulation ranges with an information-based system using validity indicators. Instead of physically testing various manipulation ranges, the system uses the first indicator correlated with data distribution to computationally determine the valid manipulation range, reducing the difficulty of detection and measurement
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A control program for causing a computer to execute a process includes, when an intermediate representation of input data input to a learned model is changed, evaluating validity of current output data generated from the intermediate representation by the learned model based on a first indicator correlated with an existence probability of output data in a data distribution of learning data used for learning of the learned model, and changing the intermediate representation based on an evaluation result such that validity of output data generated by the learned model increases.