Trained Model Metadata Generation for Input Conformance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Trained models, such as neural networks, often fail to maintain consistent accuracy when applied to input data that does not conform to the characteristics of the training data, leading to 'out-of-spec' usage, which can result in inaccurate predictions or classifications.
Innovation Solution
A system and method that generate metadata encoding numerical characteristics of the training data, allowing the trained model to determine conformance of input data, warning or declining application when out-of-spec usage is detected by comparing the input data's intermediate output against the encoded metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a trained model is applied to input data that does not conform to training data characteristics, then the model may be more versatile, but prediction accuracy deteriorates
Solution Approach 1:
The system performs preliminary characterization of the training data by applying the trained model to the training data and determining numerical characteristics before actual use. This preliminary action creates a reference profile that enables later verification of input data conformance, preventing accuracy degradation from out-of-spec usage
Solution Approach 2:
The system implements feedback by comparing numerical characteristics of input data against stored metadata from training data. This feedback mechanism identifies when input data deviates from training data characteristics, allowing the system to warn or decline processing to maintain prediction accuracy
2Reliability
If metadata encoding and conformance checking is implemented, then prediction reliability is improved, but system complexity increases
Solution Approach 1:
The system extracts only the essential numerical characteristics from the training data and stores them as compact metadata, rather than storing the entire training dataset or complex models. This extraction approach maintains reliability verification capability while minimizing the added complexity and storage requirements
Solution Approach 2:
The system transforms the complex problem of ensuring model conformance into a simpler parameter comparison task by converting training data characteristics into numerical parameters stored in metadata. This parameter transformation simplifies the verification process while maintaining reliability
Data Source
Figure 1~2
Figure 3
Figure 4~6
AI summary
The invention relates to a trained model, such as a trained neural network, which is trained on training data. System and computer-implemented methods are provided for generating metadata which encodes a numerical characteristic of the training data of the trained model, and for using the metadata to determine conformance of input data of the trained model to the numerical characteristics of the training data. If the input data does not conform to the numerical characteristics, the use of the trained model on the input data may be considered out-of-specification ('out-of-spec'). Accordingly, a system applying the trained model to the input data may, for example, warn a user of the non-conformance, or may decline to apply the trained model to the input data, etc.