Entropy-Based Reasoning Models for Broad Coverage With Smaller Size
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer-based reasoning systems face challenges in balancing the need for broad training data coverage with the requirement of reducing model size, leading to inefficiencies due to large memory usage and processing costs.
Innovation Solution
The use of entropy-based techniques to determine the informational value of data elements, where elements with high surprisal are retained and those with low surprisal are removed or flagged for reduced model size while maintaining broad coverage, through methods like calculating expected surprisal and information gain across supervised and unsupervised learning applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If broad training data coverage is obtained by collecting data elements from many points in time and many decisions, then the coverage and completeness of the model is improved, but the model size increases leading to higher computing resource usage
Solution Approach 1:
The patent extracts only the most informative data elements from the training set by calculating surprisal values. Data elements with high surprisal (those that provide the most new information) are retained, while redundant elements with low surprisal are removed. This extraction process reduces model size while preserving essential coverage.
Solution Approach 2:
The patent changes the parameter of data element selection from uniform inclusion to selective inclusion based on surprisal calculation. By introducing the surprisal parameter and setting thresholds, the system dynamically adjusts which data elements are included, transforming the model building process from static to adaptive.
2Adaptability or versatility
If the computer-based reasoning model size is increased to improve coverage, then the breadth of training data is improved, but the computing resources required to control the system increase
Solution Approach 1:
The patent extracts only the essential data elements needed for model operation by removing redundant elements. This extraction reduces the computational burden while maintaining the model's ability to handle diverse situations, thereby reducing energy consumption without sacrificing coverage.
Solution Approach 2:
The patent discards redundant data elements that do not contribute significantly to model performance. By calculating surprisal values and removing low-surprisal elements, the system recovers computing resources that would otherwise be wasted on processing unnecessary data, while retaining the essential information needed for broad coverage.
3Adaptability or versatility
If the model includes more features and data elements to improve coverage, then the comprehensiveness is improved, but inefficiencies arise due to improper parameters and excessive size
Solution Approach 1:
The patent introduces surprisal as a new parameter for evaluating data element quality. By changing from uniform parameter treatment to parameter-based selection, the system identifies and retains only the most informative features, improving both coverage and efficiency simultaneously.
Solution Approach 2:
The patent applies different quality standards to different data elements based on their individual surprisal values. Rather than treating all features equally, the system selectively retains high-quality (high surprisal) features and removes low-quality (low surprisal) ones, creating a model with optimized local quality that improves overall efficiency.
Data Source
AI summary
Techniques are provided herein for creating well-balanced computer-based reasoning systems and using those to control systems. The techniques include receiving a request to determine whether to include one or more particular data elements in a computer-based reasoning model and determining two probability density or mass functions (“PDMFs”), one for the data set including the one or more particular data elements, once for the data set excluding it. Surprisal is determined based on those two PDMFs, and inclusion in the computer-based reasoning model is determined based on surprisal. A system is later controlled using the computer-based reasoning model.