Cascaded Binary Classification With Layer Uncertainty Metrics

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Machine learning models lack insight into the accuracy of their classifications, as they do not express uncertainties generated by layer-level classification models, leading to potential inaccuracies in final classifications due to the inclusion of unreliable models.

Innovation Solution

A cascaded binary classification system that generates uncertainty metrics at each layer, allowing a meta-model to adjust the final classification by disregarding classifications with high uncertainty and providing these metrics to users for review, thereby improving model reliability and performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If traditional machine learning models are used for classification, then the classification process is simple and fast, but the models lack insight into the accuracy of their classifications and do not express uncertainties

Engineering Contradiction:

Improveclassification reliabilityVSAvoidmodel complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The model is segmented into multiple layer-level classification models (e.g., first layer, second layer, third layer) that process classifications in stages. Each layer generates its own uncertainty metric, allowing the system to track reliability at each processing stage while maintaining overall classification functionality.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

A meta-model is introduced as an intermediary component that receives classifications and uncertainty metrics from multiple layer-level models. The meta-model integrates these inputs to produce a final classification with an aggregate confidence metric, mediating between the individual layer models and the final output.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Measurement precision

If multiple layer level classification models are used to improve classification accuracy, then the reliability of classifications improves, but the complexity of the system increases

Engineering Contradiction:

Improveclassification precisionVSAvoidsystem complexity

Core Design Contradiction:

Measurement precisionVSDevice complexity

Solution Approach 1:

The classification system is divided into multiple layer-level models operating in sequence, with each layer contributing to the final classification decision. This segmentation allows precision to improve through multiple processing stages while structuring the complexity in a manageable, hierarchical manner.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

Uncertainty metrics from each layer-level model are fed back to the meta-model, which uses this information to adjust the final classification confidence. This feedback mechanism allows the system to leverage multiple models for improved precision while using the uncertainty information to manage complexity through intelligent integration.

Inventive Principle:

Principle #23Feedback

3Loss of information

If uncertainty metrics are generated at each layer level, then the confidence in final classification can be quantified, but the computational overhead increases

Engineering Contradiction:

Improveinformation lossVSAvoidcomputational energy

Core Design Contradiction:

Loss of informationVSUse of energy by moving object

Solution Approach 1:

The uncertainty metric generation is extracted as a specific function of each layer-level model, separate from the main classification logic. This allows the system to quantify confidence information without fundamentally changing the core classification algorithms, minimizing additional computational overhead while preserving information about classification reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

4Manufacturing precision

If the meta-model uses uncertainty metrics to adjust final classification, then the accuracy of final classification improves, but the processing time increases

Engineering Contradiction:

Improveclassification accuracyVSAvoidprocessing time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

Uncertainty metrics are generated at each layer level during the classification process itself, rather than as a separate post-processing step. This preliminary generation of uncertainty information allows the meta-model to use it efficiently for adjusting final classification accuracy without requiring additional processing time after the main classification is complete.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentUS11809976B1Machine learning model with layer level uncertainty metrics

Publication Date: 2023.11.07 INTUIT INC

AI summary

Systems and methods are disclosed for classifying objects by a machine learning (ML) model. The ML model includes one or more layer level classification models to generate classifications and uncertainty metrics in the classifications and a meta-model to generate a final classification and confidence based on the underlying classifications and uncertainty metrics. In some implementations, the ML model provides an object to be classified to one or more layer level classification models, and the layer level classification models generate a classification for the object and an uncertainty metric in the classification. The meta-model receives the classifications and uncertainty metrics from the one or more layer level classification models and generates the final classification and confidence in the final classification. The uncertainty metrics may also be output by the ML model or used to adjust the meta-model to improve the final classification and confidence.