Embedding-Based Feedback Filtering for ML Label Quality

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Existing techniques for providing manual feedback on data labels for machine learning models are costly and prone to errors, especially when domains share common characteristics, leading to increased costs and potential deterioration of model quality.

Innovation Solution

A method and system for filtering feedback on data labels by classifying feedback as high-quality or low-quality based on proximity to anchor embeddings in embedding spaces, allowing only high-quality feedback to update prediction labels and discarding low-quality feedback.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If manual feedback is provided on data labels to improve labeling accuracy, then data labeling quality is improved, but costs and time consumption increase

Engineering Contradiction:

Improvedata labeling qualityVSAvoidtime consumption

Core Design Contradiction:

Measurement precisionVSLoss of time

Solution Approach 1:

The system enables self-service by allowing the machine learning model to automatically evaluate and filter feedback quality using embedding proximity measurements, eliminating the need for costly manual review of every feedback label while maintaining high data labeling quality

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent replaces the mechanical manual review process with an automated computational system that uses embedding spaces and proximity measurements to objectively assess feedback quality, substituting human time and effort with efficient algorithmic processing

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Measurement precision

If manual feedback is provided on data labels to improve labeling accuracy, then data labeling quality is improved, but costs increase

Engineering Contradiction:

Improvedata labeling qualityVSAvoidcosts

Core Design Contradiction:

Measurement precisionVSLoss of energy

Solution Approach 1:

The system performs self-service by automatically filtering feedback quality through computational embedding comparisons, eliminating the need for expensive manual review processes while maintaining high data labeling quality

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent substitutes expensive manual feedback review with an automated computational system that uses embedding proximity to objectively evaluate feedback quality, significantly reducing costs while preserving labeling accuracy

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

3Reliability

If user feedback is reviewed by another user to avoid errors, then feedback quality is improved, but more human resources are required increasing costs

Engineering Contradiction:

Improvefeedback qualityVSAvoidhuman resources

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent replaces the mechanical process of human feedback review with an automated computational system that uses embedding spaces and proximity measurements to objectively assess feedback quality, eliminating the need for additional human reviewers while maintaining or improving feedback reliability

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

Solution Approach 2:

The system introduces an intermediary computational mechanism (embedding proximity measurement) that mediates between raw user feedback and final label acceptance, providing an objective quality assessment without requiring additional human judgment

Inventive Principle:

Principle #24Intermediary (Mediator)

4Measurement precision

If manual feedback is provided for domain-specific data, then labeling accuracy is improved, but the same feedback cannot be reused for different domains increasing costs

Engineering Contradiction:

Improvelabeling accuracyVSAvoidcross-domain applicability

Core Design Contradiction:

Measurement precisionVSAdaptability or versatility

Solution Approach 1:

The patent implements universality by creating a domain-agnostic feedback filtering mechanism based on embedding proximity that can be applied across different domains, allowing the same system to maintain labeling accuracy while adapting to various data types and domains without requiring domain-specific customization

Inventive Principle:

Principle #6Universality (Multi-functionality)

Data Source

PatentUS12566822B1Method and a system for feedback filtering in a machine learning model

Publication Date: 2026.03.03 HONEYWELL INTERNATIONAL INC

AI summary

The present disclosure relates to a method and a system for filtering feedback on data labels in a machine learning model. The method comprises receiving a new data object for one or more embedding spaces, the new data object having an embedding and each embedding space includes one or more clusters of data objects having a predefined label and an anchor embedding, generating a prediction label for the new data object, receiving a feedback label based on the prediction label, identifying, based on the feedback label, the new data object with at least one cluster of the data objects in the embedding space, determining, based on a pre-defined threshold, proximity of the embedding of the new data object with the anchor embedding of the at least one cluster, and classifying the feedback label as any one of high-quality feedback or low-quality feedback based on the proximity, wherein when the feedback label is classified as high-quality feedback, updating the prediction label with the feedback label, and when the feedback label is classified as low-quality feedback, discarding the feedback label.